Why learn & understand when you can search?

A recently and widely (in my nerd circles) shared XKCD comic featured ineffective sorting techniques. The alt text (what you get when you hover over the comic) proposed an additional sort:

StackSort connects to StackOverflow, searches for 'sort a list', and downloads and runs code snippets until the list is sorted.

Internet nerdom being fairly OCD, someone (Gregory Koberger) of course went and implemented this, and it works!

As Gregory says, this is potentially a security nightmare waiting to happen, so it’s not clear that you should actually run his code. Being a brave and foolish (and somewhat careful) soul, however, I’ve fallen on this sword so that you don’t have to, and I can confirm that it works. After trying several dozen StackOverflow pages (most of which yielded either “Could not extract a function to run” or “Contained potentially bad code”), it stumbled across a working version of Quicksort in JavaScript.

Ta Da!

The success of this approach, however, does suggest interesting things about how people approach certain kinds of problem solving these days. Where “old people” like me would have bought a book to learn something new, my students are much more prone to assemble a patchwork understanding from tons of Googling. While I think this often leads to a quite fragile and incomplete understanding of the topic at hand, it is often sufficient to get them through what is assigned. In other words, they’re very skilled at answering the question that was posed, which was itself often somewhat shallow because that’s what we all have time for. So while faculty would like to think that we’re creating experts (whatever that means) in certain topics, we rarely have the time to give them assignments and tasks that require a deep level of expertise.

In the land and time of books, we might have believed that everyone read and understood the 8 assigned chapters, but the assignments almost certainly didn’t strictly require that. In fact I’m willing to bet a whole lot skimming actually went on that was not so different from the Googling that happens now. The difference is that when I skimmed a book, there was a reasonably coherent thread connecting the dots that were touched on. “Skimming” the Internet, however, is a much less coherent experience. The authors, examples, and assumptions aren’t the same from search result to search result. Worse (in computing) the versions aren’t necessarily the same, so we often end up with inconsistent and downright contradictory results! A few years ago I taught a class that used Python 3, and it turned out that almost all Googling pointed us at Python 2 examples and answers, which lead to all manner of confusion amongst my students.

So call me old-fashioned, but I still like a book, even if it’s an e-book, when I’m learning something conceptually new, as I do really appreciate a coherent voice and structure. If I need to remind myself of a library function or piece of syntax, it’s to the Googles. If I’m learning something that’s very similar to something I already know, then I’m happy with on-line docs if they’re well written. If, however, I’m expanding my horizons in more significant ways, a book is still the thing.

This entry was posted in Books, Computing, Education and tagged , , , , , , , , , , . Bookmark the permalink.

9 Responses to Why learn & understand when you can search?

  1. Scott lembcke says:

    I can’t count the number of times I’ve tried to argue the same point and most seem to be of the opinion that it’s pointless to understand things fully. Most of the same even seem to think its a complete waste of time.

  2. Phi says:

    I run into that with students occasionally as well and (as a teacher) do find it quite frustrating. I suspect part of it is the reward system and why they’re doing things. If people intrinsically want to learn and understand things, i.e., they’re innately curious and interested, then they’ll put in the effort. It’s clear, however, that a lot of people just want to do the minimum to get by, and for them some successful searching may be/appear sufficient.

    I suspect it’s true that you’ll never do interesting science or make a significant contribution to the (software) world without a deep understanding of something, but that makes assumptions about other people’s goals that pretty clearly don’t hold universally. My personal assumption, however, is that our students are going to do something interesting, even if they don’t share that thought :-). The trick then is to design experiences that at least reward some depth of understanding if not actually require it (since that’s so hard to actually measure).

  3. I agree with your point, but I think this is all symptomatic of a larger issue about how people handle problem anxiety. It will get easier and easier for students to cobble together half-understandings — and it will get harder and harder for us to “game” the assignments so that they learn in spite of themselves. It seems like an arms race we’re bound to lose if we focus on making the “sufficiency” level of assignments more nuanced. Instead, I wonder if we can’t lower their resistance to actually solving the problem.

    There’s a video floating around where John Cleese explains his “5 things” that make a creative person, and one of his points really resonated with me. He said that both more creative and less creative people when they encountered a problem have the same experience: They feel anxiety. The difference is that the less creative person cannot put up with the anxiety and so goes with the first solution that occurs to them, while the more creative person can sit and be uncomfortable for a while until they’ve explored the issue more thoroughly.

    I think the same is true with our students. It seems that their objective is to satisfy the assignment criteria, which will lead to shortcuts because it is impossible to align our assignment requirements perfectly with our pedagogical goals. In reality, I believe their goal is to resolve their anxiety as quickly as possible. So there are two parts of this problem: 1) The students can much, much more easily satisfy assignment goals with partial and shallow understandings because of Internet tools, and 2) Students are increasingly unwilling to sit with a problem for a while. So the gradient for their choice is clear.

    When I have students in my office struggling with a problem, I find more and more that they are *annoyed* if the solution to the problem is not immediately obvious to them. If they are at all stumped by it and cannot find the procedure for solving it in their notes or on the Internet, then I’ve done something wrong as a teacher. I don’t believe that these students aren’t at all curious, I just think that they cannot “access” their curiosity because the anxiety of not understanding what to do is just too high for them.

    What typically distinguishes really exceptional students from the others is that the exceptional students actively *want* to be stumped. They have not only learned to deal with the discomfort of not quite getting something, but they actively court situations in which this discomfort is likely to occur. This is (I believe) not *just* because they are more curious, but *also* because they are not as affected by the anxiety of the problem. They’ve learned to be stumped for a while and be okay with that — that the satisfaction of finding a really good solution is *worth* the anxiety.

    So I think part of what we should be helping the students learn is how to be comfortable with problem anxiety and how to be dissatisfied with your *first* answer, etc. I’ve been trying to work this concept into my courses, as much as I am capable. I’ve been incorporating Michalewicz “Puzzle-Based Learning” concepts in my Quantitative Aspects of Modeling & Simulation class recently, for example. I also try to explicitly repeat these ideas in lecture, where appropriate.

    I don’t believe in one-size fits all, magic solutions, but there’s something to a kind of “puzzle mentality” that’s useful. After all, people who work sudoku, crossword puzzles, or tsumego, are willingly putting themselves in situations where there some cognitive friction. Perhaps the more and more people do such things, the less and less easily frustrated they will get.

    Sorry for the length of my response. This is just something I’ve been thinking about a lot for last couple of years, so your post tapped into that vein.

    • Phi says:

      No worries about the length – lots of really interesting ideas here.

      I do see the anxiety that you talk of, and I definitely see the difference between students who like being stumped vs. the ones that find that very stressful. I’m not sure that correlates completely with the question of how they respond to that stress, however. I’ve got very good students who are quite comfortable being stumped who still would almost never read a book as a way of becoming unstumped. They are curious, but their working assumption is still that some good or lucky searching can satisfy that curiosity and get them the answer/help they need.

      I definitely agree with the importance to helping them accept that being confused isn’t a Bad Thing. I think that one of the big challenges there is most of their models (teachers, textbooks) always succeed, usually in inordinately tidy, clean ways. I think that some of the best teaching can happen when I end up stumped in class. There’s a real razor edge there, though, and those days can also be some of the most confusing and unhelpful experiences for students. They’re also inherently uncomfortable for us as teachers (we, after all, are being paid to “be the experts” and “get it right”), so we tend to go to considerable lengths to avoid such events.

      Maybe solving puzzles is nice way to deal with that. If you have a large pool of puzzles to draw from (I’m using this collection of Knights and Knaves puzzles in my discrete math class this semester) you can easily create situations where no one (even the prof) knows the answer, but where (at least) the prof can almost certainly find the answer using the tools that you’re emphasizing in class.

      A piece I’m less clear on, though, is what we do about students who are simply uninterested in doing the puzzles. They’ll do the minimum assigned set, but they take no enjoyment in doing them and would never spend time work on additional puzzles “just for fun”. I filled legal pad after legal pad with puzzle solutions starting back in grade school, but that wasn’t “normal” then and it’s equivalent is hardly “normal” now.

      Thanks again for the thoughtful response!

  4. sysadmin1138 says:

    I’ve been giving serious consideration to writing some lessons at Ops School (https://ops-school.readthedocs.org/en/latest/) recently, which has me considering just these topics. This is a curriculum aimed squarely at systems administrators, which is where my job-title has called home for many years now.

    After talking with my peers over the years, one thing I’ve learned is that how we solve problems has radically changed. In the era before all the technical vendors had everything on their support sites, we used to have shelves full of manuals, and to solve problems we had to read the darned things or trust in our support contract and pay someone else to fix it (the manuals were frequently cheaper than the support contract, so that was preferred for financial reasons). Side-channels like ask on CompuServe or Usenet were handy as well, but only worked sometimes.

    These days it’s all about the Google skills. The support contracts are still there for commercial software, but for open-source stuff documentation may not exist (or is several versions out of date, since no one like writing doc) so we’re left with peer-support options like IRC, forums and mailing-lists. There is no longer an M to RTF, so we’re relying entirely on our research skills to find solutions to problems.

    And if you iterate your research in a topic area often enough, long enough, you being to develop some depth in it. The sysadmin space is chock full of people who learned their technical specialties (though not their systematic thinking) entirely through on-the-job training and running into and then solving problems. There is continual and heated debate over the desirability and effectiveness of certification programs like those offered by Cisco. Most of us learned OTJ, not through a formal process, and it’s building a formal process to cut down the time-to-effectiveness duration for OTJ-style that OpsSchool came into being.

    These are people who do what we do for the variety. The problem of the day is ever changing, and can cover a very wide range of topics. It’s hard to become a specialist in one area, since the demands of the job strongly prefer generalists who can think. Sure some of us are in rather narrow silos (Storage Engineer: thinking about very large scale Fibre Channel storage networks and how they go wrong) to extremely broad job title such as the one-person IT shop (needs to know everything from why Outlook don’t work that way, file formats Apple Pages can’t handle, how to make a PHP-based Apache server that resists hacking, and keep the printers running). These are people who don’t have TIME to read a book every time they get passed something new, so they build up a folklore over time instead.

    Which circles around to my opening point. I’m seriously considering writing a large hunk of lesson there, and I’m stumped on how I should pitch it.

    Should I build it for the impatient googlers, which is the majority of the people in-title already, and create a series of short articles on the topic?

    Should I build it for the people who are seeking depth, such as students and prospective career-changers, and go into a full multi-lesson deep dive?

    A hybrid approach suggests itself, which means building an interesting summary document and then hyperlinking the heck out of it into deep-dive topic areas. This is an entirely self-directed curriculum so I have to make it compelling all on its own, and short enough to survive the limited time-span people give things while parked in front of a glowing screen.

  5. Phi says:

    That is indeed a serious challenge. I tend to think the hybrid approach you proposed is likely to be your best bet. I think that one thing that’s important is to give people the ability to drill as deeply as makes sense for them in the moment; the structure of the steps at Arch Linux site is somewhat nice that way, although more successful in some places than others. Maybe what we want is some sort of combination of Wikipedia and StackOverflow?

    • sysadmin1138 says:

      A combination like that would be very powerful. A cross between step by step explanations and fairly detailed case-histories of specific problems experienced in that topic area as a way to show the practical applications. The sysadmin space has some resources out there like ServerFault which have usage licenses compatible with this kind of curriculum style and could be very useful.

      It’s also a good idea for my surly target market, since practical is how most of us learned how to do this stuff.

  6. I think the notion of “‘practical’ learning vs. ‘academic’ learning” is largely a false dichotomy. I’m pleased if students can improve critical thinking skills (and particularly when I can improve my own), and less concerned with whether they master a particular technical bit of material. To solve problems well, you have to have good critical thinking skills, wherever you acquired them and however you employ them. Lots of tools can help, but there’s no substitute for learning to think.

    I have absolutely nothing against using Internet tools to noodle things out (I do it myself). It’s not that students google their answer, it’s that they use that *in place of* trying to understand the material and finding a real solution to a presented problem. The difference between a non-academic industry setting and a purely academic classroom setting (which can also be practical) is that in the non-academic setting the real problem actually has to get solved, while in the academic setting students merely need to satisfy the assignment criteria.

    I don’t mind that people consult tools to help them (I want them to); I mind that their expectation is that this should be *sufficient* to solve any problem. I mind when students do not *think* about a problem. If you are good at your job and are an efficient, flexible problem solver, I guarantee that you also spend time thinking about the problem, whatever tools you may use. The trouble is that more and more students are expecting that they should not have to vex over problems at all … that the method to solve each and every problem should be apparent to them or easily found. This is a bad habit that will not serve them well in academia or industry.

    But I don’t think it’s because they are lazy or apathetic, or because Google and Wikipedia are evil. I think it’s (mostly) because they are inexperienced at dealing with the anxiety that comes along with not knowing how to solve a problem. They are so uncomfortable with it that they will do almost anything to get out from under that anxiety. You should see the lengths some students go to to knowingly cheat. Their efforts are often substantially more than it would have been to just do the assignment. Or students that are happy to be given a lot of mindless, tedious work instead of a smaller assignment that is more open-ended because there’s no anxiety in the tedium.

    I think most students today have been in an educational system that has always taught them, “Here is method A that applies to problems like X. Now solve problem X and Y (hint: Y is like X).” Instead, we should be asking, “How can you solve problem X?” Students might search, they might ask others, they might create their own solution or cobble together something … but, in the end, they will have to think about it. That’s all I really care about when all gets said and done.

    This appears to me to be a extremely practical position.

  7. Phi says:

    I guess I don’t find that my students are so adverse to challenge or difficult, open-ended problems, but I suspect that these things vary widely from place to place, and even time to time.

    One arena where I do think they often put in considerable effort and often struggle is in gaming. Sometimes this is essentially puzzle solving, and sometimes it’s mastering a specific physical skill (getting the timing right on a move, for example), but they’ll definitely put a lot of effort into these things, and must experience considerable anxiety in the process. I’m not sure what that tells us, though; at a minimum I’m sure they see the experience of gaming as wildly different from that of being a student (even if that’s not really sensible).

    It’s clear, though, that we both favor more open-ended exercises that stretch students, and I think for essentially similar reasons. Now if I could just figure out how to grade such things in a more timely manner :-)

Comments are closed.