Cucumber, et al, do have a role, me thinks

While there may be issues with how acceptance testing/behavior driven development tools like Cucumber have developed and been used, I continue to find them very useful, especially in circumstances where it’s important to decouple the tests from the design of the solution.

Uncle Bob just posted a nice piece about how Cucumber and other Acceptance Testing and BDD (Behavior Driven Development) tools (e.g., Fit) are in important ways a failed experiment. A short (and probably not complete) version of the argument is something like:

  • A key goal of these tools was to allow the business side of the house to write acceptance tests, or at least be able to read acceptance tests and agree that they are reasonably accurate and complete.
  • That mostly didn’t happen.
  • The tools became increasingly programmer centric, making it even less likely to ever happen.
  • This ultimately leads to developers having to support two test streams, acceptance tests and unit tests.
  • Developers ain’t keen, so they tend to drop one.
  • The one they drop is usually unit tests, which Uncle Bob strongly believes is the wrong one.

I can totally follow this train and see where this can lead to Bad Things. That said, however, I must say that I’ve found tools like Cucumber, Aruba, and Geb very useful in my university classes.

In the simple case, tools like Cucumber and Aruba often great for writing executable specifications for class projects. This is somewhat similar to the goal of having the business team write the tests, but obviously I’m in the fortunate position of being a developer as well as the person writing the specs. It’s very nice for teaching, though, because I can specify what I want them to do without over-specifying how they do it. Unit tests will typically need to reflect all sorts of design decisions, and part of the point is often for them to work through the design, so it’s nice to have tests that both they and I can use to sanity check their solutions without tying us all down to a particular set of design choices.

In the more complex case, our students have often found these tools to be extremely useful for their own tests, especially when they’re struggling with a new problem domain and/or a new set of languages, tools, libraries, etc. In many ways a unit test requires the ability to envisions at least some of the structure of the solution; you have to be able to see where a unit will be. If they’re writing their first web app from scratch, using languages and libraries they’re just learning, it can be really hard to know what the first test might look like. So they tend not to write it, instead doing “spikes” to try to understand how the pieces work. These “spikes” go on and on, get bigger and bigger, and often become the project instead of being thrown away, and none of that development was supported by any sort of tests.

While it can be really difficult to envision units in a totally new space, hopefully they can understand use cases and user behavior. Tools like Geb allow them to specify what the tool should do without (yet) understanding what the insides will look like.

In many ways the two cases are quite similar: A.T./BDD lets us write tests without committing to design decisions yet. In the first I don’t want to dictate those decisions, and in the second they simply don’t yet know what those will look like yet, but either way we get to specify what is hopefully some useful and significant behavior right away without getting bogged down in those issues.

In a perfect world, they’d start writing unit tests for key pieces of internal logic as the design develops, especially for units whose behavior is complex enough that it’s hard to catch all the combinations through the user interface. That doesn’t always happen, but eventually the instability of the UI often drives teams to bring some of the testing from the UI into more stable areas of the code.

So, do I think every project should start with Cucumber tests? No, definitely not. But I do think they’re damn useful in certain circumstances and I’m quite glad to have them.

Related posts

Why learn & understand when you can search?

A recently and widely (in my nerd circles) shared XKCD comic featured ineffective sorting techniques. The alt text (what you get when you hover over the comic) proposed an additional sort:

StackSort connects to StackOverflow, searches for 'sort a list', and downloads and runs code snippets until the list is sorted.

Internet nerdom being fairly OCD, someone (Gregory Koberger) of course went and implemented this, and it works!

As Gregory says, this is potentially a security nightmare waiting to happen, so it’s not clear that you should actually run his code. Being a brave and foolish (and somewhat careful) soul, however, I’ve fallen on this sword so that you don’t have to, and I can confirm that it works. After trying several dozen StackOverflow pages (most of which yielded either “Could not extract a function to run” or “Contained potentially bad code”), it stumbled across a working version of Quicksort in JavaScript.

Ta Da!

The success of this approach, however, does suggest interesting things about how people approach certain kinds of problem solving these days. Where “old people” like me would have bought a book to learn something new, my students are much more prone to assemble a patchwork understanding from tons of Googling. While I think this often leads to a quite fragile and incomplete understanding of the topic at hand, it is often sufficient to get them through what is assigned. In other words, they’re very skilled at answering the question that was posed, which was itself often somewhat shallow because that’s what we all have time for. So while faculty would like to think that we’re creating experts (whatever that means) in certain topics, we rarely have the time to give them assignments and tasks that require a deep level of expertise.

In the land and time of books, we might have believed that everyone read and understood the 8 assigned chapters, but the assignments almost certainly didn’t strictly require that. In fact I’m willing to bet a whole lot skimming actually went on that was not so different from the Googling that happens now. The difference is that when I skimmed a book, there was a reasonably coherent thread connecting the dots that were touched on. “Skimming” the Internet, however, is a much less coherent experience. The authors, examples, and assumptions aren’t the same from search result to search result. Worse (in computing) the versions aren’t necessarily the same, so we often end up with inconsistent and downright contradictory results! A few years ago I taught a class that used Python 3, and it turned out that almost all Googling pointed us at Python 2 examples and answers, which lead to all manner of confusion amongst my students.

So call me old-fashioned, but I still like a book, even if it’s an e-book, when I’m learning something conceptually new, as I do really appreciate a coherent voice and structure. If I need to remind myself of a library function or piece of syntax, it’s to the Googles. If I’m learning something that’s very similar to something I already know, then I’m happy with on-line docs if they’re well written. If, however, I’m expanding my horizons in more significant ways, a book is still the thing.

Related posts