Cucumber, et al, do have a role, me thinks

While there may be issues with how acceptance testing/behavior driven development tools like Cucumber have developed and been used, I continue to find them very useful, especially in circumstances where it’s important to decouple the tests from the design of the solution.

Uncle Bob just posted a nice piece about how Cucumber and other Acceptance Testing and BDD (Behavior Driven Development) tools (e.g., Fit) are in important ways a failed experiment. A short (and probably not complete) version of the argument is something like:

  • A key goal of these tools was to allow the business side of the house to write acceptance tests, or at least be able to read acceptance tests and agree that they are reasonably accurate and complete.
  • That mostly didn’t happen.
  • The tools became increasingly programmer centric, making it even less likely to ever happen.
  • This ultimately leads to developers having to support two test streams, acceptance tests and unit tests.
  • Developers ain’t keen, so they tend to drop one.
  • The one they drop is usually unit tests, which Uncle Bob strongly believes is the wrong one.

I can totally follow this train and see where this can lead to Bad Things. That said, however, I must say that I’ve found tools like Cucumber, Aruba, and Geb very useful in my university classes.

In the simple case, tools like Cucumber and Aruba often great for writing executable specifications for class projects. This is somewhat similar to the goal of having the business team write the tests, but obviously I’m in the fortunate position of being a developer as well as the person writing the specs. It’s very nice for teaching, though, because I can specify what I want them to do without over-specifying how they do it. Unit tests will typically need to reflect all sorts of design decisions, and part of the point is often for them to work through the design, so it’s nice to have tests that both they and I can use to sanity check their solutions without tying us all down to a particular set of design choices.

In the more complex case, our students have often found these tools to be extremely useful for their own tests, especially when they’re struggling with a new problem domain and/or a new set of languages, tools, libraries, etc. In many ways a unit test requires the ability to envisions at least some of the structure of the solution; you have to be able to see where a unit will be. If they’re writing their first web app from scratch, using languages and libraries they’re just learning, it can be really hard to know what the first test might look like. So they tend not to write it, instead doing “spikes” to try to understand how the pieces work. These “spikes” go on and on, get bigger and bigger, and often become the project instead of being thrown away, and none of that development was supported by any sort of tests.

While it can be really difficult to envision units in a totally new space, hopefully they can understand use cases and user behavior. Tools like Geb allow them to specify what the tool should do without (yet) understanding what the insides will look like.

In many ways the two cases are quite similar: A.T./BDD lets us write tests without committing to design decisions yet. In the first I don’t want to dictate those decisions, and in the second they simply don’t yet know what those will look like yet, but either way we get to specify what is hopefully some useful and significant behavior right away without getting bogged down in those issues.

In a perfect world, they’d start writing unit tests for key pieces of internal logic as the design develops, especially for units whose behavior is complex enough that it’s hard to catch all the combinations through the user interface. That doesn’t always happen, but eventually the instability of the UI often drives teams to bring some of the testing from the UI into more stable areas of the code.

So, do I think every project should start with Cucumber tests? No, definitely not. But I do think they’re damn useful in certain circumstances and I’m quite glad to have them.

Related posts

Remembering Richard Crandall

This morning while frying an egg I picked up the latest copy of Reed Magazine, which had been sitting around the kitchen for a week or so, and was very sad to learn of the death in December of Richard Crandall, Reed alum and Physics prof for many years. I had a weird half relationship with Crandall in my time at Reed, which I remember with a combination of excitement and gratitude, also mixed with some regret.

Richard team taught the intro physics course I took my first year at Reed (where I learned that I was decent but not great at physics). On the strenuous recommendations of two good friends and physics majors (Peter Shirley and Neil Alexander) I later took the classroom part of second year physics from Richard. It was, as Peter had said, the best applied math course on campus, and Richard was a total genius with colored chalk, leaving the board covered with true works of art at the end of class. This was one of the most profound examples of the difficulty of capturing certain kinds of process in one’s notes, an issue I still struggle with as a teacher. Richard’s lectures were amazing to watch, but near impossible to wrestle into a notebook. I tried all sorts of things, including using colored pencils, to try to capture the glory of his board work, but it all failed. I suspect the only other person I had who’s lecturing was both as beautiful and as frustrating was Edsger Dijkstra.

More important than the classes I had with Richard, however, was his impact on me as a budding computer scientist. I’d taught myself BASIC in high school, but had never had any kind of instruction on programming or computing before my first semester at Reed, when we learned PASCAL in a series of labs in that intro physics course; labs which clearly owed their existence to Richard’s grasp of the growing importance of computation in math and science. Richard was friends with Steve Jobs, and it was through that connection and Jobs’ fond memories of his time as a quasi-student at Reed that Reed got an early Lisa, and then some of the first Macintosh computers. I remember playing with MacPaint on that clunky Lisa (I think with Peter Shirley again) and just being blown away – it was totally clear that we were seeing the future.

It was in significant part through Richard that I got to spend the summer of ’84 working at Reed designing fonts and programming for the earliest Macs. This was a very exciting opportunity, and I really felt like I was contributing to something. Probably my most notable achievement was the design of a bitmap math font (we’re just a little before PostScript here). Richard was very excited by the bitmap graphics on the Mac (taken for granted now, but still a novelty in the early 80’s), and as a scientist and mathematician he recognized the potential and value of not being tied to the ASCII character set that dominated computing at the time. So as a math nerd with computing interest and calligraphy experience (I was fortunate enough to take one of Robert Palladino’s last calligraphy courses at Reed in 1983), I was tasked with making a math font. This mostly consisted of designing a set of Greek characters, along with other common math symbols. It was hardly LaTeX (which would have been a nightmare to run on those boxes), but it was useful and widely used, and available for several years in big free font packs that you could get on a set of floppies from a fellow in (I think) Utah that had taken on the task of coordinating the distribution of free fonts. Those early bitmap fonts were, however, supplanted in short order by the much better PostScript fonts, and when I wrote my undergrad math thesis in MacWrite a year and a half later, I used the PostScript Symbol font instead of my own work. Sigh.

I also did some programming (I think I worked some on GriffinTerm, an early Mac terminal emulator), but my regret is that I didn’t make more of that opportunity. Richard was a strange guy who I at least found hard to read, but he was also clearly visionary and well connected. It’s hard to know what might have happened if I’d pushed myself more and made more of that opportunity. Sadly, I didn’t really understand the value of making things, and was mostly focused on “learning things” by taking courses and doing homework. I compare that to Thomas, who has done more extra-curricular “making” in his first year than I may have ever done at in my time at Reed, and it reminds me of why it’s so important to support our students at UMM in their “making”.

So farewell and thanks to Richard Crandall, a fellow who certainly made things, and who helped give me a great opportunity to make things.

Related posts