Don’t get me wrong – I love cucumber. I’ve been on projects where it has been used successfully and I’d partly attribute the success of those projects to the use of cucumber. It is a fantastic tool for BDD. When used right:
- its scenarios provide a definitive specification of app functionality
- its output provides one of the most useful metrics of progress: running passing tests
- it gets everyone talking the same language
- it provides a layer of abstraction between the required behavior and the app implementation (if you keep your steps declarative)
- it helps to keep the focus on what code really needs to be written
All of this from one tool! Pretty good, huh? Well, like anything good, it comes with costs – those’ll be covered in a later post.
Whether the costs are outweighed by the benefits that cucumber can provide is dependent on how closely the customers, devs and testers are working together. The more blurred the roles of dev and test are and the more involved the customer is, the more value you’ll get from using cucumber. The further the devs are from the testers and the less involved the customer is, the less value cucumber will give you.
There comes a point where the costs of using cucumber outweigh the benefits – and that point comes along pretty quick IMHO. Most of the projects that I’ve seen using cucumber really shouldn’t be. Yes, it’s the flavor of the month. Yes, it’s what all the cool kidz are using. But it’s not a panacea. Using cucumber will not shower your project with pink unicorns or make your team collaborative. Too often I’ve seen cucumber used for CVDD/RDD instead of BDD – almost always a bad move…
Having been on quite a few projects that have used cucumber, I’ve come up with an unscientific and clumsy acid test that will show you if you should stop using cucumber and move to something with lower maintenance costs (eg: rspec/testunit/whatever). Here, in all its glory, is The Chicken Test™:
Instead of your usual feature description, try putting the following at the top of your next feature file:
Feature: [whatever feature is being written] As a chicken I want to cross the road In order to get to the other side
Possible results of The Chicken Test:
- You get caught while typing - Pass. To have got caught this early you must be writing your scenarios collaboratively. Continue using cucumber.
- You manage to get it checked in - I have a bad feeling about this… Get people talking again.
- No one notices for a couple of days even though the scenarios are being run - Fail. If people aren’t even reading the scenarios, they’re not serving their purpose. Use rspec.
- No one notices for a couple of months – Epic fail. Wipe that smug look off your face – you’ve made your point. The devs don’t want you testers “in the way”, so you may as well make life easier for yourself – move to rspec.
All too often, projects I’ve seen fail the chicken test. Does yours?