Tag Archives: cucumber

Tag based logic in Cucumber

Posted on by
Reply

Sometimes cucumber’s Before hook just doesn’t cut it. Here’s a nice hack that allows you to perform some logic during execution of cucumber scenarios when a tag is first come across:

It’s a cucumber formatter that detects when a tag is first come across. When a new tag is found, the perform_logic_for method is called, passing the tag to it. Once you’re in the perform_logic_for method, you can do what you like. I’m doing stuff like checking to see if a directory matches the tag name; if I find one, I load the data into an environment (a great example of when the Before tag isn’t enough). You have free reign in here to do what you like with the tag.

To use it, save the file to features/support/tag_logic.rb, add your own tag logic, and then execute it with:

#cucumber -f pretty -f TestManagement::TagLogic -o /dev/null

I hope that helps!

Small print:
Use the Before hook instead of this if you possibly can.

Precision Failure

Posted on by
Reply

When tests fail it’s nice to know why. The more precise a failure message is and the less time required to investigate why the test failed, the better.

When trying to find out what broke the test, this…

Failure/Error: search_field.should_not be_visible
       expected visible? to return false, got true

…is infinitely preferable to this…

Failure/Error: search_field.visible?.should == false
       expected: false
            got: true (using ==)

The first failure tells you that the test expected the search field to be invisible, the second that it expected true to equal false – not very helpful. I’d rather have the first error message than the second, and unless you’re a masochist you probably would like the same (though having reviewed a lot of test code, I’m not so sure…).

Cucumber and rspec make “precision failure” easy through the use of matchers. Instead of checking whether “.visible?” returns true or false, you can use rspec matchers to write the following code:

search_field.should_not be_visible
...or...
search_field.should be_visible

…instead of this:

search_field.visible?.should == false
...or...
search_field.visible?.should == true

Your test code will be more understandable, and when tests fail you’ll have a high chance of knowing exactly what went wrong.

So, go and learn about RSpec Matchers!

Cucumber formatter for unused steps

Posted on by
1

As time goes on in a project using cucumber you’ll end up with some steps which no longer get called, and you probably want to delete them. Annoyingly, there isn’t a built-in formatter you can use to find out where they are. Sure, the ‘usage’ formatter prints off unused steps, but it prints off all the other steps too! What’s needed is a formatter that will hunt down all steps that are not being used and then print off each step’s name and its location (file location and line number).

What follows is a very basic cucumber formatter that will do just that. To use it, copy the code into a file called ‘unused.rb‘ in your ‘features/support‘ folder and then use the formatter when you run cucumber:

cucumber -d -f Cucumber::Formatter::Unused

or

cucumber -d -f Unused

Is Cucumber adding value to your project? The Chicken Test!

Posted on by
2

Don’t get me wrong – I love cucumber. I’ve been on projects where it has been used successfully and I’d partly attribute the success of those projects to the use of cucumber. It is a fantastic tool for BDD. When used right:

  • its scenarios provide a definitive specification of app functionality
  • its output provides one of the most useful metrics of progress: running passing tests
  • it gets everyone talking the same language
  • it provides a layer of abstraction between the required behavior and the app implementation (if you keep your steps declarative)
  • it helps to keep the focus on what code really needs to be written

All of this from one tool! Pretty good, huh? Well, like anything good, it comes with costs – those’ll be covered in a later post.

Whether the costs are outweighed by the benefits that cucumber can provide is dependent on how closely the customers, devs and testers are working together. The more blurred the roles of dev and test are and the more involved the customer is, the more value you’ll get from using cucumber. The further the devs are from the testers and the less involved the customer is, the less value cucumber will give you.

There comes a point where the costs of using cucumber outweigh the benefits – and that point comes along pretty quick IMHO. Most of the projects that I’ve seen using cucumber really shouldn’t be. Yes, it’s the flavor of the month. Yes, it’s what all the cool kidz are using. But it’s not a panacea. Using cucumber will not shower your project with pink unicorns or make your team collaborative. Too often I’ve seen cucumber used for CVDD/RDD instead of BDD – almost always a bad move…

Having been on quite a few projects that have used cucumber, I’ve come up with an unscientific and clumsy acid test that will show you if you should stop using cucumber and move to something with lower maintenance costs (eg: rspec/testunit/whatever). Here, in all its glory, is The Chicken Test™:

Instead of your usual feature description, try putting the following at the top of your next feature file:

Feature: [whatever feature is being written]
  As a chicken
  I want to cross the road
  In order to get to the other side

Possible results of The Chicken Test:

  • You get caught while typing - Pass. To have got caught this early you must be writing your scenarios collaboratively. Continue using cucumber.
  • You manage to get it checked in - I have a bad feeling about this… Get people talking again.
  • No one notices for a couple of days even though the scenarios are being run - Fail. If people aren’t even reading the scenarios, they’re not serving their purpose. Use rspec.
  • No one notices for a couple of months – Epic fail. Wipe that smug look off your face – you’ve made your point. The devs don’t want you testers “in the way”, so you may as well make life easier for yourself – move to rspec.

All too often, projects I’ve seen fail the chicken test. Does yours?

IronRuby vs. Gherkin – a temporary fix

Posted on by
Reply

IronRuby doesn’t currently work with the latest versions of cucumber. This is a known bug and is a real pain. There is a work around, and it’s kinda rubbish: don’t use any versions of cucumber > 0.6.3. After that version, Gherkin became a .c extension – and IronRuby doesn’t like .c extensions.

Keith Burnell (the .Net Dev Dude) has blogged about getting Cucumber working with IronRuby – you may want to take a look if you’re getting stuck.

Netbeans ‘Cucumber Features’ plugin in Beta!

Posted on by
Reply

The best ruby IDE, Netbeans, now has a Cucumber plugin in beta! It’s the old “QuBiT” plugin that I’ve been using for a while though it looks like it’s been rebranded as the “Cucumber Features” plugin. Here it is:

As well as syntax highlighting and pretty icons, the new version has the following new (to me) features:

  • right-click “Run Feature” in the editor window and the file browser window
  • Better formatting (plugin now in line with latest cucumber changes)
  • Formatting of Examples tables (killer feature for me!)

Nice! Now, if only it would provide right-click-run-scenario…

Remove junk from IronRuby cucumber output

Posted on by
Reply

When running cucumber tests under IronRuby, your output will be full of junk like the following:


Feature: example feature

Scenario: a scenario←[90m     # features\testoutput.feature:3←[0m
←[32mGiven some test setup←[90m  # features/step_definitions/output_steps.rb:1←[0m←[0m
←[32mWhen I do something←[90m    # features/step_definitions/output_steps.rb:5←[0m←[0m
←[32mThen something happens←[90m # features/step_definitions/output_steps.rb:9←[0m←[0m

1 scenario (←[32m1 passed←[0m)
3 steps (←[32m3 passed←[0m)
0m0.160s

This is not great. All those “[32m” things make the output difficult to read and cause (me at least) some serious distraction. Thankfully, it’s easily fixed… if you add the –no-color option when you call the icucumber command, ie:

icucumber --no-color

…the output will change to be:


Feature: example feature

Scenario: a scenario     # features\testoutput.feature:3
Given some test setup  # features/step_definitions/output_steps.rb:1
When I do something    # features/step_definitions/output_steps.rb:5
Then something happens # features/step_definitions/output_steps.rb:9

1 scenario (1 passed)
3 steps (3 passed)
0m0.260s

Much better, and easily solved!

Updated script to list all cucumber step definitions

Posted on by
2

In a previous post I put up a small script that would dump out all step definitions available in a cucumber project. Here’s an updated version… it hasn’t changed much apart from the output is now a html table that for each step definition contains the regular expression, any regex modifiers, any parameters to the step definition and a link to the file that contains the definition.

Anyway, here’s the script:

step_definition_dir = "./features/steps"

f = File.new("output.htm", "w")

f << "<table><th>Regex</th><th>Modifiers</th><th>Step Definition Args</th><th>Source file</th>"

Dir.glob(File.join(step_definition_dir,'**/*.rb')).each do |step_file|
  File.new(step_file).read.each_line do |line|
    next unless line =~ /^\s*(?:Given|When|Then)\s+\//
    matches = /(?:Given|When|Then)\s*\/(.*)\/([imxo]*)\s*do\s*(?:$|\|(.*)\|)/.match(line).captures
    matches << step_file
    f << "<tr>"
    f << "<td>#{matches[0]}</td>"
    f << "<td>#{matches[1]}</td>"
    f << "<td>#{matches[2]}</td>"
    f << "<td><a href=\"#{matches[3]}\">#{matches[3]}</a></td>"
    f << "</tr>"
  end
end

f << "</table>"

—— UPDATE ——

Turns out that in the latest version of cucumber (0.6.1) there’s a new formatter called stepdefs which prints out the step regexs and the step definition file that they live in. It doesn’t report regex modifiers or the step arguments – apart from that it’s great. Here’s how to use it:

cucumber -d -f stepdefs

…though it only seems to work for steps called in an ordinary scenario, not a scenario outline.