Tag Archives: cucumber

Is Cucumber adding value to your project? The Chicken Test!

Posted on by
2

Don’t get me wrong – I love cucumber. I’ve been on projects where it has been used successfully and I’d partly attribute the success of those projects to the use of cucumber. It is a fantastic tool for BDD. When used right:

  • its scenarios provide a definitive specification of app functionality
  • its output provides one of the most useful metrics of progress: running passing tests
  • it gets everyone talking the same language
  • it provides a layer of abstraction between the required behavior and the app implementation (if you keep your steps declarative)
  • it helps to keep the focus on what code really needs to be written

All of this from one tool! Pretty good, huh? Well, like anything good, it comes with costs – those’ll be covered in a later post.

Whether the costs are outweighed by the benefits that cucumber can provide is dependent on how closely the customers, devs and testers are working together. The more blurred the roles of dev and test are and the more involved the customer is, the more value you’ll get from using cucumber. The further the devs are from the testers and the less involved the customer is, the less value cucumber will give you.

There comes a point where the costs of using cucumber outweigh the benefits – and that point comes along pretty quick IMHO. Most of the projects that I’ve seen using cucumber really shouldn’t be. Yes, it’s the flavor of the month. Yes, it’s what all the cool kidz are using. But it’s not a panacea. Using cucumber will not shower your project with pink unicorns or make your team collaborative. Too often I’ve seen cucumber used for CVDD/RDD instead of BDD – almost always a bad move…

Having been on quite a few projects that have used cucumber, I’ve come up with an unscientific and clumsy acid test that will show you if you should stop using cucumber and move to something with lower maintenance costs (eg: rspec/testunit/whatever). Here, in all its glory, is The Chicken Test™:

Instead of your usual feature description, try putting the following at the top of your next feature file:

Feature: [whatever feature is being written]
  As a chicken
  I want to cross the road
  In order to get to the other side

Possible results of The Chicken Test:

  • You get caught while typing - Pass. To have got caught this early you must be writing your scenarios collaboratively. Continue using cucumber.
  • You manage to get it checked in - I have a bad feeling about this… Get people talking again.
  • No one notices for a couple of days even though the scenarios are being run - Fail. If people aren’t even reading the scenarios, they’re not serving their purpose. Use rspec.
  • No one notices for a couple of months – Epic fail. Wipe that smug look off your face – you’ve made your point. The devs don’t want you testers “in the way”, so you may as well make life easier for yourself – move to rspec.

All too often, projects I’ve seen fail the chicken test. Does yours?

IronRuby vs. Gherkin – a temporary fix

Posted on by
Reply

IronRuby doesn’t currently work with the latest versions of cucumber. This is a known bug and is a real pain. There is a work around, and it’s kinda rubbish: don’t use any versions of cucumber > 0.6.3. After that version, Gherkin became a .c extension – and IronRuby doesn’t like .c extensions.

Keith Burnell (the .Net Dev Dude) has blogged about getting Cucumber working with IronRuby – you may want to take a look if you’re getting stuck.

Netbeans ‘Cucumber Features’ plugin in Beta!

Posted on by
Reply

The best ruby IDE, Netbeans, now has a Cucumber plugin in beta! It’s the old “QuBiT” plugin that I’ve been using for a while though it looks like it’s been rebranded as the “Cucumber Features” plugin. Here it is:

As well as syntax highlighting and pretty icons, the new version has the following new (to me) features:

  • right-click “Run Feature” in the editor window and the file browser window
  • Better formatting (plugin now in line with latest cucumber changes)
  • Formatting of Examples tables (killer feature for me!)

Nice! Now, if only it would provide right-click-run-scenario…

Remove junk from IronRuby cucumber output

Posted on by
Reply

When running cucumber tests under IronRuby, your output will be full of junk like the following:


Feature: example feature

Scenario: a scenario←[90m     # features\testoutput.feature:3←[0m
←[32mGiven some test setup←[90m  # features/step_definitions/output_steps.rb:1←[0m←[0m
←[32mWhen I do something←[90m    # features/step_definitions/output_steps.rb:5←[0m←[0m
←[32mThen something happens←[90m # features/step_definitions/output_steps.rb:9←[0m←[0m

1 scenario (←[32m1 passed←[0m)
3 steps (←[32m3 passed←[0m)
0m0.160s

This is not great. All those “[32m” things make the output difficult to read and cause (me at least) some serious distraction. Thankfully, it’s easily fixed… if you add the –no-color option when you call the icucumber command, ie:

icucumber --no-color

…the output will change to be:


Feature: example feature

Scenario: a scenario     # features\testoutput.feature:3
Given some test setup  # features/step_definitions/output_steps.rb:1
When I do something    # features/step_definitions/output_steps.rb:5
Then something happens # features/step_definitions/output_steps.rb:9

1 scenario (1 passed)
3 steps (3 passed)
0m0.260s

Much better, and easily solved!

Updated script to list all cucumber step definitions

Posted on by
3

In a previous post I put up a small script that would dump out all step definitions available in a cucumber project. Here’s an updated version… it hasn’t changed much apart from the output is now a html table that for each step definition contains the regular expression, any regex modifiers, any parameters to the step definition and a link to the file that contains the definition.

Anyway, here’s the script:

step_definition_dir = "./features/steps"

f = File.new("output.htm", "w")

f << "<table><th>Regex</th><th>Modifiers</th><th>Step Definition Args</th><th>Source file</th>"

Dir.glob(File.join(step_definition_dir,'**/*.rb')).each do |step_file|
  File.new(step_file).read.each_line do |line|
    next unless line =~ /^\s*(?:Given|When|Then)\s+\//
    matches = /(?:Given|When|Then)\s*\/(.*)\/([imxo]*)\s*do\s*(?:$|\|(.*)\|)/.match(line).captures
    matches << step_file
    f << "<tr>"
    f << "<td>#{matches[0]}</td>"
    f << "<td>#{matches[1]}</td>"
    f << "<td>#{matches[2]}</td>"
    f << "<td><a href=\"#{matches[3]}\">#{matches[3]}</a></td>"
    f << "</tr>"
  end
end

f << "</table>"

—— UPDATE ——

Turns out that in the latest version of cucumber (0.6.1) there’s a new formatter called stepdefs which prints out the step regexs and the step definition file that they live in. It doesn’t report regex modifiers or the step arguments – apart from that it’s great. Here’s how to use it:

cucumber -d -f stepdefs

…though it only seems to work for steps called in an ordinary scenario, not a scenario outline.

How to list all cucumber step definitions

Posted on by
4

If you’re using cucumber you probably have step definitions split across several files, and you’re probably using a tool that doesn’t do a good job of listing the available step definitions (one of the biggest downsides of using cucumber).

What follows is a script that will list them all. It’s fairly rudimentary in that for each step definition it just lists the regex, any regex modifiers and the step definition arguments. It does everything I need at the moment so I haven’t developed it any further. Do with it what you will.


root_test_folder = "../my_project/features/support"

Dir.glob(File.join(root_test_folder,'**/*.rb')).each do |support_file|
  File.new(support_file).read.each_line do |line|
    next unless line =~ /^\s*(?:Given|When|Then)\s+\//
    matches = /(?:Given|When|Then)\s*\/(.*)\/([imxo]*)\s*do\s*(?:$|\|(.*)\|)/.match(line).captures
    matches[0] = Regexp.new(matches[0])
    puts matches.inspect
  end
end

Set the root_test_folder variable to the relevant location, run the file, and your console will be filled with what amounts to a step definition dictionary!

Cucumber step definition for debugging

Posted on by
Reply

Building on a previous post about debugging cucumber tests with ruby-debug, here’s a simple cucumber step definition you can use in your cucumber scenario in order to debug it:

Then /^I debug$/ do
  breakpoint
  0
end

Use it as follows:

Feature: Debug a cucumber scenario
  As a test writer
  In order to make my life easier
  I want to be able to debug

  Scenario: Calling the debugger in the middle of a test
    Given some condition
    Then I debug #this will call the debugger
    And I do something subsequent to debugging

Debugging cucumber tests with ruby-debug

Posted on by
1

Testing with cucumber blows test/unit out of the water, apart from one small aspect… I can’t use the Netbeans’ GUI debugger. The Netbeans debugger has been a joy to use, but sadly I have to say goodbye. So I’m left with using the traditional ruby debugger. It uses the same approach as gdb – console based debugging. Tedious and painful. Here’s how to use it…

Preparing your cucumber project

1) Install the gem with:
gem install ruby-debug

2) In your env.rb file, add the following at the top of the file:

require 'rubygems'
require 'ruby-debug'

3) If you want to be able to see stack traces, follow the above 2 lines with:
Debugger.start
(It might be worth wrapping that in some sort of debug-mode check as it does slow things down a little.)

Now you’re ready to debug like it’s 1979!

Using ruby-debug

To get started with ruby-debug, you need to set a breakpoint. How? Like this:


def some_method
  my_obj.thing = 4
  breakpoint #look! a breakpoint!
end

Breakpoints are set by writing breakpoint. Weird, huh? But there is a gotcha – breakpoint can’t be the last line of a method, or the debugger will stop at the calling method. So, you need to do something like this:


def some_method
  my_obj.thing = 4
  breakpoint
  0
end

…which is rubbish.

Once your breakpoint is set and you run the test, you’ll eventually see a command prompt – in Netbeans it’ll be in the output window. It’s time to debug.

Here are some of the commands that are available – they cover 95% of what I usually need from a debugger.

To end the debugging session and return control to the app:
quit

To find out where you are:
where
Note, this will print out a stack trace only if you’ve added the Debugger.start as specified above. If you haven’t, where will only print out the line number and file name of where execution has reached.

To see where you are in context (current line with a few lines before and after):
list

To print out the value of an individual variable:
p my_var

To print out all local and instance variables in the current stack frame:
info variables

To go to the next step (equivalent of ‘step into’):
step

To step over the next line:
next

To let the app run to the next breakpoint:
continue

To restart the app:
restart

You can see that ruby-debug gives you pretty much everything you need. The commands listed above hardly scratch the surface of what it can do. If you want to know more, here’s a good reference: http://bashdb.sourceforge.net/ruby-debug.html

BTW, here’s a post on how to wrap the breakpoint call in a cucumber step definition.