Tag based logic in Cucumber

Sometimes cucumber’s Before hook just doesn’t cut it. Here’s a nice hack that allows you to perform some logic during execution of cucumber scenarios when a tag is first come across:

It’s a cucumber formatter that detects when a tag is first come across. When a new tag is found, the perform_logic_for method is called, passing the tag to it. Once you’re in the perform_logic_for method, you can do what you like. I’m doing stuff like checking to see if a directory matches the tag name; if I find one, I load the data into an environment (a great example of when the Before tag isn’t enough). You have free reign in here to do what you like with the tag.

To use it, save the file to features/support/tag_logic.rb, add your own tag logic, and then execute it with:

#cucumber -f pretty -f TestManagement::TagLogic -o /dev/null

I hope that helps!

Small print:
Use the Before hook instead of this if you possibly can.

Precision Failure

When tests fail it’s nice to know why. The more precise a failure message is and the less time required to investigate why the test failed, the better.

When trying to find out what broke the test, this…

Failure/Error: search_field.should_not be_visible
       expected visible? to return false, got true

…is infinitely preferable to this…

Failure/Error: search_field.visible?.should == false
       expected: false
            got: true (using ==)

The first failure tells you that the test expected the search field to be invisible, the second that it expected true to equal false – not very helpful. I’d rather have the first error message than the second, and unless you’re a masochist you probably would like the same (though having reviewed a lot of test code, I’m not so sure…).

Cucumber and rspec make “precision failure” easy through the use of matchers. Instead of checking whether “.visible?” returns true or false, you can use rspec matchers to write the following code:

search_field.should_not be_visible
...or...
search_field.should be_visible

…instead of this:

search_field.visible?.should == false
...or...
search_field.visible?.should == true

Your test code will be more understandable, and when tests fail you’ll have a high chance of knowing exactly what went wrong.

So, go and learn about RSpec Matchers!

Bewildr 0.1.13

[So much for the hope of doing "more frequent releases"...] It’s update time again. A few additions to the API, and a couple of changes. First, the additions:

  • Added a ‘wait_for’ alias to ‘wait_for_existence_of’ – much nicer
  • Added Bewildr::Application#process to allow access to the underlying windows process
  • Added a static ‘attach_to_process_id’ method on Bewildr::Application

Now for the API changes:

  • For combo boxes and list items, calling ‘selected’ will return a Bewildr::Element, not the element’s name. In previous versions, to get a list’s/combobox’s selected item name, you would write: ‘@my_list.selected’. From now on, it’s: ‘@my_list.selected.name’. Same for comboboxes.

Happy bewildring!

How to get the RSpec test result in the after block

—UPDATE—

The change I asked for made it into rspec so you can now call example.exception out of the box!

—/UPDATE—

RSpec allows you to run a block of code at the end of each test using the after(:each) method. A change is going to be included in a future (hopefully near future!) version of RSpec that allows you to know what exception the test failed with so that you can decide what to do based on the exception. You’ll be able to do that by asking the example for its exception:

I’ve been using a monkeypatch for a little while that allows me to know if a test passed or failed – I’ve wanted to write out a variable on test failure but until now rspec hasn’t exposed the test result. Here’s the patch:

If you place the above code in your project somewhere, you’ll be able to use it as shown in the following example of its use:

When the next version of RSpec exposes the exception that caused the test to fail, this patch will still work, so it’s fairly well future proof…

Custom RSpec ‘progress-with-names’ formatter

RSpec‘s progress formatter (the one that produces output like ......F..*...FF....) produces very concise output – which is usually all anybody wants. But there’s a problem with that: if your tests are being run from hudson/jenkins and you want to watch the progress go by on the console screen, no output is displayed… until the output generates a newline character which only happens when a test fails. Annoying.

In order to force each test result to be displayed immediately in the console window, the result needs to include a newline character after either the ‘.‘, ‘F‘, or the ‘*‘. But output like that would be ugly. Adding the test name to the output would make the output look less ridiculous and would also let us know the results of individual tests immediately instead of having to wait for all the tests to finish. So I wrote an rspec formatter to do just that – it’s basiclly a rip-off of the progress formatter. Here it is:

To use it, save it to a file called ‘progress_with_names.rb‘, put it in your working directory and invoke it as explained below…

Example output

The progress_with_names formatter produces output like the following:

nat$ rspec test_spec.rb -f ProgressWithNames
. This should pass
F This should fail
* This should be pending

…which is way more informative (and hudson/jenkins friendly) than the corresponding progress formatter output:

nat$ rspec test_spec.rb
.F*

How to use from rake

A very simple rspec rake task that uses the progress_with_names formatter:

RSpec::Core::RakeTask.new do |t|
t.pattern = '**/*_spec.rb'
t.rspec_opts = ["-r", "./progress_with_names.rb", "-f", "ProgressWithNames"]
end

Example RSpec project

I’ve included a very simple rspec project that demonstrates the use of the formatter. Download it here, and then run:

rake spec

…to see the output!

Hope that helps…

RSpec and ci_reporter

In order to use rspec within hudson, you need to use the ci_reporter gem. The gem extends rspec’s behaviour to include the junit-style output required by hudson. Annoyingly, the documentation for creating a rspec rake task that uses ci_reporter is a bit sparse and not particularly “googleable”, so I’m putting some example code here for posterity. The following is all that is required in a Rakefile for the most basic rspec task that integrates with hudson:


require 'rspec/core/rake_task'
require 'ci/reporter/rake/rspec'

RSpec::Core::RakeTask.new(:all => ["ci:setup:rspec"]) do |t|
  t.pattern = '**/*_spec.rb'
end

You can now run the rspec task named “all” within hudson and get pretty results, graphs, and all the other goodness that hudson derives from junit output files. To run the above task:

rake all

Hope that saves you some searching around!

Ruby’s each, select and reject methods

Often, when test automation people come over to ruby, they bring constructs from their previous language – “you can write fortran in any language” – missing out on the expressiveness that ruby can give you. A great example of this is in array manipulation. Some common scenarios:

  • you need to iterate over each element (eg: clicking each radio button on a page)
  • you need to select elements that match certain criteria (eg: getting the text value for every other row in a table)
  • you need to reject elements that match certain criteria (eg: if there is a disabled text field, ignore it)
  • you need to transform each element in a particular way (eg: you need to convert a list of names to upper case)

Ruby provides expressive and pretty ways of doing the above. What we’re going to do next is look at the sort of code that ruby-n00bs often write to deal with the above, then contrast it with ‘the ruby way’ of doing the same thing. Hopefully, you’ll agree that the ruby way is considerably cleaner, more expressive and cuts out lots of needless boilerplate code. So…

Iterating over each element in an array

In the old school world, the normal way to iterate over an array is to use a for loop. To figure out how many times to iterate you’d get the length of the array. The for loop gives you an index, which you’d then use to access the next element in the array. Here’s an example in ruby that prints off each element of an array:


a = [1, 2, 3, 4, 5]

for i in 0..(a.size - 1)
  puts a[i]
end

Not very expressive, is it. Now for the ruby equivalent:


a = [1, 2, 3, 4, 5]
a.each {|number| puts number}

Now, it’s fairly uncommon to see the above mistake, even in ruby-n00b code. Learning the ‘each’ method seems to be a rite-of-passage that almost everyone goes through.

Transforming each element of an array

So, as we mentioned, very few ruby programmers don’t know about or don’t use ‘each’. Annoyingly, it is often incorrectly used by n00bs to transform each element of an array. The following is an example where an array of lower case words is transformed into an array of upper case words:


lower_case = ["hi", "these", "are", "some", "words"]

upper_case = []
lower_case.each do |word|
  upper_case << word.upcase
end

puts upper_case.inspect

#=> ["HI", "THESE", "ARE", "SOME", "WORDS"]


Every element of the array is looped through (correct), the transformation is done (‘word.upcase’ – correct); the mistake comes when adding that element to a new array. Ruby has a method that does all this for you; it’s called ‘collect’.


lower_case = ["hi", "these", "are", "some", "words"]

upper_case = lower_case.collect { |word| word.upcase }

puts upper_case.inspect

#=> ["HI", "THESE", "ARE", "SOME", "WORDS"]


What collect does is iterate over the array, ‘collect’ the result of the block (in this case the changing to uppercase of the block argument), store the result in a new array. It’s much shorter, but the main thing is that it’s more expressive. Here’s an example of where you could use it. Say you had a class that represented a page that you’re testing, and say that it contains a method that returns the text of every link on the page. Here’s the old school way:


class MyPage
  def links_text
    text_array = []
    @browser.links(:xpath, "//a").each do |link|
      text_array << link.text
    end
    text_array
  end
end

If you change it to use ‘collect’ instead of ‘each’, you’ll have the following instead:


class MyPage
  def links_text
    @browser.links(:xpath, "//a").collect {|link| link.text}
  end
end

Much nicer! Note that you can use ‘map’ instead of ‘collect’ if you like – one is an alias for the other.

Selecting only elements that meet some criteria

Another case of ‘each’ misuse. . . It’s a common scenario to want to select only certain items from an array – specifically elements that meet certain criteria. An example: given an array containing the numbers 1 to 10, I want to get all the even numbers. Well, one way to describe that criteria is:
element % 2 == 0
If, when the element is divided by 2, there is no remainder; element is an even number. So now, lets look at a ruby-n00b way of getting those elements:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
even_numbers = []

all_numbers.each do |number|
  even_numbers << number if number%2 == 0
end

#=> [2, 4, 6, 8, 10]


What’s happening? We’re looping through ‘each’ element, and performing our check. ‘If’ the element meet the criteria, add it to an array called ‘even_numbers’. It works, but it’s long winded, and not very expressive. Here’s the ruby way:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

even_numbers = all_numbers.select {|number| number%2 == 0}

#=> [2, 4, 6, 8, 10]


Much better. No need to create a new array before performing the check, no boilerplate add-the-element-to-the-new-array code, no dirty ‘if’s; just nice expressive code.

An example of when you’d want to use it? Say you have a table with a bunch of rows and you want a method to return only the rows that have a certain background color. Here’s n00b-style code:


class MyPage
  def blue_rows
    my_blue_rows = []
    @browser.row(:xpath, "//tr").each do |row|
      my_blue_rows << row if row.attribute("bgcolor") == "blue"
    end
    my_blue_rows
  end
end

Again, lots of fluff, hard to tell at first glance what’s going on. Here’s the same thing but this time using the ‘select’ method:


class MyPage
  def blue_rows
    @browser.row(:xpath, "//tr").select {|row| row.attribute("bgcolor") == "blue"}
  end
end

Much nicer. Expressive code. No guff.

Rejecting elements that meet certain criteria

Sometimes you want all the elements in an array apart from those which meet certain criteria. It’s almost identical to ‘select’, just. . . the opposite! Instead of selecting items which meet the supplied criteria, ‘reject’ will reject items which meet the criteria. This time, instead of selecting even numbers, we want to reject them, thus getting an array of odd numbers (a bit contrived, I know; we could just select the elements where element%2==1, but this is a tutorial).


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
odd_numbers = []

all_numbers.each do |number|
  odd_numbers << number unless number%2 == 0
end

#=> [1, 3, 5, 7, 9]


This time, we don’t add the number ‘if’ it meets the criterial; instead we add it ‘unless’ it meets the criteria. Again, it’s horrible code. The ruby way:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

odd_numbers = all_numbers.reject {|number| number%2 == 0}

#=> [1, 3, 5, 7, 9]


So when would you want to use it? With a similar example to what we have above, this time we want all the rows that haven’t got a red background. Old school code:


class MyPage
  def non_red_rows
    my_non_red_rows = []
    @browser.row(:xpath, "//tr").each do |row|
      my_non_red_rows << row unless row.attribute("bgcolor") == "red"
    end
    my_non_red_rows
  end
end

Long winded, ugly, hard to see what’s going on. Now, we’ll change it to use ruby’s ‘reject’ method:


class MyPage
  def non_red_rows
    @browser.row(:xpath, "//tr").reject {|row| row.attribute("bgcolor") == "red"}
  end
end

Hard to argue against, right? It’s short, expressive and to the point.

Summary: ruby provides nice methods for array manipulation. ‘Each’, ‘select’ and ‘reject’ are only a few of those methods, but they’re the most frequently used (or should be!). I hope this helps make your code shorter, more expressive and easier to maintain.