RSpec and ci_reporter

In order to use rspec within hudson, you need to use the ci_reporter gem. The gem extends rspec’s behaviour to include the junit-style output required by hudson. Annoyingly, the documentation for creating a rspec rake task that uses ci_reporter is a bit sparse and not particularly “googleable”, so I’m putting some example code here for posterity. The following is all that is required in a Rakefile for the most basic rspec task that integrates with hudson:


require 'rspec/core/rake_task'
require 'ci/reporter/rake/rspec'

RSpec::Core::RakeTask.new(:all => ["ci:setup:rspec"]) do |t|
  t.pattern = '**/*_spec.rb'
end

You can now run the rspec task named “all” within hudson and get pretty results, graphs, and all the other goodness that hudson derives from junit output files. To run the above task:

rake all

Hope that saves you some searching around!

Cartoon Tester

I just stumbled across the Cartoon Tester site. Take a look!

I found “Is that a bug in a software testing site?” particularly pertinent… :P

Ruby’s each, select and reject methods

Often, when test automation people come over to ruby, they bring constructs from their previous language – “you can write fortran in any language” – missing out on the expressiveness that ruby can give you. A great example of this is in array manipulation. Some common scenarios:

  • you need to iterate over each element (eg: clicking each radio button on a page)
  • you need to select elements that match certain criteria (eg: getting the text value for every other row in a table)
  • you need to reject elements that match certain criteria (eg: if there is a disabled text field, ignore it)
  • you need to transform each element in a particular way (eg: you need to convert a list of names to upper case)

Ruby provides expressive and pretty ways of doing the above. What we’re going to do next is look at the sort of code that ruby-n00bs often write to deal with the above, then contrast it with ‘the ruby way’ of doing the same thing. Hopefully, you’ll agree that the ruby way is considerably cleaner, more expressive and cuts out lots of needless boilerplate code. So…

Iterating over each element in an array

In the old school world, the normal way to iterate over an array is to use a for loop. To figure out how many times to iterate you’d get the length of the array. The for loop gives you an index, which you’d then use to access the next element in the array. Here’s an example in ruby that prints off each element of an array:


a = [1, 2, 3, 4, 5]

for i in 0..(a.size - 1)
  puts a[i]
end

Not very expressive, is it. Now for the ruby equivalent:


a = [1, 2, 3, 4, 5]
a.each {|number| puts number}

Now, it’s fairly uncommon to see the above mistake, even in ruby-n00b code. Learning the ‘each’ method seems to be a rite-of-passage that almost everyone goes through.

Transforming each element of an array

So, as we mentioned, very few ruby programmers don’t know about or don’t use ‘each’. Annoyingly, it is often incorrectly used by n00bs to transform each element of an array. The following is an example where an array of lower case words is transformed into an array of upper case words:


lower_case = ["hi", "these", "are", "some", "words"]

upper_case = []
lower_case.each do |word|
  upper_case << word.upcase
end

puts upper_case.inspect

#=> ["HI", "THESE", "ARE", "SOME", "WORDS"]


Every element of the array is looped through (correct), the transformation is done (‘word.upcase’ – correct); the mistake comes when adding that element to a new array. Ruby has a method that does all this for you; it’s called ‘collect’.


lower_case = ["hi", "these", "are", "some", "words"]

upper_case = lower_case.collect { |word| word.upcase }

puts upper_case.inspect

#=> ["HI", "THESE", "ARE", "SOME", "WORDS"]


What collect does is iterate over the array, ‘collect’ the result of the block (in this case the changing to uppercase of the block argument), store the result in a new array. It’s much shorter, but the main thing is that it’s more expressive. Here’s an example of where you could use it. Say you had a class that represented a page that you’re testing, and say that it contains a method that returns the text of every link on the page. Here’s the old school way:


class MyPage
  def links_text
    text_array = []
    @browser.links(:xpath, "//a").each do |link|
      text_array << link.text
    end
    text_array
  end
end

If you change it to use ‘collect’ instead of ‘each’, you’ll have the following instead:


class MyPage
  def links_text
    @browser.links(:xpath, "//a").collect {|link| link.text}
  end
end

Much nicer! Note that you can use ‘map’ instead of ‘collect’ if you like – one is an alias for the other.

Selecting only elements that meet some criteria

Another case of ‘each’ misuse. . . It’s a common scenario to want to select only certain items from an array – specifically elements that meet certain criteria. An example: given an array containing the numbers 1 to 10, I want to get all the even numbers. Well, one way to describe that criteria is:
element % 2 == 0
If, when the element is divided by 2, there is no remainder; element is an even number. So now, lets look at a ruby-n00b way of getting those elements:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
even_numbers = []

all_numbers.each do |number|
  even_numbers << number if number%2 == 0
end

#=> [2, 4, 6, 8, 10]


What’s happening? We’re looping through ‘each’ element, and performing our check. ‘If’ the element meet the criteria, add it to an array called ‘even_numbers’. It works, but it’s long winded, and not very expressive. Here’s the ruby way:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

even_numbers = all_numbers.select {|number| number%2 == 0}

#=> [2, 4, 6, 8, 10]


Much better. No need to create a new array before performing the check, no boilerplate add-the-element-to-the-new-array code, no dirty ‘if’s; just nice expressive code.

An example of when you’d want to use it? Say you have a table with a bunch of rows and you want a method to return only the rows that have a certain background color. Here’s n00b-style code:


class MyPage
  def blue_rows
    my_blue_rows = []
    @browser.row(:xpath, "//tr").each do |row|
      my_blue_rows << row if row.attribute("bgcolor") == "blue"
    end
    my_blue_rows
  end
end

Again, lots of fluff, hard to tell at first glance what’s going on. Here’s the same thing but this time using the ‘select’ method:


class MyPage
  def blue_rows
    @browser.row(:xpath, "//tr").select {|row| row.attribute("bgcolor") == "blue"}
  end
end

Much nicer. Expressive code. No guff.

Rejecting elements that meet certain criteria

Sometimes you want all the elements in an array apart from those which meet certain criteria. It’s almost identical to ‘select’, just. . . the opposite! Instead of selecting items which meet the supplied criteria, ‘reject’ will reject items which meet the criteria. This time, instead of selecting even numbers, we want to reject them, thus getting an array of odd numbers (a bit contrived, I know; we could just select the elements where element%2==1, but this is a tutorial).


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
odd_numbers = []

all_numbers.each do |number|
  odd_numbers << number unless number%2 == 0
end

#=> [1, 3, 5, 7, 9]


This time, we don’t add the number ‘if’ it meets the criterial; instead we add it ‘unless’ it meets the criteria. Again, it’s horrible code. The ruby way:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

odd_numbers = all_numbers.reject {|number| number%2 == 0}

#=> [1, 3, 5, 7, 9]


So when would you want to use it? With a similar example to what we have above, this time we want all the rows that haven’t got a red background. Old school code:


class MyPage
  def non_red_rows
    my_non_red_rows = []
    @browser.row(:xpath, "//tr").each do |row|
      my_non_red_rows << row unless row.attribute("bgcolor") == "red"
    end
    my_non_red_rows
  end
end

Long winded, ugly, hard to see what’s going on. Now, we’ll change it to use ruby’s ‘reject’ method:


class MyPage
  def non_red_rows
    @browser.row(:xpath, "//tr").reject {|row| row.attribute("bgcolor") == "red"}
  end
end

Hard to argue against, right? It’s short, expressive and to the point.

Summary: ruby provides nice methods for array manipulation. ‘Each’, ‘select’ and ‘reject’ are only a few of those methods, but they’re the most frequently used (or should be!). I hope this helps make your code shorter, more expressive and easier to maintain.

Bewildr 0.1.8

This is a quick release just to add the visible? method to the Element class. It’s not very sophisticated at the moment as it just checks the IsOffscreen property of the underlying AutomationElement object – it’ll get more robust as time goes on.

gem update bewildr

Go! Go! Go!

Bewildr 0.1.7

Bewildr – the IronRuby based gem for automated testing of WPF apps – has been updated! We’re now up to v0.1.7. The main changes are:

  • Start exes with command line arguments (see last step of this file)
  • Navigate the object hierarchy using element.parent and element.children methods (see here)
  • Use “\n” in rich text boxes instead of the clumsy “{ENTER}” syntax (see here)

There have also been some changes in the background… elements now build themselves dynamically based on what patterns are supported by the underlying MS UI automation element. There’s still a bit of ugliness here, but it’s fast being stomped out.

gem update bewildr

What are you waiting for? Go get it!

Upgrading from RSpec 1 to RSpec 2

If you’re upgrading from rspec 1 to rspec 2, details of the differences are here:

http://github.com/rspec/rspec-core/blob/master/Upgrade.markdown

The specific changes you’ll need to make to your rake file can be found here:

http://pivotallabs.com/users/alex/blog/articles/1451-upgrading-your-rakefile-from-rspec-1-3-to-rspec-2

Ruby at ThoughtWorks

An insight into the use of Ruby at ThoughtWorks:

http://martinfowler.com/articles/rubyAtThoughtWorks.html

I’ve only just come across this article now – I’m not sure how it escaped my notice so far. Anyway, it has some great insights on the viability of ruby for serious development, the productivity gains that come from using the language, along with other aspects of developing in ruby, based on *hard data*. Give it a read.

Is Cucumber adding value to your project? The Chicken Test!

Don’t get me wrong – I love cucumber. I’ve been on projects where it has been used successfully and I’d partly attribute the success of those projects to the use of cucumber. It is a fantastic tool for BDD. When used right:

  • its scenarios provide a definitive specification of app functionality
  • its output provides one of the most useful metrics of progress: running passing tests
  • it gets everyone talking the same language
  • it provides a layer of abstraction between the required behavior and the app implementation (if you keep your steps declarative)
  • it helps to keep the focus on what code really needs to be written

All of this from one tool! Pretty good, huh? Well, like anything good, it comes with costs – those’ll be covered in a later post.

Whether the costs are outweighed by the benefits that cucumber can provide is dependent on how closely the customers, devs and testers are working together. The more blurred the roles of dev and test are and the more involved the customer is, the more value you’ll get from using cucumber. The further the devs are from the testers and the less involved the customer is, the less value cucumber will give you.

There comes a point where the costs of using cucumber outweigh the benefits – and that point comes along pretty quick IMHO. Most of the projects that I’ve seen using cucumber really shouldn’t be. Yes, it’s the flavor of the month. Yes, it’s what all the cool kidz are using. But it’s not a panacea. Using cucumber will not shower your project with pink unicorns or make your team collaborative. Too often I’ve seen cucumber used for CVDD/RDD instead of BDD – almost always a bad move…

Having been on quite a few projects that have used cucumber, I’ve come up with an unscientific and clumsy acid test that will show you if you should stop using cucumber and move to something with lower maintenance costs (eg: rspec/testunit/whatever). Here, in all its glory, is The Chicken Test™:

Instead of your usual feature description, try putting the following at the top of your next feature file:

Feature: [whatever feature is being written]
  As a chicken
  I want to cross the road
  In order to get to the other side

Possible results of The Chicken Test:

  • You get caught while typing - Pass. To have got caught this early you must be writing your scenarios collaboratively. Continue using cucumber.
  • You manage to get it checked in - I have a bad feeling about this… Get people talking again.
  • No one notices for a couple of days even though the scenarios are being run - Fail. If people aren’t even reading the scenarios, they’re not serving their purpose. Use rspec.
  • No one notices for a couple of months – Epic fail. Wipe that smug look off your face – you’ve made your point. The devs don’t want you testers “in the way”, so you may as well make life easier for yourself – move to rspec.

All too often, projects I’ve seen fail the chicken test. Does yours?