Category Archives: Example Code

Tag based logic in Cucumber

Posted on by
Reply

Sometimes cucumber’s Before hook just doesn’t cut it. Here’s a nice hack that allows you to perform some logic during execution of cucumber scenarios when a tag is first come across:

It’s a cucumber formatter that detects when a tag is first come across. When a new tag is found, the perform_logic_for method is called, passing the tag to it. Once you’re in the perform_logic_for method, you can do what you like. I’m doing stuff like checking to see if a directory matches the tag name; if I find one, I load the data into an environment (a great example of when the Before tag isn’t enough). You have free reign in here to do what you like with the tag.

To use it, save the file to features/support/tag_logic.rb, add your own tag logic, and then execute it with:

#cucumber -f pretty -f TestManagement::TagLogic -o /dev/null

I hope that helps!

Small print:
Use the Before hook instead of this if you possibly can.

Step-by-step example of BDD’ing a WPF app with Cucumber, RSpec, IronRuby and Bewildr

Posted on by
4

In this tutorial, we’ll go through the BDD cycle step-by-step and develop a small Microsoft WPF app to solve a contrived problem. We’ll go through how to specify the desired features in cucumber, how to implement the steps in ironruby using bewildr to automate the UI, how to use rspec from ironruby to do the unit testing, we’ll write the C# code to pass the unit tests, and the WPF to create a UI to pass the cucumber scenarios! Get comfortable – this may take some time…

Things I’m going to assume

  • You know enough C# to write a simple WPF GUI
  • You know enough ruby, rspec and cucumber to automate some basic tests
  • You have a reasonable grasp of the concepts behind BDD and TDD
  • The following code will all be done in c:\wpfbdd

Ingredients

You will need the following installed…

With that, we are ready to BDD some WPF!

Introducing the Problem
Software exists to solve problems. What’s the problem that our contrived little app needs to solve? It’s time to jump into tutorial-scenario-land where everything is nice’n'easy and problems are well defined…

Farmer Giles has a flock of sheep that he moves from field to field. If any of the flock get left behind, they’ll get eaten by the Big Bad Wolf, which Farmer Giles would find sad and costly. What he wants is some way to keep track of his sheep as he moves them all from one field to the next to make sure that he hasn’t left any stragglers behind.

It’s time for a cucumber feature file
Hmmm… that’s enough material to start with. It’s time to begin writing our feature file. Here it is:

Save that to features\sheep_counter.feature. We have a conversation with Farmer Giles and he’s happy that we seem to have a general idea of what he wants. Next, we need to add some scenarios to our feature file. After a bit more chatting with Giles we decide that what he wants is a sheep counter, and these are the scenarios we come up with:

OK, now we’re getting somewhere. Giles is happy for us to build a sheep counter and we’ve got a few scenarios of how he’d like to use it… it’s time to start writing the step definitions!

Filling in the Step Definitions
Before we fill in the step definitions, we’ll quickly create the features\support\env.rb file with the following contents:

Now that bewildr has been added to the project, we can go ahead and create the features\step_definitions\sheep_counter_steps.rb file with the following contents:

If you’re struggling to keep up, those are the code blocks that will get called when you run the icucumber command (NB: that’s icucumber, not cucumber). Before you run icucumber, take the time to bask in the beauty that is the bewildr API displayed in the above steps… Enjoyed that? Good. Bask a bit more. And again. Hmmm… that’ll probably do. Back to work. Run the icucumber command, and take a look at the output:

Ugly. There’s a theme in the ugliness though. Here is the problem that’s causing the scenarios to fail:

Can't find: c:\wpfbdd\SheepCounter\SheepCounter\bin\Debug\SheepCounter.exe (RuntimeError)

That’s an error from bewildr telling us that it can’t find SheepCounter.exe. Well, that’s no big surprise given that it doesn’t exist! We now need to start writing some code to make these scenarios go green.

Our first bit of production ‘stuff’!
It’s time to break open Visual Studio and create a new project of type “WPF Application”. Name the project “SheepCounter” and save it to c:\wpfbdd. You’ll be greeted with a window something like this:

We’ll get to the code in a minute, just “Start Debugging” the new WPF project. That should be enough to compile it and get a blank WPF window displayed. Close the WPF window and go back to your command prompt and give icucumber another go. This time, you’ll see that the scenarios are opening the app, but then fail with ElementDoesntExist errors. Here’s the console output you should be getting now:

So, the scenarios are still all failing, but at least our new SheepCounter.exe is opening. But… it’s opening one instance per test and leaving them around. We need to change things so that the app gets closed at the end of the test. We need to update our features\support\env.rb file to include an After block:

If you run icucumber again you’ll notice that though the scenarios still fail, at least the SheepCounter app is being tidied up at the end of each test.

We’ll do a final bit of tidying up ourselves and rename our WPF window from “MainWindow” to “Sheep Counter” (it’s the window’s “Title” property that you need to change). When you’ve done that, run the WPF app again to verify that the window name has changed.

Now that the WPF app’s window has the correct title, the first of our step definitions should pass. Run icucumber again and you should see the following console output:

Better! The first step of each scenario is now passing.

This is where BDD/TDD really shine. You decide what you want your app to do, you express the desired behaviour in scenarios and then implement just enough production stuff to make the scenarios pass. When they do, you can stop coding in the knowledge that your software behaves the way you wanted it to!

First attempt at implementing the UI
From executing our scenarios we know that the app is starting up. But as soon as it starts, the scenarios fail because they can’t find the various UI elements that are mentioned in the step definitions. We need to go through the step definitions and add any UI elements they mention to the WPF UI.

As a minimum you need to add a label whose name is ‘sheep_count’ and a button whose name is ‘increment_sheep’. Here’s what I’ve got at this point (note, I’ve given the button some content and I’ve got an extra label with the contents “Sheep Count:”, just to make the app a litte more pretty and little less utilitarian):

Run the WPF app again (in debug mode) to make sure that all the elements are there (this will also compile the app so that the scenarios are dealing with the latest changes). Now kill the app and run icucumber again…

… the first two scenarios will run (and fail) but on the third test, it’ll get stuck in an infinite loop – the step is expecting the sheep count to increment every time the increment button is clicked, but we haven’t implemented that yet. Simply close the WPF window and the scenario will end. Your console output should now be something like:

Progress! We are now getting logic errors rather than UI errors, eg: the scenarios are complaining that they were expecting the label to contain “0″, but it got “Label” instead. When you get to this sort of point, it’s usually time to start writing some unit tests…

A Rake intermission…
Just before we get started with unit testing, we’ll add a Rakefile to our project. Rake is a great tool for managing project tasks, and since we’re going to be adding another one when we add unit tests, we may as well involve rake right now. Add a file called ‘Rakefile’ in your project root with the following contents:

All done. Now, instead of running icucumber to run our acceptance tests, we run irake cucumber. The task we’ll use to run our unit tests is irake spec.

Some unit tests…
It’s time to break out rspec and use it to design an object that can keep track of sheep for us – we of course don’t want this logic in the UI code, we need to separate it out. We need to create a c:\wpfbdd\spec directory to contain our unit tests. Once done, add a file named spec\counter_spec.rb. Now go ahead and run irake spec and you should get the following output:

It’s just telling us that we have no tests.

So what are we designing with our unit tests? The class that will manage the count of sheep as they move from field to field. Since we’re in tutorial-scenario-land, this is nice and easy. We need a class that basically keeps track of a count. It needs to start at 0, it needs to increment. That’s pretty much it for now. So we’ll start by adding a few pending tests to our spec file:

When you run that, you’ll see mentions of pending tests:

Next, we’ll write the first test:

…and when we run that with irake spec we get the following output:

So we’re finally at the stage where the test is failing for a proper reason – the dll that we want to test doesn’t yet exist. To get things working, we need to…

Add a Counter project in VS
We now need to fire up Visual Studio, and add a new ‘Class Library’ project to the solution; we’ll call the project ‘Counter’. The pre-defined class in the new Counter project is called ‘Class1.cs’. Rename that to ‘Counter.cs’. You should see something like this:

If you debug the project (to compile our new class) and then run the tests, you should now get the following obtuse error:

The key bit of info from that fail is the following: undefined method `count' for Counter.Counter:Counter::Counter. The class that we’re testing needs a count() method. So let’s add one:

Now, after debugging (to compile the new code), run the tests, and you should get:

That’s our first test passing! The C# code we added now returns the count, and starts the count at 0 when the Counter.cs class is initialized.

It’s time to move on to our next test. We now need to be able to increment the count, so we’ll add the following test code:

…and if we run that, we get the following:

…which tells us that we need an increment method. We’ll update the Counter.cs with the following:

…and after debugging the code, the tests will produce the following output:

See those 2 dots? Both our tests are passing!!! We now have the confidence that our Counter.cs class does enough to power the SheepCounter UI!

Wiring up the Counter class to the WPF UI
Before dealing with the UI, we need to add a project reference from the ‘SheepCounter’ WPF project to the ‘Counter’ Class Library project. Your Solution Explorer should look something like this:

Now that the UI project knows about the Counter class, we can update the ‘MainWindow.xaml.cs’ file with the following:

If you Debug the WPF app now, you should see that when it starts, the initial count is 0. If you run the cucumber acceptance tests (irake cucumber), you’ll find that the first test now passes! (Don’t forget about that last test that get stuck in an infinite loop – when it starts, kill it).

The test output will be something like:

The first test is now passing. We’ll move onto the second test; the one that tests the increment behaviour. To do that, first go to the WPF window layout, and double click the ‘Increment’ button you added to the UI. You should now see that the MainWindow.xaml.cs file has had an ‘increment_sheep_Click()’ method added to it. Save the project and then update the ‘MainWindow.xaml.cs’ file with the following:

Debug the project to compile it, and then run the acceptance tests (irake cucumber). You should see that all of the acceptance tests now pass!!!

The customer rejoices!!!
We find Farmer Giles, and show him our shiny new SheepCounter app. He’s very pleased, and tells us that he’ll go and use it for a few days and see how it works in the real world.

Feature request…
Farmer Giles comes back from the fields and tells us, “It’s great, this sheep counter, but there’s one annoying thing about it… Once I’ve moved all my sheep from one field to another, I have to restart the app to get the count back to 0. Could you add a feature that resets the count somehow?” “Hmmm… no problem… give us a few minutes…”

We need to add a new scenario to the feature file, update the UI to include a ‘Reset’ button, add a ‘reset()’ method to our Counter class and wire up the UI. Here goes:

Expanding the feature file
We add the following scenario to our existing ‘sheep_counter.feature’ file…

…and the following to the ‘sheep_counter_steps.rb’ file…

…and run our tests, we get the following:

Our old tests pass, but the new one fails telling us that there isn’t a reset button. So let’s add one!

Adding a Reset button
Back in VS, add a button named ‘reset’ to the UI:

After debugging the app, the new acceptance test should fail, but this time with a different problem:

… it got the wrong result. Though this time the button was found and clicked, the number didn’t get reset. We need to update our Counter class to be able to reset.

Updating the Counter class
Before updating the class, we need to add a unit test. Update the counter_spec.rb file with the following test:

Running the unit tests will give us the following result:

So there’s no ‘reset()’ method on our Counter class. Let’s add the following method to the Counter class…

…run the debugger and then run our unit tests again:

Good. All out unit tests are passing, including the new one for the reset() function.

Wiring up the reset button on the UI
The final thing we need to do is to wire up the Reset button on our WPF UI to the reset() method on the Counter class. To do this, double click on the reset button in the UI. You should see that a reset_Click() method has been added to the MainWindow.xaml.cs file. Save the project and then change the reset_Click() method to look like the following:

After debugging the update WPF app, run the acceptance tests again. You should see the following output:

Nice! Our new feature is working, and we haven’t broken anything – we know that because our other tests passed!

One happy customer!
And with that, we got Farmer Giles to spend a few days in the field with his updated SheepCounter app. He loved it – he could count his sheep as they moved from field to field, he could reset the count when he was done, and the big bad wolf died from hunger. Success!

Conclusion
I hope the above worked example, though contrived, has shown that BDD’ing a WPF app is not a big deal. The tools are easy-to-use, stable, popular and free (ironruby/cucumber/rspec/bewildr). The BDD process, though a source of contention around the interwebs, is at its core quite simple – it’s about conversations, specifying expected behaviour, writing tests first and short feedback loops. If you’re not doing it already, give BDD a go.

  • It takes a lot of the uncertainty out of coding – blind alleys are rarely ventured down because the customer is forced, by being involved in the development process – to think carefully about their requirements
  • You will make peace with your QA people – if you’re interested in the quality of your code then you’re on the same side
  • Your customer will love you – they’re getting what they need rather than what they thought they wanted
  • Maintenance will no longer be a terrifying ordeal – you’ll be happy to change code because your tests will tell you if you’ve broken anything
  • I could go on, but this isn’t the place really…

Anyway, I hope that helps!

Download the project
Click here to download the project.

Bewildr 0.1.13

Posted on by
Reply

[So much for the hope of doing "more frequent releases"...] It’s update time again. A few additions to the API, and a couple of changes. First, the additions:

  • Added a ‘wait_for’ alias to ‘wait_for_existence_of’ – much nicer
  • Added Bewildr::Application#process to allow access to the underlying windows process
  • Added a static ‘attach_to_process_id’ method on Bewildr::Application

Now for the API changes:

  • For combo boxes and list items, calling ‘selected’ will return a Bewildr::Element, not the element’s name. In previous versions, to get a list’s/combobox’s selected item name, you would write: ‘@my_list.selected’. From now on, it’s: ‘@my_list.selected.name’. Same for comboboxes.

Happy bewildring!

How to get the RSpec test result in the after block

Posted on by
Reply

—UPDATE—

The change I asked for made it into rspec so you can now call example.exception out of the box!

—/UPDATE—

RSpec allows you to run a block of code at the end of each test using the after(:each) method. A change is going to be included in a future (hopefully near future!) version of RSpec that allows you to know what exception the test failed with so that you can decide what to do based on the exception. You’ll be able to do that by asking the example for its exception:

I’ve been using a monkeypatch for a little while that allows me to know if a test passed or failed – I’ve wanted to write out a variable on test failure but until now rspec hasn’t exposed the test result. Here’s the patch:

If you place the above code in your project somewhere, you’ll be able to use it as shown in the following example of its use:

When the next version of RSpec exposes the exception that caused the test to fail, this patch will still work, so it’s fairly well future proof…

Custom RSpec ‘progress-with-names’ formatter

Posted on by
1

RSpec‘s progress formatter (the one that produces output like ......F..*...FF....) produces very concise output – which is usually all anybody wants. But there’s a problem with that: if your tests are being run from hudson/jenkins and you want to watch the progress go by on the console screen, no output is displayed… until the output generates a newline character which only happens when a test fails. Annoying.

In order to force each test result to be displayed immediately in the console window, the result needs to include a newline character after either the ‘.‘, ‘F‘, or the ‘*‘. But output like that would be ugly. Adding the test name to the output would make the output look less ridiculous and would also let us know the results of individual tests immediately instead of having to wait for all the tests to finish. So I wrote an rspec formatter to do just that – it’s basiclly a rip-off of the progress formatter. Here it is:

To use it, save it to a file called ‘progress_with_names.rb‘, put it in your working directory and invoke it as explained below…

Example output

The progress_with_names formatter produces output like the following:

nat$ rspec test_spec.rb -f ProgressWithNames
. This should pass
F This should fail
* This should be pending

…which is way more informative (and hudson/jenkins friendly) than the corresponding progress formatter output:

nat$ rspec test_spec.rb
.F*

How to use from rake

A very simple rspec rake task that uses the progress_with_names formatter:

RSpec::Core::RakeTask.new do |t|
t.pattern = '**/*_spec.rb'
t.rspec_opts = ["-r", "./progress_with_names.rb", "-f", "ProgressWithNames"]
end

Example RSpec project

I’ve included a very simple rspec project that demonstrates the use of the formatter. Download it here, and then run:

rake spec

…to see the output!

Hope that helps…

RSpec and ci_reporter

Posted on by
4

In order to use rspec within hudson, you need to use the ci_reporter gem. The gem extends rspec’s behaviour to include the junit-style output required by hudson. Annoyingly, the documentation for creating a rspec rake task that uses ci_reporter is a bit sparse and not particularly “googleable”, so I’m putting some example code here for posterity. The following is all that is required in a Rakefile for the most basic rspec task that integrates with hudson:


require 'rspec/core/rake_task'
require 'ci/reporter/rake/rspec'

RSpec::Core::RakeTask.new(:all => ["ci:setup:rspec"]) do |t|
  t.pattern = '**/*_spec.rb'
end

You can now run the rspec task named “all” within hudson and get pretty results, graphs, and all the other goodness that hudson derives from junit output files. To run the above task:

rake all

Hope that saves you some searching around!

Ruby’s each, select and reject methods

Posted on by
6

Often, when test automation people come over to ruby, they bring constructs from their previous language – “you can write fortran in any language” – missing out on the expressiveness that ruby can give you. A great example of this is in array manipulation. Some common scenarios:

  • you need to iterate over each element (eg: clicking each radio button on a page)
  • you need to select elements that match certain criteria (eg: getting the text value for every other row in a table)
  • you need to reject elements that match certain criteria (eg: if there is a disabled text field, ignore it)
  • you need to transform each element in a particular way (eg: you need to convert a list of names to upper case)

Ruby provides expressive and pretty ways of doing the above. What we’re going to do next is look at the sort of code that ruby-n00bs often write to deal with the above, then contrast it with ‘the ruby way’ of doing the same thing. Hopefully, you’ll agree that the ruby way is considerably cleaner, more expressive and cuts out lots of needless boilerplate code. So…

Iterating over each element in an array

In the old school world, the normal way to iterate over an array is to use a for loop. To figure out how many times to iterate you’d get the length of the array. The for loop gives you an index, which you’d then use to access the next element in the array. Here’s an example in ruby that prints off each element of an array:


a = [1, 2, 3, 4, 5]

for i in 0..(a.size - 1)
  puts a[i]
end

Not very expressive, is it. Now for the ruby equivalent:


a = [1, 2, 3, 4, 5]
a.each {|number| puts number}

Now, it’s fairly uncommon to see the above mistake, even in ruby-n00b code. Learning the ‘each’ method seems to be a rite-of-passage that almost everyone goes through.

Transforming each element of an array

So, as we mentioned, very few ruby programmers don’t know about or don’t use ‘each’. Annoyingly, it is often incorrectly used by n00bs to transform each element of an array. The following is an example where an array of lower case words is transformed into an array of upper case words:


lower_case = ["hi", "these", "are", "some", "words"]

upper_case = []
lower_case.each do |word|
  upper_case << word.upcase
end

puts upper_case.inspect

#=> ["HI", "THESE", "ARE", "SOME", "WORDS"]

Every element of the array is looped through (correct), the transformation is done (‘word.upcase’ – correct); the mistake comes when adding that element to a new array. Ruby has a method that does all this for you; it’s called ‘collect’. 


lower_case = ["hi", "these", "are", "some", "words"]

upper_case = lower_case.collect { |word| word.upcase }

puts upper_case.inspect

#=> ["HI", "THESE", "ARE", "SOME", "WORDS"]

What collect does is iterate over the array, ‘collect’ the result of the block (in this case the changing to uppercase of the block argument), store the result in a new array. It’s much shorter, but the main thing is that it’s more expressive. Here’s an example of where you could use it. Say you had a class that represented a page that you’re testing, and say that it contains a method that returns the text of every link on the page. Here’s the old school way:


class MyPage
  def links_text
    text_array = []
    @browser.links(:xpath, "//a").each do |link|
      text_array << link.text
    end
    text_array
  end
end

If you change it to use ‘collect’ instead of ‘each’, you’ll have the following instead:


class MyPage
  def links_text
    @browser.links(:xpath, "//a").collect {|link| link.text}
  end
end

Much nicer! Note that you can use ‘map’ instead of ‘collect’ if you like – one is an alias for the other.

Selecting only elements that meet some criteria

Another case of ‘each’ misuse. . . It’s a common scenario to want to select only certain items from an array – specifically elements that meet certain criteria. An example: given an array containing the numbers 1 to 10, I want to get all the even numbers. Well, one way to describe that criteria is:
element % 2 == 0
If, when the element is divided by 2, there is no remainder; element is an even number. So now, lets look at a ruby-n00b way of getting those elements:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
even_numbers = []

all_numbers.each do |number|
  even_numbers << number if number%2 == 0
end

#=> [2, 4, 6, 8, 10]

What’s happening? We’re looping through ‘each’ element, and performing our check. ‘If’ the element meet the criteria, add it to an array called ‘even_numbers’. It works, but it’s long winded, and not very expressive. Here’s the ruby way:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

even_numbers = all_numbers.select {|number| number%2 == 0}

#=> [2, 4, 6, 8, 10]

Much better. No need to create a new array before performing the check, no boilerplate add-the-element-to-the-new-array code, no dirty ‘if’s; just nice expressive code.

An example of when you’d want to use it? Say you have a table with a bunch of rows and you want a method to return only the rows that have a certain background color. Here’s n00b-style code:


class MyPage
  def blue_rows
    my_blue_rows = []
    @browser.row(:xpath, "//tr").each do |row|
      my_blue_rows << row if row.attribute("bgcolor") == "blue"
    end
    my_blue_rows
  end
end

Again, lots of fluff, hard to tell at first glance what’s going on. Here’s the same thing but this time using the ‘select’ method:


class MyPage
  def blue_rows
    @browser.row(:xpath, "//tr").select {|row| row.attribute("bgcolor") == "blue"}
  end
end

Much nicer. Expressive code. No guff.

Rejecting elements that meet certain criteria

Sometimes you want all the elements in an array apart from those which meet certain criteria. It’s almost identical to ‘select’, just. . . the opposite! Instead of selecting items which meet the supplied criteria, ‘reject’ will reject items which meet the criteria. This time, instead of selecting even numbers, we want to reject them, thus getting an array of odd numbers (a bit contrived, I know; we could just select the elements where element%2==1, but this is a tutorial).


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
odd_numbers = []

all_numbers.each do |number|
  odd_numbers << number unless number%2 == 0
end

#=> [1, 3, 5, 7, 9]

This time, we don’t add the number ‘if’ it meets the criterial; instead we add it ‘unless’ it meets the criteria. Again, it’s horrible code. The ruby way:


all_numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

odd_numbers = all_numbers.reject {|number| number%2 == 0}

#=> [1, 3, 5, 7, 9]

So when would you want to use it? With a similar example to what we have above, this time we want all the rows that haven’t got a red background. Old school code:


class MyPage
  def non_red_rows
    my_non_red_rows = []
    @browser.row(:xpath, "//tr").each do |row|
      my_non_red_rows << row unless row.attribute("bgcolor") == "red"
    end
    my_non_red_rows
  end
end

Long winded, ugly, hard to see what’s going on. Now, we’ll change it to use ruby’s ‘reject’ method:


class MyPage
  def non_red_rows
    @browser.row(:xpath, "//tr").reject {|row| row.attribute("bgcolor") == "red"}
  end
end

Hard to argue against, right? It’s short, expressive and to the point.

Summary: ruby provides nice methods for array manipulation. ‘Each’, ‘select’ and ‘reject’ are only a few of those methods, but they’re the most frequently used (or should be!). I hope this helps make your code shorter, more expressive and easier to maintain.