I needed to get a list of all the tags used in a cucumber project, and what follows is the formatter I wrote to list them all for me.
To use it, save the code in a file called list_tags.rb and then execute the following command:
cucumber -d -f Cucumber::Formatter::ListTags
Done!
The page object model works very well, but there are a few traps you can fall into. Alister Scott goes through a few of them and mentions one that particularly grates me: “Pages stored as instance variables”. Here’s a demonstration of the problem:
So why is that a problem? Well, there’s lots of noise – 3 lines are there just to create variables. This significantly reduces readability as you’ll end up with many, many lines of test code just creating instances of page objects. You also now have a whole load of instance variables to keep track of: @login_page, @account_page, @account_history_page. And when you’re using cucumber to run your tests, this will lead to *big* maintenance issues. I’ve had to rescue a few cucumber-based test projects and one of the most frequent causes for test-rot is that the testers lost track of their instance variables. Been in this situation before?
“Can I use @account_page here? Did I previously declare it? Hmmm… No, it’s nil when I try to use it. OK, I’ll instantiate it here. [runs the tests]. Cool, that works. Oh no! Doing that has broken some other tests that referenced @account_page but expected something else!?!? Should I fix up the other tests? Rename the @account_page variable to something else? If I do that will I break anything else?” Not fun. Big spaghetti problems.
But what to do about it? Alister suggests using blocks (provided by the page-object gem) that look something like the following:
On first glance, this looks great. No instance variables to keep track of. Just deal with the classes themselves and use only local variables inside the blocks. Nothing to keep track of. Great!
But…
The above proposal causes other maintenance hassles – the page object’s class name is now scattered throughout the code. Lots of “visit LoginPage” all over the place. What happens when the class name changes? You’ll have to make changes throughout the code. Not fun.
Solution: use an instance of an App class, this App class being a representation of the app you’re testing (the whole app, not individual pages). This App class contains one method per page class, each of these methods return an instance of the relevant page class.
“Whhhaaattt?”
OK, here’s an example:
In your tests, you would then have the following line:
When I visit the login page
…which would match the following step…
If you structure your tests such that they always begin by mentioning where the user starts (a good idea as it gives context), you can rely on the fact that @app has been instantiated so you can just use it. For example:
So how is this an improvement? Well, there’s no need to manage instance variables for different pages – just call methods on a known instantiation of the App class(@app) and they’ll return instances of the pages you want. There’s no need to mention class names; they are hidden behind methods. If the class name changes you only need to make one change (change the class referenced in the methods in the App class).
Subjective statement: I’d also argue that you also get great readability with this way of structuring things.
This is how I’ve normally organised things. And it’s worked great both on small projects of only 10′s of tests to large projects where the number of tests is 1000+. It’s the best solution I’ve come up with, it doesn’t suffer from having instance variables all over the place, neither are there class names all over the place.
I’ve written up in brief how this works if you’re using SitePrism to manage your page objects: http://rdoc.info/gems/site_prism/file/README.md#Epilogue
Skorks, a great ruby/dev blog, gave the following challenge last year:
what is the minimum amount of code needed to make a viable unit testing framework?
An interesting question… Well, he chose to replicate rspec, and managed it in 44 lines of code. Not bad! I decided to try the same but instead of rspec, I’d try to create a minimal version of the granddaddy of them all: test::unit. Here are the tests that I’d use to write the test runner against:
And this is the kind of output I’d want the tests to produce:
So… how to make this work… in as little code as possible…
Test::unit works around the idea of test classes. Any method that begins with test_ that is defined in a class that inherits from Test::Unit::TestCase would get executed in the context of a new instance of the class. Instead of the long-winded class name from test::unit, my test class name is TinyTest. The only assertion I’m going to provide is assert which tests that its argument is true. If what’s passed is not true then TinyTest will raise an error saying which line of which file the failed assertion was on; its class and method would also be reported. I also want setup and teardown functionality.
Like Test::Unit, I don’t want to have to tell the tests to execute, I want that to happen by magic. So, all the execution logic would need to go in an at_exit block. I want a passing test to print a ‘.‘, and a failing test to print an ‘F‘. Just like Test::Unit does. Finally, at the end of executing all the tests, I want to know how many of them passed and how many failed; and for each failure I want to know the file name, the line number, the class name and the method where the failure occurred.
Here’s what I put together:
28 lines! Not bad!!! How’s about that, huh? OK, it’s no cucumber, rspec, minitest or even test::unit… but it works! It does the job of a super simple test runner! I guess I could squash it further by using semi-colons instead of new lines, but that would be cheating. 28 lines it is!
Ajax has long been a pain for browser-based test automation; it is often the main culprit when looking for the reasons behind some flakey tests. The normal scenario is this:
Well, if you’re using capybara to test a site that’s using jQuery, the following may help – it’s a method called wait_for_ajax that blocks until there are no active connections between the browser and the server. Basically, it *should* sort out any ajax-related problems. No guarantees though. But it does work for me…
To use it, call wait_for_ajax just after the line of code that kicks off the ajax transaction.
The method is in a module called AjaxWaiter that gets included into ‘World‘ (this example uses cucumber, ‘World‘ is not relevant if you’re not using cucumber). Anything added to World can be used in any of your steps. To use the above it in its current form, stick it in your cucumber project’s env.rb file. Here’s an example of wait_for_ajax‘s use in cucumber:
Well, I hope that helps. I know it’s improved my life…
Previously, I’ve written up something on how to implement logic around cucumber’s tags. Well, here’s an improvement… instead of using a formatter, it turns out that you can get to a scenario’s tags as an array of strings, eg:
["@complete", "@slow"]
The place to do is in the Before or After blocks in your env.rb file, eg:
That’s a fair bit easier than using a formatter!
Sometimes cucumber’s Before hook just doesn’t cut it. Here’s a nice hack that allows you to perform some logic during execution of cucumber scenarios when a tag is first come across:
It’s a cucumber formatter that detects when a tag is first come across. When a new tag is found, the perform_logic_for method is called, passing the tag to it. Once you’re in the perform_logic_for method, you can do what you like. I’m doing stuff like checking to see if a directory matches the tag name; if I find one, I load the data into an environment (a great example of when the Before tag isn’t enough). You have free reign in here to do what you like with the tag.
To use it, save the file to features/support/tag_logic.rb, add your own tag logic, and then execute it with:
#cucumber -f pretty -f TestManagement::TagLogic -o /dev/null
I hope that helps!
Small print:
Use the Before hook instead of this if you possibly can.
When tests fail it’s nice to know why. The more precise a failure message is and the less time required to investigate why the test failed, the better.
When trying to find out what broke the test, this…
Failure/Error: search_field.should_not be_visible
expected visible? to return false, got true
…is infinitely preferable to this…
Failure/Error: search_field.visible?.should == false
expected: false
got: true (using ==)
The first failure tells you that the test expected the search field to be invisible, the second that it expected true to equal false – not very helpful. I’d rather have the first error message than the second, and unless you’re a masochist you probably would like the same (though having reviewed a lot of test code, I’m not so sure…).
Cucumber and rspec make “precision failure” easy through the use of matchers. Instead of checking whether “.visible?” returns true or false, you can use rspec matchers to write the following code:
search_field.should_not be_visible ...or... search_field.should be_visible
…instead of this:
search_field.visible?.should == false ...or... search_field.visible?.should == true
Your test code will be more understandable, and when tests fail you’ll have a high chance of knowing exactly what went wrong.
So, go and learn about RSpec Matchers!
As time goes on in a project using cucumber you’ll end up with some steps which no longer get called, and you probably want to delete them. Annoyingly, there isn’t a built-in formatter you can use to find out where they are. Sure, the ‘usage’ formatter prints off unused steps, but it prints off all the other steps too! What’s needed is a formatter that will hunt down all steps that are not being used and then print off each step’s name and its location (file location and line number).
What follows is a very basic cucumber formatter that will do just that. To use it, copy the code into a file called ‘unused.rb‘ in your ‘features/support‘ folder and then use the formatter when you run cucumber:
cucumber -d -f Cucumber::Formatter::Unused
or
cucumber -d -f Unused