RSpec JUnit Formatter for Jenkins

While RSpec is great, it is missing a built in JUnit formatter so you don’t get the pretty analysis stuff from Jenkins that you otherwise would. There have been a few attempts at writing an RSpec JUnit formatter, but I’ve never seen something work consistently across versions of RSpec. In my experience, the ci_reporter gem works well, but only with some versions of RSpec. Since it doesn’t work with the current version (2.10), I decided to write my own simple JUnit formatter that will allow me to run my tests in Jenkins. To use it, save the code that follows to a file called junit.rb and then call rspec with the following options:

rspec my_spec.rb -r ./junit.rb -f JUnit -o results.xml

That will cause RSpec to require the file containing the formatter, invoke the formatter and save the JUnit-formatted results to a file called results.xml. A better option would be to require the junit.rb file from your spec_helper.rb file (which in turn is being required by your .rspec file, right?). If you do that then you’ll only need to call the following:

rspec my_spec.rb -f JUnit -o results.xml

Anyway, here’s the code. Hope it helps!

How to bring a browser to the front (on a mac)

One of the sad realities of browser testing is that for some functions to work the browser just has to be on top. If it’s not on top, tests will fail. There has to be a way to bring browsers to the front…

Well, if you’re running your tests on a Mac, you can easily bring any application to the front using AppleScript. If you’ve never heard of AppleScript, there’s a reason: it’s awful and no one uses it. But in this instance it does the job.

The following code shows you (in ruby) how to bring Chrome to the front and then resize the window:

Here’s the same thing for Firefox:

It took me a while to hunt around for a solution and then a little while more to figure out the AppleScript required to do the job. Hope it helps!

SitePrism: Capybara Page Objects

Capybara is a great browser interaction library for automated testing, but until now it hasn’t been much fun to follow the Page Object pattern with it. So, I wrote SitePrism that lets you do just that: SitePrism is a Page Object Model DSL for Capybara.

We’ve been using it at my current client since December 2011 and it’s proven a great success. Until now I’ve kept it quiet to allow some time to develop it to the point that it does everything we need, but a few days ago SitePrism went to version 1.0 – early adopters tell me that SitePrism’s intuitive API lets them create expressive and maintainable Page Objects with ease, so take a look! The SitePrism ReadMe will give you an idea of what it’s like to put together a Page Object model of the site you’re testing using SitePrism with capybara…

Here’s the code: https://github.com/natritmeyer/site_prism
And here’s the documentation: http://rdoc.info/gems/site_prism/frames

Go take a look, it might save you a lot of frustration!

(Possibly) The World’s Smallest Ruby Unit Test Tool

Skorks, a great ruby/dev blog, gave the following challenge last year:

what is the minimum amount of code needed to make a viable unit testing framework?

An interesting question… Well, he chose to replicate rspec, and managed it in 44 lines of code. Not bad! I decided to try the same but instead of rspec, I’d try to create a minimal version of the granddaddy of them all: test::unit. Here are the tests that I’d use to write the test runner against:

And this is the kind of output I’d want the tests to produce:

So… how to make this work… in as little code as possible…

Test::unit works around the idea of test classes. Any method that begins with test_ that is defined in a class that inherits from Test::Unit::TestCase would get executed in the context of a new instance of the class. Instead of the long-winded class name from test::unit, my test class name is TinyTest. The only assertion I’m going to provide is assert which tests that its argument is true. If what’s passed is not true then TinyTest will raise an error saying which line of which file the failed assertion was on; its class and method would also be reported. I also want setup and teardown functionality.

Like Test::Unit, I don’t want to have to tell the tests to execute, I want that to happen by magic. So, all the execution logic would need to go in an at_exit block. I want a passing test to print a ‘.‘, and a failing test to print an ‘F‘. Just like Test::Unit does. Finally, at the end of executing all the tests, I want to know how many of them passed and how many failed; and for each failure I want to know the file name, the line number, the class name and the method where the failure occurred.

Here’s what I put together:

28 lines! Not bad!!! How’s about that, huh? OK, it’s no cucumber, rspec, minitest or even test::unit… but it works! It does the job of a super simple test runner! I guess I could squash it further by using semi-colons instead of new lines, but that would be cheating. 28 lines it is!

A Solution to AJAX errors with Capybara against a jQuery site

Ajax has long been a pain for browser-based test automation; it is often the main culprit when looking for the reasons behind some flakey tests. The normal scenario is this:

  1. Your test clicks a button which fires off an ajax transaction
  2. While the browser is talking to the server, the next line in your test executes
  3. Result? A failing test and a nice big irrelevant stack trace
  4. Frustration
  5. Reduction in self-esteem
  6. Spend a while trying to pin down and make allowances in the code for the problem

Well, if you’re using capybara to test a site that’s using jQuery, the following may help – it’s a method called wait_for_ajax that blocks until there are no active connections between the browser and the server. Basically, it *should* sort out any ajax-related problems. No guarantees though. But it does work for me…

To use it, call wait_for_ajax just after the line of code that kicks off the ajax transaction.

The method is in a module called AjaxWaiter that gets included into ‘World‘ (this example uses cucumber, ‘World‘ is not relevant if you’re not using cucumber). Anything added to World can be used in any of your steps. To use the above it in its current form, stick it in your cucumber project’s env.rb file. Here’s an example of wait_for_ajax‘s use in cucumber:

Well, I hope that helps. I know it’s improved my life…

Better cucumber tag based logic

Previously, I’ve written up something on how to implement logic around cucumber’s tags. Well, here’s an improvement… instead of using a formatter, it turns out that you can get to a scenario’s tags as an array of strings, eg:

["@complete", "@slow"]

The place to do is in the Before or After blocks in your env.rb file, eg:

That’s a fair bit easier than using a formatter!

Tag based logic in Cucumber

Sometimes cucumber’s Before hook just doesn’t cut it. Here’s a nice hack that allows you to perform some logic during execution of cucumber scenarios when a tag is first come across:

It’s a cucumber formatter that detects when a tag is first come across. When a new tag is found, the perform_logic_for method is called, passing the tag to it. Once you’re in the perform_logic_for method, you can do what you like. I’m doing stuff like checking to see if a directory matches the tag name; if I find one, I load the data into an environment (a great example of when the Before tag isn’t enough). You have free reign in here to do what you like with the tag.

To use it, save the file to features/support/tag_logic.rb, add your own tag logic, and then execute it with:

#cucumber -f pretty -f TestManagement::TagLogic -o /dev/null

I hope that helps!

Small print:
Use the Before hook instead of this if you possibly can.