Category Archives: General Testing Stuff

How to bring a browser to the front (on a mac)

Posted on by
Reply

One of the sad realities of browser testing is that for some functions to work the browser just has to be on top. If it’s not on top, tests will fail. There has to be a way to bring browsers to the front…

Well, if you’re running your tests on a Mac, you can easily bring any application to the front using AppleScript. If you’ve never heard of AppleScript, there’s a reason: it’s awful and no one uses it. But in this instance it does the job.

The following code shows you (in ruby) how to bring Chrome to the front and then resize the window:

Here’s the same thing for Firefox:

It took me a while to hunt around for a solution and then a little while more to figure out the AppleScript required to do the job. Hope it helps!

(Possibly) The World’s Smallest Ruby Unit Test Tool

Posted on by
1

Skorks, a great ruby/dev blog, gave the following challenge last year:

what is the minimum amount of code needed to make a viable unit testing framework?

An interesting question… Well, he chose to replicate rspec, and managed it in 44 lines of code. Not bad! I decided to try the same but instead of rspec, I’d try to create a minimal version of the granddaddy of them all: test::unit. Here are the tests that I’d use to write the test runner against:

And this is the kind of output I’d want the tests to produce:

So… how to make this work… in as little code as possible…

Test::unit works around the idea of test classes. Any method that begins with test_ that is defined in a class that inherits from Test::Unit::TestCase would get executed in the context of a new instance of the class. Instead of the long-winded class name from test::unit, my test class name is TinyTest. The only assertion I’m going to provide is assert which tests that its argument is true. If what’s passed is not true then TinyTest will raise an error saying which line of which file the failed assertion was on; its class and method would also be reported. I also want setup and teardown functionality.

Like Test::Unit, I don’t want to have to tell the tests to execute, I want that to happen by magic. So, all the execution logic would need to go in an at_exit block. I want a passing test to print a ‘.‘, and a failing test to print an ‘F‘. Just like Test::Unit does. Finally, at the end of executing all the tests, I want to know how many of them passed and how many failed; and for each failure I want to know the file name, the line number, the class name and the method where the failure occurred.

Here’s what I put together:

28 lines! Not bad!!! How’s about that, huh? OK, it’s no cucumber, rspec, minitest or even test::unit… but it works! It does the job of a super simple test runner! I guess I could squash it further by using semi-colons instead of new lines, but that would be cheating. 28 lines it is!

Testing a website on different versions of IE

Posted on by
Reply

So, no matter how much you argue that it’s an ancient, irrelevant browser; there’s no way you can wriggle out of having to test your web app against IE6 on WinXP. But… trying to find a machine with it lying around might be difficult. Magnanimous Microsoft have made testing your web app on different IE/Windows combinations less tedious than it could be: combine Virtual PC with a collection of pre-built images and you’re on your way. Here are the details…

First, you’ll need to install Virtual PC:

  • Download the latest version of Virtual PC from here if you’re running Windows 7
  • Download Virtual PC 2007 from here if you’re running Windows XP/Vista

Next, you’ll need to download as many of the following combinations of Windows/IE as you want from here.

The Windows/IE combinations available are:

  • IE6 / XP SP3
  • IE7 / XP SP3
  • IE7 / Vista SP1
  • IE7 / Vista SP2
  • IE7 / Vista SP3
  • IE8 / XP SP3
  • IE8 / Vista SP1
  • IE8 / Vista SP2
  • IE8 / Vista SP3

Download the images you need, start them, get your testing done before the VM runs out of time (they’re time limited) and then get back to doing something less painful!

Enjoy your multi-IE-version testing. Rather you than me ;)

Explaining Watir, Selenium and WebDriver

Posted on by
Reply

Something I seem to be explaining to people all the time… the relationships between Watir, Watir “2.0″, Selenium, Selenium “2.0″ and WebDriver.

Alister Scott has done an excellent job of explaining the impact of WebDriver on Selenium and Watir and what it mean for their respective futures. Read it here.

Summary:

  • Selenium 1 => dead
  • Selenium 2 == WebDriver
  • Watir 1.* (+ derivs) => dead
  • Watir 2.0 => Watir API using WebDriver to communicate with the browser

===Update===
Added version number to Watir to avoid the confusion seen here.

The Duct Tape Programmer

Posted on by
Reply

If you’re designing, writing or maintaining a UI test automation framework, read this before you start:

http://www.joelonsoftware.com/items/2009/09/23.html

A great quote from the article:

A 50%-good solution that people actually have solves more problems and survives longer than a 99% solution that nobody has because it’s in your lab where you’re endlessly polishing the damn thing.

I’ve written up similar stuff: Finding the balance between hacky and over-engineered UI test automation frameworks.

Finding the balance between hacky and over-engineered UI test automation frameworks

Posted on by
1

There are very few real requirements for a UI test automation framework:

  1. It should provide accurate test results
  2. It should provide accurate test results every time you ask it for results
  3. It should make it easy to write tests
  4. It should require little maintenance – time should be spent writing tests and analyzing results, not on coding the framework
  5. It should be easy to tweak in order to deal with last minute changes to the app being tested.

Projects rarely have the time or patience to deal with “we can’t run the tests just now, we’ve got a framework issue”, so frameworks tend to get built alongside the tests; and unless you’re careful, frameworks written under these (quite common) conditions normally die in one of 2 ways:

  1. Due to time constraints, any changes that have to be made to the test framework tend to be band-aids/hacks. “oh,-didn’t-we-tell-you-about-[insert-new-feature-that-will-break-lots-of-tests]-oh-and-can-you-kick-off-a-run-in-5-minutes?-Just-make-it-work!”, etc. That’s just the nature of the job. But, as the many dead UI frameworks that litter IT shops will attest to, there’re only so many band-aids you can stick onto a framework before it collapses under it’s own weight. Eventually, a change comes along that can’t be fixed just by “adding another band-aid” – a big refactor is required to deal with the new feature which in turn causes other framework instability problems. Test runs become unreliable resulting in the framework being abandoned.
  2. The other way frameworks die is when the test automation team are given time and money and are told to come back with a test automation framework… they have lots of time, so they spend lots of it on making things super-abstract, modeling business entities, writing test parsers etc. The tests that are written using the framework are all ‘semantic’, but they can’t deal with those “oh,-didn’t-we-tell-you…” changes to the app being tested. The super-abstracted nature of the framework makes it difficult to “just make it work” – there’s no one place to stick the band-aid, it needs to be spread across the framework. Many files need updating, the beautiful (but ultimately useless) business model is broken, and major refactors are required to ‘fix’ the model. During this time the tests can’t run. The framework ends up on a shelf gathering dust.

Like most things, a middle ground needs to be found:

  • A framework should be flexible and simple enough to be able to deal with last minute changes in the application under test. But, small chunks of time should then be given to allow small refactors of the framework do deal with the change ‘properly’ so that the quick hack can be removed. This way, the framework stays lean and can deal with new changes on a whim.
  • The framework shouldn’t be over-engineered – simplicity is key. Abstraction for abstraction’s sake is an utter waste of time. Modeling business entities in classes usually isn’t required, and when it is, only small elements of the model are usually needed for testing purposes. Doubtless, often it makes sense to model fundamental things like users, but rarely have I needed to keep track of more than the username, password and a few other simple fields. Keep business model classes simple – that way they’ll deal with application changes without much work.

Hacks for hacks’ sake aren’t good. Abstractions for abstraction’s sake aren’t good either. Write what needs to be written, don’t write what doesn’t need to be written, keep things simple, and tidy up after yourself when things get hacky.