Testing a web app in an unstable environment is a pain. Many tests will fail for environmental reasons and filtering out those results can take a while. To make this sort of thing less painful, Watir and its derivatives (eg: firewatir) provide the ability to run any number of methods each time a page is loaded. They call these methods “checkers”… because… they check stuff. Here’s how to use them:
1) Write your checker
Checkers are Ruby Procs – essentially, nameless methods stored in variables. Here’s an example:
check_for_bad_things = Proc.new do
puts "Server Error!" if @browser.text.include?("server error")
end
2) Add the checker to your instance of the browser
@browser = Watir::Browser.new
@browser.add_checker(check_for_bad_things)
3) Run your test. Every time a page is loaded, the check_for_bad_things proc is executed; and instead of seeing obscure failures when the test environment dies, you’ll see a message saying “Server Error!” making it easy to spot tests that failed for environmental/non-regression reasons.
You can make the checkers more useful by making them throw exceptions instead of just printing a line to the console. This way, they’ll cause tests to end with error instead of failed, making it even easier to sort out real failed tests from fails due to environment instability.
“add_checker” runs a lot more regularly than just page load. I discovered this when using “add_checker” to keep a copy of the previous page html to report on failure (when investigating a failure, the previous page is often more useful than a screenshot of the page where the error occured, infact errors dismissed as flakey testing turned out to be caching issues because we had the html of the previous page to refer to). The “previous” html was continually a copy of the “current” html, and it was then that I discovered that a lot of events trigger add_checkers.
Also, add_checker can easily miss page loads as well. Again while investigating “flakey” tests, I saw that when the application navigated from page A to page C via a quick visit to page B, occasionally add_checker failed to run on page B. Unfortunately the add_checker running on page B was crucial to test reliability!
Anyway, these are my observations on add_checker, and you’ll be pleased know that test reliability is much improved now that I fully understand add_checker.