Bewildr 0.1.7

Bewildr – the IronRuby based gem for automated testing of WPF apps – has been updated! We’re now up to v0.1.7. The main changes are:

  • Start exes with command line arguments (see last step of this file)
  • Navigate the object hierarchy using element.parent and element.children methods (see here)
  • Use “\n” in rich text boxes instead of the clumsy “{ENTER}” syntax (see here)

There have also been some changes in the background… elements now build themselves dynamically based on what patterns are supported by the underlying MS UI automation element. There’s still a bit of ugliness here, but it’s fast being stomped out.

gem update bewildr

What are you waiting for? Go get it!

Announcing ‘bewildr’ – test your WPF UI apps with IronRuby

Get it here: http://www.bewildr.info

After writing an automation framework to test a WPF GUI using IronRuby and White, I decided to write a ruby-specific gem for automating WPF UI tests. Kinda like Watir, but for WPF instead of the web. The gem is called bewildr - I’ve been working on it for a while and it’s finally in a releasable state.

Here’s an example of bewildr being used in rspec to show you what it’s all about:

require 'rubygems'
require 'spec'
require 'bewildr'

describe "my example app" do
  it "should not allow invalid users to log in" do
    #start the app and wait for the main window
    @app, @main_window = Bewildr::Application.start_app_and_wait_for_window("c:\\app.exe", /App v1.\d+/)

    username_field = @main_window.get(:id => "username")
    password_field = @main_window.get(:id => "password")
    login_button   = @main_window.get(:type => :button, :name => "Go")

    #some initial checks...
    username_field.should be_enabled
    password_field.should be_a_password_field

    #attempt login with invalid user
    username_field.text = "invalidUser"
    password_field.text = "s3cr3t"
    login_button.click

    #check we're not logged in
    @main_window.get(:id => "login_message").text.should match("Wrong username/password")
  end
end

And, since bewildr was written ‘BDD-style’ there are loads of examples of its use in cucumber here: http://github.com/natritmeyer/bewildr/tree/master/features/

Cool:

  • It’s written in ruby
  • It’s free (as in speech – BSD license)
  • It’s free (as in beer – there are no bazillion-dollar yearly license fees)
  • It’s easy to install (here’s how: gem install bewildr)
  • It has a clean API (makes for idiomatic tests in cucumber/rspec)
  • It has a strong test focus (…of the testers, by the testers, for the testers…)
  • It’s been written BDD-style
  • It’s updated frequently
  • It allows tests to be written in an interpreted language (with all the flexibility which that gives you – unlike White)

Not Cool:

  • It’s limited to IronRuby, not MRI/YARV (it needs access to .Net automation)
  • It’s new – expect bugs (please raise them when you find them)
  • It’s built on top of MS UI Automation (expect quirkiness)
  • There is quite a bit to do before it does everything I want it to, eg: it doesn’t yet allow you to test for visibility, there’s no drag-and-drop; see here for more

Note: I have no intention to support anything but WPF. No WinForms, no Silverlight, no [insert non-WPF tech here]. Not yet anyway…

So, if you’re looking for a UI testing tool to automate your WPF app with, but QTP is too expensive and White is too [insert pejorative here]… Well, what are you waiting for?  http://www.bewildr.info

How to test a WPF app using IronRuby and White

UPDATE
Since I wrote this post, I have put together a ruby gem designed for testing WPF UIs called ‘bewildr’. I wrote up an introductory post about bewildr here. Bewildr removes the need for using White – it’s written in ruby, for ruby! You’ll find its API clean and idiomatic. Anyway, back to the original post…
—————–

…and you thought ruby was only good for web testing…

So… having previously failed at getting ruby-based automated testing of WPF working, I attacked it from a different angle and succeeded! The difference is that this time I used IronRuby instead of ruby. In doing so, the White Automation library could be used out-of-the-box! It’s time to throw away your expensive, proprietary test tools and replace them with open source tools instead! Here’s how it’s done:

Prerequisites

  • Install .Net Framework 2.0, 3.0 (and possibly 3.5 – I’ve got it installed and haven’t tested this stuff without it)
  • Install IronRuby (but put it in c:\ironruby, not the default location that contains spaces in the path)
  • Download the latest version of White (0.19 at time of writing)
  • Install UISpy.exe to investigate objects on screen
  • Bookmark the White API – you’ll use this a lot

I’m presuming that you know to run ruby files in IronRuby using the ir command, not the standard ruby command, iirb instead of irb and irake instead of rake.

Requires, Dlls and some helpful methods

First thing, you need to gain access to White in your project. To do that, put the following at the top of your app (eg: your env.rb file in cucumber):


$LOAD_PATH << File.expand_path("lib/White_Bin_0.19")
require "White.Core.dll"
require 'UIAutomationTypes, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'

module IronRubyHelpers
 def r_array_to_cs_array(args)
   System::Array[System::String].new(args.map {|arg| arg.to_s.to_clr_string})
 end

 def r_string_to_c_string(r_string)
   System::String.new(r_string)
 end
end

This requires some explanation… The first line puts the directory containing the White.Core.dll file on the load path. In this case, I’ve taken the White_Bin_0.19 directory that was created when I expanded the zip I downloaded from the White site, and put it inside the lib directory in my NetBeans project. The next line requires the White library. The line after that loads the UIAutomationTypes.dll – you’ll need this if you want to find WPF objects by their type.

Next, the IronRubyHelpers module… it contains 2 methods: one converts a ruby array to a dot net array (required whenever a White object method takes an array as an argument, eg: menu interaction), and the other converts a ruby string to a dot net string (sometimes required when a White object arg takes a string).

That’s all the setup you need.

Starting and Killing WPF Apps

Now we start dealing with White… Here’s how to start a WPF app :

@app = White::Core::Application.Launch("c:\my_app\example.exe")

And here’s how to kill it:

@app.kill

The Launch(string) method is one of many ways to start an app. Check out the Application.cs documentation to see how else it can be done (I can’t get AttachOrLaunch to work – ymmv).

Grabbing Windows

You get access to objects by using the window that they’re in. That means you need to get the window first. Here’s how it’s done:

@main_window = @app.get_windows.select{|window| window.title == "My Window Title"}.first

That will get you a window whose title matches “My Window Title” exactly. If you want a bit more flexibility (eg: if your app displays its version number in the title and you don’t want to have to change your recognition string everytime you get a new build), you can use the following regex-powered window finder:

@main_window = @app.get_windows.select{|window| window.title =~ /My App, version (.*)/}.first

Your window takes a while to appear? Here’s how to wait for it:

Timeout.timeout(10) do
  sleep 0.2 until @app.get_windows.select{|window| window.title == "My App"}.size > 0
end

Now you’ll wait for up to 10 seconds checking ~5 times a second to see if a window entitled “My App” appears (it only takes a tiny tweak to make it use regex). If it takes longer than 10 seconds you’ll get a Timeout exception.

Grabbing and Interacting with Objects

Now that we can get hold of windows, we can look inside them and get access to the object that they contain. White uses static methods on SearchCriteria to access object. Generally, you’ll need 2 bits of info:

  1. How to search (eg: by id, by control type, by text, etc)
  2. The value to search for (eg: “btnLogin”, ControlType.Edit, “Login…”)

Here are some examples:

@chk_visibility = @main_window.get(White::Core::UIItems::Finders::SearchCriteria.ByAutomationId(r_string_to_c_string("visibleCheckBox")))

Hey, no one said it was pretty. Here we’re getting a checkbox that lives in the main window. We’re identifying it by its automation id, which happens to be “visibleCheckBox”. Another example:

@cmb_zoom = @main_window.get(White::Core::UIItems::Finders::SearchCriteria.ByControlType(System::Windows::Automation::ControlType.ComboBox))

Here we’re getting the only combo box in the main window. This time we’re searching by ControlType. You can find a full list of the available control types here under the “Fields” section.

There are a few more ways to use SearchCriteria, you can read about them in the SearchCriteria.cs documentation.

And now to object interaction… Once you’ve got your object reference, you can call methods on it and hope it responds. Here are some examples:

@chk_visibility.checked

…will return true or false based on whether the checkbox is checked – read about the available methods on checkbox here. Another example:

@cmb_zoom.select(r_string_to_c_string("200%"))

This will select the “200%” option from the combo box.

Depending on the object type you could call .click, .value = "hello", .double_click, etc.

Navigating the Window Menu

Since using the window menu doesn’t fit the standard object-locator pattern, we’ll cover that here.

def select_menu_item(*args)
 my_menu_bar = @main_window.menu_bar
 my_menu_item = my_menu_bar.menu_item(r_array_to_cs_array(args))
 my_menu_item.click
end

select_menu_item("File", "Open...")

Using window menus is a bit weird in White. The easiest way to go about it is to use the select_menu_item above. It gets the menu bar out of the window, does some ruby-to-dot-net magic with the arguments and then clicks the menu item. I’ve included an example call to the method that does the standard File->Open action.

A few more things

Because we’re using IronRuby, there’s a quirk you need to be aware of – you may have spotted it already… IronRuby translates White’s CamelCase methods to the standard ruby_underscore_case. Eg:, if the White documentation says that a particular object supports the “DoubleClick” method, you can call that method in your ruby scripts with “double_click”. You get used to it.

Speed and Reliability

…or lack of same. White is slow. And buggy. IronRuby startup time is slow too. I’ve found loads of places where White claims to do something (eg: table interaction), but what’s there isn’t unusable. Some of the stuff that does work only works often, not all the time. I’ve rewritten some of the buggy functionality using the low level interaction libraries provided by microsoft and the replacement code runs several orders of magnitude faster than white. Please don’e expect fast execution – you just won’t get it. Be prepared for molasses-speed testing.

Example project using Paint.NET

I’ve put together a badly designed set of tests around Paint.NET (3.5.3 at time of writing) using Test::Unit. It’s a NetBeans project, but even so you’ll need to run the tests off the command line (there’s no IronRuby-NetBeans support that I can find). Make sure to have Paint.NET installed in its default location and then try running “irake test” in the project directory… and have patience – it crawls. It takes around 2 minutes to run 4 tests.

>>> Download my example project here <<<

Enjoy!