Announcing ‘responsalizr’ – test HTTP Response Codes in Ruby

Get it here:

gem install responsalizr

Responsalizr is a very small and very simple gem I’ve written that extends the standard ruby Net::HTTPResponse class with a few methods to allow for idiomatic testing of HTTP response codes. The methods added are:

  1. A numeric response code comparer:
    • code?(integer) #eg: code?(200), code?(404)
  2. Response type name comparers:
    • ok? – corresponds to code 200
    • not_found? – corresponds to code 404
    • moved_permanently? – corresponds to a 301
    • etc…

    The response type name compare methods are derived from the subclasses of Net::HTTPResponse; you can find a table with their names at the bottom of this post.

Here is an example rspec script that demonstrates what you can do with responsalizr. Each test does the same thing twice; once doing a numeric response code compare, the other doing a response type name compare:


require 'rubygems'
require 'responsalizr'

include Responsalizr

describe "Google" do
  it "should respond to the uk homepage with a 200 or an ok" do
    Response.from("http://www.google.co.uk").should be_ok
    Response.from("http://www.google.co.uk").should be_a_code 200
  end

  it "should respond with a 404 or not found" do
    Response.from("http://www.google.com/bing.aspx").should be_not_found
    Response.from("http://www.google.com/bing.aspx").should be_a_code 404
  end

  it "should respond with a 302 or a moved permanently" do
    Response.from("http://finance.google.com").should be_moved_permanently
    Response.from("http://finance.google.com").should be_a_code 301
  end

  it "should be testable through a proxy" do
    Response.from("http://www.google.co.uk", {:proxy_host => "63.223.106.54", :port => 80}).should be_ok
    Response.from("http://www.google.co.uk", {:proxy_host => "63.223.106.54", :port => 80}).should be_a_code(200)
  end
end

Chose whichever way you like (codes/names) and – I’ve only used both above for demonstration purposes.

So, if all you’re doing is testing HTTP responses for their code (whether you want to do that with codes – 404 – or names – not_found) then responsalizr is the thing for you.

About the specific response name methods and their respective codes, here’s a table that summarizes them:

Response Type Name Response Code
information? 1xx
continue? 100
switch_protocol? 101
success? 2xx
ok? 200
created? 201
accepted? 202
non_authoritative_information? 203
no_content? 204
reset_content? 205
partial_content? 206
redirection? 3xx
multiple_choice? 300
moved_permanently? 301
found? 302
see_other? 303
not_modified? 304
use_proxy? 305
temporary_redirect? 307
client_error? 4xx
bad_request? 400
unauthorized? 401
payment_required? 402
forbidden? 403
not_found? 404
method_not_allowed? 405
not_acceptable? 406
proxy_authentication_required? 407
request_time_out? 408
conflict? 409
gone? 410
length_required? 411
precondition_failed? 412
request_entity_too_large? 413
request_uri_too_long? 414
unsupported_media_type? 415
requested_range_not_satisfiable? 416
expectation_failed? 417
server_error? 5xx
internal_server_error? 500
not_implemented? 501
bad_gateway? 502
service_unavailable? 503
gateway_time_out? 504
version_not_supported? 505
unknown_response? xxx

Because of the rspec (and cucumber) predicate magic, you can use, eg, the bad_gateway?, the forbidden? and the internal_server_error? method as follows:


...should be_a_bad_gateway
...should be_forbidden
...should be_an_internal_server_error

Happy HTTP Response Code testing!

Counting strings in a file: Ruby vs Windows Command shell

This is not the usual material that I put up, but I’d like to immortalize an event that demonstrated yet again the beauty of Ruby for basic file manipulation, especially in contrast to doing the same in a Windows command shell. Here goes:

“Nat, I need a script that displays a count of the number of instances of a string in a file. The output must be a number and nothing else.”
“No worries, that won’t take 2 seconds.”
“Stop right there – I don’t want any of your ruby nonsense – it must be a batch file.”
“Hmmm… Can the batch file call a ruby script?”
“No.”
“Err… ok… I’ll see what I can do.”

So off I went trawling google, stackoverflow, random blogs, and websites which can’t have seen hits since 1995. One hour, some frustration, and several cups of tea later, this is what I came up with:

findstr /C:"search string" "c:\my\file.txt" | find /C /V "nonsense"

And that, ladies and gentlemen, works! Let me explain what’s going on… The script uses 2 commands: findstr and find. findstr is used for finding strings in files, and find is also used for finding strings in files. It of course makes perfect sense to have two commands that do the same thing – the very definition of the word “intuitive”. In the above example, findstr returns lines from the file that contain the search string. These lines are piped to find which then displays the number of lines that don’t contain a particular string, in the above case: "nonsense". That will return a number. It’s the only way you can get find, findstr or a combination of the two to return a-number-and-only-a-number of the instances of a string in a file. I would love to see this improved – leave a comment if you know a better way to do it.

To demonstrate to myself why doing the above in DOS is crazy, I wrote the same line in ruby:

File.open("c:/my/file.txt").read.scan(/search string/).count

It doesn’t take much explanation: It opens a file, reads it, scans it for a search string and then returns the number of instances it found.

Now. Can we all start using the right tool for the right job please? I know it may involve a bit of learning, but that never hurt anyone. That is all.

Announcing ‘bewildr’ – test your WPF UI apps with IronRuby

Get it here: http://www.bewildr.info

After writing an automation framework to test a WPF GUI using IronRuby and White, I decided to write a ruby-specific gem for automating WPF UI tests. Kinda like Watir, but for WPF instead of the web. The gem is called bewildr - I’ve been working on it for a while and it’s finally in a releasable state.

Here’s an example of bewildr being used in rspec to show you what it’s all about:

require 'rubygems'
require 'spec'
require 'bewildr'

describe "my example app" do
  it "should not allow invalid users to log in" do
    #start the app and wait for the main window
    @app, @main_window = Bewildr::Application.start_app_and_wait_for_window("c:\\app.exe", /App v1.\d+/)

    username_field = @main_window.get(:id => "username")
    password_field = @main_window.get(:id => "password")
    login_button   = @main_window.get(:type => :button, :name => "Go")

    #some initial checks...
    username_field.should be_enabled
    password_field.should be_a_password_field

    #attempt login with invalid user
    username_field.text = "invalidUser"
    password_field.text = "s3cr3t"
    login_button.click

    #check we're not logged in
    @main_window.get(:id => "login_message").text.should match("Wrong username/password")
  end
end

And, since bewildr was written ‘BDD-style’ there are loads of examples of its use in cucumber here: http://github.com/natritmeyer/bewildr/tree/master/features/

Cool:

  • It’s written in ruby
  • It’s free (as in speech – BSD license)
  • It’s free (as in beer – there are no bazillion-dollar yearly license fees)
  • It’s easy to install (here’s how: gem install bewildr)
  • It has a clean API (makes for idiomatic tests in cucumber/rspec)
  • It has a strong test focus (…of the testers, by the testers, for the testers…)
  • It’s been written BDD-style
  • It’s updated frequently
  • It allows tests to be written in an interpreted language (with all the flexibility which that gives you – unlike White)

Not Cool:

  • It’s limited to IronRuby, not MRI/YARV (it needs access to .Net automation)
  • It’s new – expect bugs (please raise them when you find them)
  • It’s built on top of MS UI Automation (expect quirkiness)
  • There is quite a bit to do before it does everything I want it to, eg: it doesn’t yet allow you to test for visibility, there’s no drag-and-drop; see here for more

Note: I have no intention to support anything but WPF. No WinForms, no Silverlight, no [insert non-WPF tech here]. Not yet anyway…

So, if you’re looking for a UI testing tool to automate your WPF app with, but QTP is too expensive and White is too [insert pejorative here]… Well, what are you waiting for?  http://www.bewildr.info

How to get the submodules of a ruby module

I needed to be able to find out what modules were defined inside a particular ruby module. Kinda like wanting to find out what child namespaces exist for a module. It’s probably more easily explained with an example. Given the following construct…

module A
  module B; end
  module C; end
  module D
    module E; end
  end
end

…I wanted to be able to ask A what submodules are defined within it and get the following answer:

[A::B, A::C, A::D]

(btw: I don’t want to see E in the list as it is not a direct child of A).

So… since the Module class doesn’t provide anything to support doing that, I wrote a method that does it for me. Here’s the monkey-patch:


class Module
  def submodules
    constants.collect {|const_name| const_get(const_name)}.select {|const| const.class == Module}
  end
end

With that, you can now call the submodules method against any module and you’ll be returned an array of modules!

Programmatically take screenshot in IronRuby

After figuring out how to take a screenshot using .Net, I translated the C# I came up with to IronRuby. Now, when any of my IronRuby-powered tests fail, I take a screenshot – saves loads of time when trying to work out why a test failed!

Here’s the code you need:


require 'System.Drawing'
require 'System.Windows.Forms'

bitmap = System::Drawing::Bitmap.new(
        System::Windows::Forms::Screen.PrimaryScreen.Bounds.Width,
        System::Windows::Forms::Screen.PrimaryScreen.Bounds.Height,
        System::Drawing::Imaging::PixelFormat.Format32bppArgb)

System::Drawing::Graphics.FromImage(bitmap).CopyFromScreen(
        System::Windows::Forms::Screen.PrimaryScreen.Bounds.X,
        System::Windows::Forms::Screen.PrimaryScreen.Bounds.Y,
        0,
        0,
        System::Windows::Forms::Screen.PrimaryScreen.Bounds.Size,
        System::Drawing::CopyPixelOperation.SourceCopy)

bitmap.Save("c:\\screenshot.png", System::Drawing::Imaging::ImageFormat.Png)

A screenshot is taken of the primary screen (and *only* the primary screen) and is saved as a PNG to c:\screenshot.png . Modify to your heart’s content.

Hole-in-the-open-source-market Alert: cross platform gem for taking screenshots. Please don’t make me write it!

How to connect to an Oracle database in IronRuby

After spending a few hours trying to connect to an oracle database in ironruby using various gems, I gave up. None of the gems out there would work, each for a different reason. It was time to write my own class to do the job of managing connections and executing queries.

You’ll need a few things:

  1. A copy of the ‘Oracle.DataAccess.dll’ that you can find somewhere inside this outrageously large download:
    http://www.oracle.com/technology/software/tech/windows/odpnet/index.html
    Copy the ‘Oracle.DataAccess.dll’ into your load path (you’ll find it somewhere in the bowels of the directory structure that the above installs)
  2. An oracle database you can point at
  3. The connection string required to connect to the database. It’ll look something like:
    Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROT...

Once you’ve got those details, you can use the following class:

require 'System'
require 'System.Data'
require File.join(File.expand_path(File.dirname(__FILE__)), "Oracle.DataAccess.dll")

class IronRubyOracleClient
  #pass in oracle connection string, eg, for Test env:
  def initialize(connection_string)
    @connection = Oracle::DataAccess::Client::OracleConnection.new(connection_string)
  end

  #opens connection
  def open
    @connection.open
  end

  #returns 2D array
  def execute(query)
    @query = query
    cmd = Oracle::DataAccess::Client::OracleCommand.new(@query, @connection)
    cmd.CommandType = System::Data::CommandType.Text
    data_reader = cmd.ExecuteReader()
    column_count = data_reader.visible_field_count.to_i

    result_rows = ::System::Collections::ArrayList.new

    while(data_reader.read) do
      row = ::System::Collections::ArrayList.new
      column_count.times do |i|
        row.add(data_reader.get_oracle_value(i).to_string)
      end
      result_rows.add(row)
    end

    result_set = []

    result_rows.each do |result_row|
      ruby_row = []
      result_row.each do |cell|
        ruby_row << cell.to_s
      end
      result_set << ruby_row
    end

    result_set
  end

  #close connection
  def close
    @connection.close
  end
end

And here’s how you use it:


#create your connection string
connection_string  = "Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=..." #etc...
#create an instance of the client, passing in the connection string
my_client = IronRubyOracleClient.new(connection_string)
#open a connection to the database
my_client.open
#execute a query and save the result
results = my_client.execute("select * from some_table")
#dump the results (a 2D array of values)
puts results.inspect
#close the connection
my_client.close

It’s fairly slow, but it works. Which is an improvement on what’s out there…

Note that everything is returned as a string. For some reason, the unless the data is a basic string or is a number that fits into an integer, the data gets garbled somewhere between the dll and ironruby. I can’t find out where, so everything-returned-as-a-string is the current compromise. If you can get it to work with all data types, send it along!

Testing redirects with ruby

—UPDATE—
Since writing this post, I’ve put together a gem called ‘responsalizr‘ which is a way better solution than what follows in this post. Read about it here. And now back to the original post…

Testing redirects from a web app is simple enough – make a request and check the response code making sure it’s a 301, 302 or whatever you’re expecting. The test you end up writing isn’t nice idiomatic ruby though. So, I wrote a quick monkey patch… here it is:

class Net::HTTPResponse
  #returns true if the response is a 200
  def ok?
    instance_of?(Net::HTTPOK)
  end

  #returns true if the response is a 301
  def found?
    instance_of?(Net::HTTPFound)
  end

  #returns true if the response is a 302
  def moved_permanently?
    instance_of?(Net::HTTPMovedPermanently)
  end

  #returns true for any kind of redirect (301..307)
  def redirect?
    kind_of?(Net::HTTPRedirection)
  end

  #returns the url being redirected to if this response is a redirect
  def redirect_url
    redirect? ? self['location'] : raise("Not a redirect response")
  end
end

Basically, it adds the following predicate methods to the Net::HTTPResponse class: “ok?” (returns true if response code is 200), “found?” (returns true if response code is 301), “moved_permanently?” (returns true if response code is 302) and “redirect?” (returns true if the response is a redirect – from 301..307). Because these are predicate methods, they can be used by rspec – your test code suddenly becomes much cleaner! The moneky patch also adds the “redirect_url” method which returns the url to be redirected to if the response is a redirect of some sort. The advantage of this is that your tests become much more idiomatic:


#the urls we're testing with...
@url_initially_navigated_to = "http://mail.google.com"
@expected_redirect_url = "https://www.google.com/.../etc/..."

#make the request
@response= Net::HTTP.get_response(URI.parse(@url_initially_navigated_to))

#nice idiomatic tests! - use the one you're expecting...
@response.should be_ok                #expecting a 200
@response.should be_found             #expecting a 301
@response.should be_moved_permanently #expecting a 302
@response.should be_a_redirect        #expecting anything between 301..307

#testing the url being redirected to
@response.redirect_url.should match(@expected_redirect_url)

There you go! Nice idiomatic redirect tests! Enjoy.

How to test a WPF app using IronRuby and White

UPDATE
Since I wrote this post, I have put together a ruby gem designed for testing WPF UIs called ‘bewildr’. I wrote up an introductory post about bewildr here. Bewildr removes the need for using White – it’s written in ruby, for ruby! You’ll find its API clean and idiomatic. Anyway, back to the original post…
—————–

…and you thought ruby was only good for web testing…

So… having previously failed at getting ruby-based automated testing of WPF working, I attacked it from a different angle and succeeded! The difference is that this time I used IronRuby instead of ruby. In doing so, the White Automation library could be used out-of-the-box! It’s time to throw away your expensive, proprietary test tools and replace them with open source tools instead! Here’s how it’s done:

Prerequisites

  • Install .Net Framework 2.0, 3.0 (and possibly 3.5 – I’ve got it installed and haven’t tested this stuff without it)
  • Install IronRuby (but put it in c:\ironruby, not the default location that contains spaces in the path)
  • Download the latest version of White (0.19 at time of writing)
  • Install UISpy.exe to investigate objects on screen
  • Bookmark the White API – you’ll use this a lot

I’m presuming that you know to run ruby files in IronRuby using the ir command, not the standard ruby command, iirb instead of irb and irake instead of rake.

Requires, Dlls and some helpful methods

First thing, you need to gain access to White in your project. To do that, put the following at the top of your app (eg: your env.rb file in cucumber):


$LOAD_PATH << File.expand_path("lib/White_Bin_0.19")
require "White.Core.dll"
require 'UIAutomationTypes, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'

module IronRubyHelpers
 def r_array_to_cs_array(args)
   System::Array[System::String].new(args.map {|arg| arg.to_s.to_clr_string})
 end

 def r_string_to_c_string(r_string)
   System::String.new(r_string)
 end
end

This requires some explanation… The first line puts the directory containing the White.Core.dll file on the load path. In this case, I’ve taken the White_Bin_0.19 directory that was created when I expanded the zip I downloaded from the White site, and put it inside the lib directory in my NetBeans project. The next line requires the White library. The line after that loads the UIAutomationTypes.dll – you’ll need this if you want to find WPF objects by their type.

Next, the IronRubyHelpers module… it contains 2 methods: one converts a ruby array to a dot net array (required whenever a White object method takes an array as an argument, eg: menu interaction), and the other converts a ruby string to a dot net string (sometimes required when a White object arg takes a string).

That’s all the setup you need.

Starting and Killing WPF Apps

Now we start dealing with White… Here’s how to start a WPF app :

@app = White::Core::Application.Launch("c:\my_app\example.exe")

And here’s how to kill it:

@app.kill

The Launch(string) method is one of many ways to start an app. Check out the Application.cs documentation to see how else it can be done (I can’t get AttachOrLaunch to work – ymmv).

Grabbing Windows

You get access to objects by using the window that they’re in. That means you need to get the window first. Here’s how it’s done:

@main_window = @app.get_windows.select{|window| window.title == "My Window Title"}.first

That will get you a window whose title matches “My Window Title” exactly. If you want a bit more flexibility (eg: if your app displays its version number in the title and you don’t want to have to change your recognition string everytime you get a new build), you can use the following regex-powered window finder:

@main_window = @app.get_windows.select{|window| window.title =~ /My App, version (.*)/}.first

Your window takes a while to appear? Here’s how to wait for it:

Timeout.timeout(10) do
  sleep 0.2 until @app.get_windows.select{|window| window.title == "My App"}.size > 0
end

Now you’ll wait for up to 10 seconds checking ~5 times a second to see if a window entitled “My App” appears (it only takes a tiny tweak to make it use regex). If it takes longer than 10 seconds you’ll get a Timeout exception.

Grabbing and Interacting with Objects

Now that we can get hold of windows, we can look inside them and get access to the object that they contain. White uses static methods on SearchCriteria to access object. Generally, you’ll need 2 bits of info:

  1. How to search (eg: by id, by control type, by text, etc)
  2. The value to search for (eg: “btnLogin”, ControlType.Edit, “Login…”)

Here are some examples:

@chk_visibility = @main_window.get(White::Core::UIItems::Finders::SearchCriteria.ByAutomationId(r_string_to_c_string("visibleCheckBox")))

Hey, no one said it was pretty. Here we’re getting a checkbox that lives in the main window. We’re identifying it by its automation id, which happens to be “visibleCheckBox”. Another example:

@cmb_zoom = @main_window.get(White::Core::UIItems::Finders::SearchCriteria.ByControlType(System::Windows::Automation::ControlType.ComboBox))

Here we’re getting the only combo box in the main window. This time we’re searching by ControlType. You can find a full list of the available control types here under the “Fields” section.

There are a few more ways to use SearchCriteria, you can read about them in the SearchCriteria.cs documentation.

And now to object interaction… Once you’ve got your object reference, you can call methods on it and hope it responds. Here are some examples:

@chk_visibility.checked

…will return true or false based on whether the checkbox is checked – read about the available methods on checkbox here. Another example:

@cmb_zoom.select(r_string_to_c_string("200%"))

This will select the “200%” option from the combo box.

Depending on the object type you could call .click, .value = "hello", .double_click, etc.

Navigating the Window Menu

Since using the window menu doesn’t fit the standard object-locator pattern, we’ll cover that here.

def select_menu_item(*args)
 my_menu_bar = @main_window.menu_bar
 my_menu_item = my_menu_bar.menu_item(r_array_to_cs_array(args))
 my_menu_item.click
end

select_menu_item("File", "Open...")

Using window menus is a bit weird in White. The easiest way to go about it is to use the select_menu_item above. It gets the menu bar out of the window, does some ruby-to-dot-net magic with the arguments and then clicks the menu item. I’ve included an example call to the method that does the standard File->Open action.

A few more things

Because we’re using IronRuby, there’s a quirk you need to be aware of – you may have spotted it already… IronRuby translates White’s CamelCase methods to the standard ruby_underscore_case. Eg:, if the White documentation says that a particular object supports the “DoubleClick” method, you can call that method in your ruby scripts with “double_click”. You get used to it.

Speed and Reliability

…or lack of same. White is slow. And buggy. IronRuby startup time is slow too. I’ve found loads of places where White claims to do something (eg: table interaction), but what’s there isn’t unusable. Some of the stuff that does work only works often, not all the time. I’ve rewritten some of the buggy functionality using the low level interaction libraries provided by microsoft and the replacement code runs several orders of magnitude faster than white. Please don’e expect fast execution – you just won’t get it. Be prepared for molasses-speed testing.

Example project using Paint.NET

I’ve put together a badly designed set of tests around Paint.NET (3.5.3 at time of writing) using Test::Unit. It’s a NetBeans project, but even so you’ll need to run the tests off the command line (there’s no IronRuby-NetBeans support that I can find). Make sure to have Paint.NET installed in its default location and then try running “irake test” in the project directory… and have patience – it crawls. It takes around 2 minutes to run 4 tests.

>>> Download my example project here <<<

Enjoy!