Radical Test Portability

8/14/2019 Radical Test Portability

1/27

Radical Test PortabilityIan Dees - texagon.blogspot.com

FOSCON 2007

Hi, I'm Ian Dees. I write software for a test equipment manufacturer in the Portland, Oregon area.This talk isn't really related to work, but our jumping-ofpoint is sort of inspired by something thathappened on the job once: I had the chance to evaluate some really crappy GUI testing toolkits.


2/27

Where there is a stink of [ ],

there is a smell of being.

Antonin Artaud

But a funny thing about crap is that it can be fantastic fertilizer for growth.

That's not what Artaud meant by this quote, by the way. But that's okay -- he was insane anyway.


3/27

Radical Test Portability

So, in keeping with FOSCON's "Really Radical Ruby" theme, let's talk about Radical Test Portability.

It shouldn't be a revolutionary concept that tests can be portable. It should be old news.


4/27

But you wouldn't know that from reading the websites of the big-time test toolkit vendors. Theysay, "No programming." What that really means is, "No programming yet." These fire-and-forgettools typically generate a pile of code that you then have to go and customize.


5/27

MoveMouse(125,163);

Delay(0.65);

LeftButtonDown();

Delay(0.074);

LeftButtonUp();

GetWindowText(hCtrl, buffer, bufferSize);if (0 != lstrcmp(buffer, L"Some text"))

LOG_FAILURE("Text didn't match\n");

Delay(0.687);

MoveMouse(204,78);

//... ad nauseam

And as often as not, you get a mess like this. Good luck trying to figure out where to insert yourpass/fail tests, or maintaining this zoo a month from now, when the GUI designer moves a button.


6/27

def press_button(caption)

send_msg 'PressButton ' + caption

end

press_button'OK'

If the vendors would just implement clean protocols and document them well, you could write yourtest script in your choice of language instead of theirs.

In this hypothetical case, it just so happens that the Ruby function name looks like the message wewant to send. So the code can et even sim ler....


7/27

def_msg :press_button, [:caption]

press_button'OK'

...by using Ruby's metaprogramming capabilities.

And when the language gets clear, a funny thing starts to happen.


8/27

conversation

The tests become the centerpiece of a conversation between designers, developers, and testers.

And in that spirit, I'd like to show you some sample code.


9/27

require'spec_helper'

describe'A text editor'do

it_should_behave_like'a document-based app'

it'supports cut and paste'do

@editor.text='chunky'

@editor.select_all

@editor.cut

@editor.text.should==''

2.times{@editor.paste}

@editor.text.should=='chunkychunky'

end

end

Imagine that we're discussing a text editor's cut/paste feature. Using the RSpec test descriptionlanguage built on Ruby, we come up with something like this. The heart of the test is the third linefrom the end, where we express the intended behavior with "should."


10/27

require'spec_helper'

describe'A text editor'do

it_should_behave_like'a document-based app'

it'can undo the most recent edit'do

@editor.text='bacon'

@editor.text=''

@editor.undo

@editor.text.should=='bacon'

end

end

Here's another example, testing the undo feature. See, you can show this stuf to anyone, nomatter how much Ruby code they've seen in their lives.

The setup code happens in that spec_helper file. We won't see the code for that -- it's on my blog.All we need to know for now is that it creates a TextEditor ob ect.


11/27

require 'Win32API'

class TextEditor

@@keybd_event = Win32API.new 'user32',

'keybd_event', ['I', 'I', 'L', 'L'], 'V'

KEYEVENTF_KEYDOWN = 0x0000

KEYEVENTF_KEYUP = 0x0002

VK_BACK = 0x08

def keystroke(*keys)keys.each do |k|

@@keybd_event.call k, 0, KEYEVENTF_KEYDOWN, 0

sleep 0.05

end

keys.reverse.each do |k|

@@keybd_event.call k, 0, KEYEVENTF_KEYUP, 0

sleep 0.05

end

end

end

And here's a small piece of the TextEditor object's implementation for Windows Notepad. Here,we're teaching Ruby how to type a single character, possibly using modifier keys like Shift or Ctrl.We use the Win32API library to call the keybd_event function -- we press the keys down, thenrelease them in reverse order.


12/27

class TextEditor

def text=(string) unless string =~ /^[a-z ]*$/

raise 'Lower case and spaces only, sorry'

end

select_all

keystroke VK_BACK

string.upcase.each_byte {|b| keystroke b} end

end

But the top-level test script never calls keybd_event directly. It calls text= instead, which breaksthe message up into characters and types each one in turn. On Windows, a character is usuallydiferent than a keystroke (the "A" key might type an upper-case "A" or a lower-case "a"). So we're

just going to deal with the easiest subset of messages where the character and keystroke are thesame.


13/27

And here's what it looks like when you run it. I promise that's Ruby doing the typing!

Now, on to portability. Since we didn't put any Windows-specific calls into our top-level test script,we can port it to other platforms, just by supplying a new implementation of the TextEditor class.

l T tEdit


14/27

class TextEditor

def initialize

@window = JFrameOperator.new 'FauxPad'

@edit = JTextAreaOperator.new @window

@undo = JButtonOperator.new @window, 'Undo' end

def text=(string)

@edit.set_text '' @edit.type_text string

end

%w(text select_all cut paste).each do |m|define_method(m) {@edit.send m}

end

def undo; @undo.do_click end

endHere's a test for a trivial editor for the Java runtime called FauxPad. You can run our same Ruby testfor it using JRuby. I was going to show you just a piece of the TextEditor class for JRuby, but itturns out the whole thing fits on one slide. The text= method, which took up an entire slide inWindows, is just four lines here, thanks to the Jemmy GUI test library from NetBeans.


15/27

Now on to OS X. Of course there's AppleScript, but not every app exposes an AppleScript interface.Fortunately, you can turn on a setting in your preferences that will let control any app through itsmenus and buttons through the System Events interface.


16/27

script.

application("System Events").process("TextEdit").

keystroke!("b")

tellapplication "System Events"

tell process "TextEdit"

keystroke "b"

endtell

endtell

We can send AppleScript to the OS X command line pretty easily, but building the script up first canbe cumbersome. All the commands except the last one start with "tell," and afterward there's anentire chain of "end tell" lines.

But with a little Rub ma ic, we can enerate that lon scri t from a little more terse startin oint.


17/27

class AppleScript

def initialize

@commands = [] end

def method_missing(name, *args)

arg = args[0]

arg = '"' + arg + '"' ifString === arg

command = name.to_s.chomp('!').gsub('_', '')

@commands


18/27

class AppleScript

def to_s

inner = @commands[-1]

rest = @commands[0..-2]rest.collect! {|c| "tell #{c}"}

rest.join("\n") +

"\n#{inner}" +

"\nend tell" * rest.length

end

def run!

clauses = to_s.collect do |line|

'-e "' + line.strip.gsub('"', '\"') + '"'

end.join ''

`osascript #{clauses}`.chomp

end

end

The OS X command to run an arbitrary chunk of AppleScript is "osascript." It doesn't like long lines,so we break long commands up into pieces.


19/27

class TextEditor

def initialize

script.application("TextEdit").activate!

end

def text=(string)

select_all

delete

string.split(//).each do |c|

script.

application("System Events").

process("TextEdit").keystroke!(c)

end

end

endNow that we can control Mac programs, we can write the TextEditor class yet again for a newapplication, Apple's TextEdit.

Here's that same text= method, written for simplicity and not speed. We're spawning an osascriptinstance for ever sin le ke stroke -- definitel not somethin ou'd want to do in a real ro ect.


20/27

portability across apps

So far, we've seen one test script that can test three diferent apps: Notepad, FauxPad, and TextEdit.


21/27

portability across platforms

And that same test script runs on Windows, Mac, and other platforms.

So what's next?


22/27

portability across languages

How about portability across languages?

Our top-level test is in RSpec, but the underlying TextEditor object is just as at home in RBehave,another test language built with Ruby.


23/27

Story "exercise basic editor functions",

%(As an enthusiast for editing documents

I want to be able to manipulate chunks of text

So that I can do my job more smoothly) do

# one or more Scenarios here...

end

While RSpec is geared toward small, self-contained examples, RBehave's basic unit of testing is thestory. Stories are typically a little longer and more representative of how a real user would interactwith the program.

For now, we're ust oin to do a strai ht ort from our RS ec test to RBehave.


24/27

Scenario "a chunky document" do

Given "I start with the text", "chunky" do |text|

@editor = TextEditor.new

@editor.text = text

end

When "I cut once and paste twice" do

@editor.select_all @editor.cut

2.times {@editor.paste}

end

Then "the document's text should be", "chunkychunky" do |text|

@editor.text.should == text

end

end

Each Story consists of multiple Scenarios, which are phrased as Given... When... Then.... It's a littlemore verbose, but at least RBehave remembers its scenarios.

So I can reuse the phrase "I start with the text" or "the document's text should be" in other scenarioswithout re eatin the code that im lements it.


25/27

Scenario "a bacon document" do Given "I start with the text", "bacon"

When "I cut and then undo the cut" do

@editor.select_all

@editor.cut @editor.undo

end

Then "the document's text should be", "bacon" end

Like this. The only new behavior we've had to define for this scenario is "I cut and then undo thecut." The other two were already defined on the previous slide.


26/27

For more explorations like these, keep your eyes peeled for my book coming out in early 2008 fromthe Pragmatic Programmers.


27/27

thank youIan Dees - texagon.blogspot.com

All of the source code for this exercise is available on my blog. Thanks for your time.

Documents

Radical Test Portability