How to improve our acceptance tests - Pyccuracy VS Splinter

How to improve our acceptance tests !

Fernando Sandes

Pyccuracy vs

What is better?

It`s not (only)

the technology about

It`s an ideological

difference

The problem is that differently of cucumber and lettuce, pyccuracy is aimed to web testing, with pre-defined "step definitions" (pyccuracy uses a different nomenclature), and when I was using it I found extremely hard to maintain the tests as the test suite grows, it's a hell!

lettuce

@gabrielfalcao Lettuce`s creator

“So I wrote lettuce to make it super flexible so that I could write my own definitions in a super

easy way, and never couple my tests to robotic step definitions.”

lettuce

@gabrielfalcao Lettuce`s creator

Pyccuracy

ROBOTIC STEP

DEFINITIONS

ROBOTIC STEP DEFINITIONS

Given I am at the login page

And I fill in the login information with "test" and password "1234"

When I click on the login button

Then I see I'm logged in as a normal user

Given I fill the field "#username" with "test"

And I fill the field "#password" with "1234"

When I click on the button 'input[type="submit"]'

Then I see the page contains the text "Hello Test User!"

ROBOTIC STEP DEFINITIONS

Given I am at the login page

And I fill in the login information with "test" and password "1234"

When I click on the login button

Then I see I'm logged in as a normal user

Given I fill the field "#username" with "test"

And I fill the field "#password" with "1234"

When I click on the button 'input[type="submit"]'

Then I see the page contains the text "Hello Test User!"

“You can see how your business it better defined here”

DONT`T BE THE

BOILING FROG

The presence of step definitions leads people down a very

bad path. While it gives people a head start in writing scenarios it also tricks people into writing scenarios are extremely

verbose and fragile. People don't always realize this, so they continue cranking out lots of crap scenarios based on pre-steps. This is like the boiling frog story

!

IT`S ABOUT THE DOMAIN !

The “domain” is defined by the value the stakeholders and users hope to achieve with the software.

This can be booking a ticket or sharing pictures with friends or an infinite number of activities. Clicking links and buttons or filling in text fields has nothing (directly) to do with the domain.

IS BUROCRATIC !

Pyccuracy

a little more…

Imperative

VS Declarative

Scenarios in User Stories

Imperative The imperative style uses highly reusable granular steps which outlines much of the

user interface.

This binds the scenario to that interface.

Scenario: Successful login Given a user "Aslak" with password "xyz" And I am on the login page And I fill in "User name" with "Aslak" And I fill in "Password" with "xyz" When I press "Log in" Then I should see "Welcome, Aslak“

Imperative The imperative style uses highly reusable granular steps which outlines much of the

user interface.

This binds the scenario to that interface.

Scenario: Successful login Given a user "Aslak" with password "xyz" And I am on the login page And I fill in "User name" with "Aslak" And I fill in "Password" with "xyz" When I press "Log in" Then I should see "Welcome, Aslak“

Scenario: User is greeted upon login Given the user "Aslak" has an account When he logs in Then he should see "Welcome, Aslak"

Declarative

Imperative The imperative style uses highly reusable granular steps which outlines much of the

user interface.

This binds the scenario to that interface.

Scenario: Successful login Given a user "Aslak" with password "xyz" And I am on the login page And I fill in "User name" with "Aslak" And I fill in "Password" with "xyz" When I press "Log in" Then I should see "Welcome, Aslak“

Scenario: User is greeted upon login Given the user "Aslak" has an account When he logs in Then he should see "Welcome, Aslak"

Declarative

Imperative

When we’re describing other parts of the system that require login, the login details like user name, password

and how to log in are only distracting. That is why

we make it a one-liner.

Declarative

Imperative Declarative

Imperative Declarative

COMMUNICATION IS THE GOAL!

To use a BDD tool "right", one has to understand

that the main goal is primarily about communication and secondarily about testing.

COMING BACK…

And

technology about

SPLINTER

Splinter was not created to be another acceptance

tool, but an abstract layer over other tools, its goal is provide a unique API that make

acceptance testing easier and funnier :)

SPLINTER

A LOT OF OPTIONS !

OR NOT HEADLESS ? HEADLESS

HEADLESS

FAST CHEAP HARD TO DEBUG IT`S NOT A BROWSER!

NOT HEADLESS

SLOW EASY TO DEBUG USE THE BROWSER

NOT HEADLESS

HEADLESS

SPLINTER

LOCAL

SERVER

LOCAL

SERVER

HEADLESS

LOCAL

SERVER

HEADLESS

NOT HEADLESS

SPLINTER and more…

http status code

multi webdrivers

css ,xpath, tag, name selectors

support to iframe and alert

execute javascript

work's with ajax and async javascript

SPLINTER Browser

• visit(‘url’)

• reload()

• quit()

• html

• url

• title

SPLINTER Searching elements

• find_by_css(‘.classe’)

• find_by_id(‘meuid’)

• find_by_xpath(‘//body’)

• find_by_tag(‘a’)

• find_by_name(‘search’)

SPLINTER Forms

• fill(‘name’, ‘value’)

• attach_file(‘name’, ‘/path/to/file.txt’)

• check(‘name’) #checkbox

• choose(‘name’, ‘value’) #radio button

• select(‘name’, ‘option’)

MORE

TESTS

SIKULI

http://sikuli.org/index.shtml

NEEDLE

from needle.cases import NeedleTestCase

class GoogleTest(NeedleTestCase):

def test_footer(self):

self.driver.get('http://www.google.com')

e = self.driver.find_element_by_id('fctr')

self.assertScreenshot(e, 'google-footer')

self.assertEqual(e.get_computed_property('font-size'), '13px')

http://needle.readthedocs.org

#AUTOMATED TESTS FOR CSS