I put on my mink and wizard behat (tutorial)

I put on my mink and

wizard behatQuesting in the world of

front end testing

9:30 Setup9:45 Introduction to Front End Testing10:15 We write some tests10:45 Coffee break11:00 I talk about some more advanced stuff11:30 We write some more tests12:15 We attempt a grid12:30 Q&A12:45 End

Schedule100% chance of incorrectness

$ sudo vim /etc/hosts

192.168.56.101 opencfp.dev

# copy T9AG1x and T9AG1x_*$ cd T9AG1x$ vagrant up$ vagrant ssh$ cd /var/www/opencfp$ sh run.sh

Setup for PracticalVagrant


10.41.6.62 opencfp.dev

# copy just T9AG1x/opencfp$ cd opencfp$ php vendor/behat/behat/bin/behat

No Vagrant

Booking.com@thomas_shone

WE ARE HIRING

Hoare Logic{P} C {Q}

Hodor Logic{P} C {Q}

Why?What's the benefit?

Meet The TeamDon’t feed the druid after midnight

TaskEach test has a different approach

BarbarianQuality Assurance

RangerUnit Test

ClericContinuous Integration

WizardFront End Test

Dreaded Bugbear

Teamwork Wizards are squishy


192.168.56.101 opencfp.dev

# copy T9AG1x and T9AG1x_*$ cd T9AG1x$ vagrant up$ vagrant ssh$ cd /var/www/opencfp$ sh run.sh

Setup for PracticalVagrant


10.41.6.62 opencfp.dev

# copy just T9AG1x/opencfp$ cd opencfp$ php vendor/behat/behat/bin/behat

No Vagrant

The glue

Behat(cucumber syntax)

Mink(browser emulation)

Goutte(web driver)

Selenium(web driver)

Zombie(web driver)

Guzzle(curl)

Selenium RC(java)

Zombie.js(node.js)

Feature: Party harmonyAs a leader, I want to ensure harmony and mutual trust, so that we work as a team

Scenario: Teach members to respect others’ property Given that the Wizard has 10 cookies And the Bard eats 1 cookie Then the Bard mysteriously catches fire

Cucumber SyntaxReadable testing language

class FeatureContext … {/** * @Given that the wizard has :num cookies */public function wizardHasCookies($num) {

// $this->wizard is a pre-existing condition... like syphilis$this->wizard->setNumberOfCookies($num);

}}

and converts it intoFeatureContext.php



Cucumber SyntaxWhat’s missing?

Scenario:Given that the wizard has 10 cookiesAnd the Bard eats 1 cookie

Fire spell fizzled (OutOfManaException)

1 scenario (1 failed)2 steps (1 passed, 1 failed)0m0.03s (14.19Mb)

Remember {P} C {Q}Set your starting states


Background:The Wizard’s fire spell is fully chargedAnd the Bard is currently not on fire


Remember {P} C {Q}Set your starting states

???As a leader, I want to ensure harmony and

mutual trust, so that we work as a team

User storiesAs a <role>, I want to <desire> so that

<benefit>

Front end testing is code coverage for your user stories

User storiesCoverage

Features are your contract with the stakeholders

Contract

Scenarios are the use cases that outline the user story

Scenarios

Legend has it...… that someone once convinced their PO to

write all their front end tests.

class MinkContext … {/** * Clicks link with specified id|title|alt|text. *

* @When /^(?:|I )follow "(?P<link>(?:[^"]|\\")*)"$/ */public function clickLink($link) {

$link = $this->fixStepArgument($link); $this->getSession()->getPage()->clickLink($link); }}

Mink provides...MinkContext.php

OK...Lets drop the metaphor and get to actual

code

$ composer require behat/behat="~3.0,>=3.0.5"

Getting started

$ composer require behat/mink-extension="~2.0"

Behat (cucumber syntax)

Mink (browser emulator)

Web drivers$ composer require behat/mink-goutte-driver="~1.0"$ composer require behat/mink-selenium2-driver="~1.2"

PRACTICAL

$ ./vendor/bin/behat --init

+d features - place your *.feature files here+d features/bootstrap - place your context classes here+f features/bootstrap/FeatureContext.php - place your definitions, transformations and hooks here

InitializeCreate a new test suite

PRACTICAL

use Behat\MinkExtension\Context\MinkContext;

class FeatureContext extends MinkContext … {…

}

ContextFeatureContext.php

PRACTICAL

$ ./vendor/bin/behat -dl

Given /^(?:|I )am on "(?P<page>[^"]+)"$/ When /^(?:|I )reload the page$/ When /^(?:|I )move backward one page$/ When /^(?:|I )move forward one page$/ When /^(?:|I )press "(?P<button>(?:[^"]|\\")*)"$/ When /^(?:|I )follow "(?P<link>(?:[^"]|\\")*)"$/ When /^(?:|I )fill in "(?P<field>(?:[^"]|\\")*)" with "(?P<value>(?:[^"]|\\")*)"$/

ContextWhat does Mink bring to the table?

PRACTICAL

default: suites: default: paths: [ %paths.base%/features/ ] contexts: [ FeatureContext ] extensions: Behat\MinkExtension: base_url: "[your website]" sessions: default: goutte: ~

Configurationbehat.yml

PRACTICAL

Feature: Authentication and authorisationAs a security conscious developer I wish to ensure that only valid users can access our website.

Scenario: Attempt to login with invalid details Given I am on "/login"

When I fill in "email" with "[email protected]" And I fill in "password" with "invalid" And I press "Login" Then I should see "Invalid Email or Password"

Our first featureauth.feature

PRACTICAL

$ ./vendor/bin/behat --config behat.yml features/auth.feature

Scenario: Attempt to login with an invalid accountGiven I am on "/login"When I fill in "email" with "[email protected]"And I fill in "password" with "invalid"And I press "Login"Then I should see "Invalid Email or Password"

1 scenarios (1 passed)5 steps (5 passed)

Victoryoutput

PRACTICAL


Scenario: Attempt to login with invalid details Given I login as "[email protected]" with password "invalid" Then I should see "Invalid Email or Password"

Simplifyauth.feature

PRACTICAL

Scenario: Attempt to login with an invalid accountGiven I login as "[email protected]" with password "invalid"Then I should see "Invalid Email or Password"

1 scenario (1 undefined) /** * @Given I login as :arg1 with password :arg2 */

public function iLoginAsWithPassword($arg1, $arg2) { throw new PendingException();

}

Simplifyoutput

PRACTICAL

class FeatureContext … {/**

* @Given I login as :username with password :password */

public function iLoginAsWithPassword($username, $password) {$this->visit("/login");$this->fillField("email", $username);$this->fillField("password", $password);$this->pressButton("Login");

}}

SimplifyFeatureContext.php

PRACTICAL

Scenario: Attempt to login with an invalid accountGiven I login as "[email protected]" with password "invalid"Then I should see "Invalid Email or Password"

1 scenarios (1 passed)2 steps (2 passed)

Simplifyoutput

PRACTICAL

// Manipulate the current web session$session = $this->getSession();

$session->visit($url);$session->setBasicAuth($user, $password = '');$session->setRequestHeader($name, $value);$session->setCookie($name, $value = null);$session->getCookie($name);$session->getCurrentUrl();$session->reload();$session->back();

SessionBehat\Mink\Session

SIDE NOTE

Page

// Navigate and manipulate the current page in a selector style$page = $this->getSession()->getPage();

$page->find($selectorType, $selector); // 'css', '.class-name'$page->findById($id);$page->hasLink($locator);$page->clickLink($locator);$page->fillField($locator, $value);$page->hasSelect($locator);$page->selectFieldOption($locator, $value, $multiple = false);$page->hasTable($locator);

Behat\Mink\Element\DocumentElementSIDE NOTE

Driver

// Access the web driver directly via xpaths$driver = $this->getSession()->getDriver();

$xpath = '//html/body/table/thead/tr/th[first()]'$driver->blur($xpath);$driver->focus($xpath);$driver->mouseOver($xpath);$driver->isVisible($xpath);$driver->dragTo($sourceXpath, $destinationXpath);// Modifier could be 'ctrl', 'alt', 'shift' or 'meta'$driver->keyPress($xpath, $char, $modifier = null);

Behat\Mink\Driver\DriverInterfaceSIDE NOTE


Scenario: Attempt to register a new user Given I am on "/signup"

When I fill in "email" with "[email protected]" And I fill in "password" with "valid" And I fill in "password2" with "valid" And I fill in "first_name" with "some"

And I fill in "last_name" with "guy" And I press "Create my speaker profile" Then I should see "You’ve successfully created your account"

Our first hurdleThis ones easy, you do…. oh….

PRACTICAL

Migration and seedingDoctrine, Propel, Laravel, Phinx

$ composer require robmorgan/phinx="~0.4"

Phinx to the rescueInstall

$ php vendor/bin/phinx initPhinx by Rob Morgan - https://phinx.org. version 0.4.3Created ./phinx.xml

Configuration

$ php vendor/bin/phinx create InitialMigration

Creating

SIDE NOTE

#!/usr/bin/env bashDATABASE="opencfp"mysql -e "DROP DATABASE IF EXISTS $DATABASE" -uroot -p123mysql -e "CREATE DATABASE $DATABASE" -uroot -p123vendor/bin/phinx migratevendor/bin/behat

It’s a bit extremerun-behat-test.sh

PRACTICAL

SAVEPOINT identifier;

# Run tests

ROLLBACK TO SAVEPOINT identifier;RELEASE SAVEPOINT identifier;

Transaction/RollbackRoll your own solution

Activation emails?smtp-sink, FakeSMTP, etc

# Stop the currently running servicesudo service postfix stop# Dumps outgoing emails to file as "day.hour.minute.second"smtp-sink -d "%d.%H.%M.%S" localhost:2500 1000 &

vendor/bin/behat

smtp-sinkrun-behat-test.sh

Or….you could just read the activation code from

the database directly

class DatabaseContext {public function __construct($dsn, $user, $pass) {

$this->dbh = new PDO($dsn, $user, $pass);}/** * @When /^there is no user called :user$/ */public function removeUser($user) {

$this->dbh->prepare("DELETE FROM `users` WHERE username=?")->query([$user]);

}}

A new contextDatabaseContext.php

SIDE NOTE

default: suites: default: paths: [ %paths.base%/features/ ] contexts:

- FeatureContext- DatabaseContext:

- mysql:host=localhost;dbname=opencfp- root- 123

Configurationbehat.yml

SIDE NOTE

Or….actually send the email and read it via SMTP

How far is too far?What are your priorities?

Taking it too farTrue story

// Make sure your server and your behat client have the same time set// Share the secret key between the two. The code should be valid for// 30 second periods$code = sha1($secret_key . floor(time() / 30));if ($request->get("code") === $code) {

// Bypass captcha}

Easier waySimple but safe bypass

Our first talkSet the stage

Feature: Manage paper submissions In order to ensure that speakers can submit their papers As an speaker I need to be able to manage my own submissions

Background:There is a user called "[email protected]" with password "secrets"I login as "[email protected]" with password "secrets"

Scenario: Add a new talk to our submissions...

PRACTICAL

Our first talkTalk submission in 3, 2, 1...

Scenario: Add a new talk to our submissions Given I am on "talk/create" And I fill in the following: | title | Behat Talk | | description | Awesome | | type | regular | | category | testing |

| level | mid | And I check "desired" And I press "Submit my talk!" Then I should see "Success: Successfully added talk."

PRACTICAL

Tyranny of JavaScriptDeleting a talk

Well that won’t worktalks.feature

Feature: Manage paper submissions In order to ensure that speakers can submit their papers As an speaker I need to be able to manage my own submissions

Scenario: Delete a talkGiven create a talk called "Behat Talk"And I am on "/dashboard"When I follow "Delete"And I should not see "Behat Talk Changed"

The text "Behat Talk Changed" appears in the text of this page, but it should not. (Behat\Mink\Exception\ResponseTextException)

// Guzzle using web scraperbehat/mink-goutte-driver

// Java-based distributed browser workers (support JavaScript)behat/mink-selenium2-driverbehat/mink-sahi-driver

// node.js headless browser proxy (support JavaScript)behat/mink-zombie-driver

DriversSome take the scenic route

default: # … extensions: Behat\MinkExtension: base_url: "[your website]" sessions: # … javascript: selenium2:

browser: "firefox"wd_host: http://[ip-address-of-host]:4444/wd/hub

ConfigurationSetting up for Selenium

PRACTICAL

$ java -jar selenium-server-standalone-2.*.jar

Selenium

@javascript # Or we could use @selenium2Feature: Manage paper submissions In order to ensure that speakers can submit their papers As an speaker I need to be able to manage my own submissions

Start Selenium Server

Specify javascript requirement

PRACTICAL

$ ./vendor/bin/behat --tags speaker,talk

TagsRun specific tags

@speakerFeature: Manage paper submissions In order to ensure that speakers can submit their papers As an speaker I need to be able to manage my own submissions

@talk Scenario: Create a new talk Given I am logged in as a speaker ...

SIDE NOTE

Feature: Submitting and managing talks As a speaker I wish be able to submit talks so I can get a chance to talk at a conference.

@javascript Scenario: Delete a talk

Given create a talk called "Behat Talk"And I am on "/dashboard"When I fill "Delete"And I accept alertsAnd I should not see "Behat Talk"

Enable JavaScripttalks.feature

PRACTICAL

Run as JavaScripttalks.feature

Feature: Submitting and managing talks As a speaker I wish be able to submit talks so I can get a chance to talk at a conference.

Scenario: Delete a talkGiven create a talk called "Behat Talk"And I am on "/dashboard"When I follow "Delete"And I accept alertsAnd I should not see "Behat Talk"

LIVE DEMO

Scenario: Edit a talkGiven I am on "/dashboard"And I remember "tr[id^='talk']" content as "Title"When I follow "Edit"And I fill in "title" with "New Title"And I press "Update my talk!"Then I should see "New Title"And I should not see "memory:Title"

TransformationsSometimes we need to remember

PRACTICAL

CSS SelectorsDrives are just like browser, no one ever

supports everything properly...

SIDE NOTE

/** * @Transform /^memory:(.*)$/ */public function fromMemory($key) {

if (!isset($this->memory[$key])) {throw new LogicException("Entry $key does not exist");

}return $this->memory[$key];

}

TransformationsFeatureContext.php

PRACTICAL

/** * @Given /^I remember "(.*)" content as "(.*)"$/ */public function rememberContentOf($selector, $key) {

$e = $this->getSession()->getPage()->find("css", $selector);if (!is_object($e)) {

throw new LogicException("Element $selector not found"); }

$value = $e->getValue() ? $e->getValue() : $e->getText();$this->memory[$key] = $value;

}

TransformationsFeatureContext.php

PRACTICAL

class FeatureContext … {public function takeAScreenshotCalled($filename) { $driver = get_class($this->getSession()->getDriver());

if ($driver == 'Behat\Mink\Driver\Selenium2Driver') {$ss = $this->getSession()->getScreenshot();file_put_contents($filename, $ss);

}}

}

ScreenshotFeatureContext.php

PRACTICAL

Advanced Usagewith extra bells and whistles

use Behat\Behat\Hook\Scope\AfterFeatureScope; // @AfterFeature \AfterScenarioScope; // @AfterScenario \AfterStepScope; // @AfterStep \BeforeFeatureScope; // @BeforeFeature \BeforeScenarioScope; // @BeforeScenario \BeforeStepScope; // @BeforeStep \FeatureScope; // @Feature \ScenarioScope; // @Scenario \StepScope; // @Step

HooksListen in close

class FeatureContext … {/** * @AfterScenarioScope */public function afterScenario(AfterScenarioScope $scope) {

$scenario = $scope->getScenario()->getTitle();$filename = make_safe_filename($scenario);// Take a screenshot and put it on a dashboard somewhere$this->takeAScreenshotCalled($filename);

}}

HooksFeatureContext.php

PRACTICAL

class FeatureContext … {/** * @AfterStep */public function afterStep(AfterStepScope $scope) {

$code = $event->getTestResult()->getResultCode();if ($code == TestResult::FAILED) {

// Take a screenshot}

}}

HooksFeatureContext.php

PRACTICAL

class FeatureContext … {/** * @Given /^(?:I )wait for AJAX to finish$/ */public function iWaitForAjaxToFinish() {

$this->getSession()->wait(5000, "(0 === jQuery.active)");}

}

AJAXThe waiting game

PRACTICAL

class FeatureContext … {/** * @Given /^(?:I )press the letter :l$/ */public function iPressTheLetter($l) {

$s = "jQuery.event.trigger({type:'keypress', which:'$l'});"; $this->getSession()->evaluateScript($s);}

}

Raw javascriptThere might be a valid use case...

PRACTICAL

default: suites: web: paths: [ %paths.base%/features/web ] contexts: [ BaseContext, WebContext ] api: paths: [ %paths.base%/features/api ] contexts: [ BaseContext, ApiContext ]

ConfigurationMultiple contexts

default: suites: admin: paths: [ %paths.base%/features/web ] contexts: [ BaseContext, AdminContext ] filters: role: admin speaker: paths: [ %paths.base%/features/web ] contexts: [ BaseContext, SpeakerContext ] filters: tags: @speaker

ConfigurationGrouping and filtering

$ ./vendor/bin/behat --suite admin

SuitesRun a specific suite

Feature: Managing the CFP In order to ensure that speakers can submit their papers As an admin I need to be able to open the call for papers

$ ./vendor/bin/behat --suite speaker

SuitesRun a specific suite

@speakerFeature: Submitting to the CFP In order to ensure that the conference has papers As an speaker I need to be able to submit papers

$ java -jar selenium-server-standalone-2.*.jar -role hub

Selenium Grid

$ java -jar selenium-server-standalone-2.*.jar -role node -hub http://10.41.6.62:4444/grid/register

Start the grid

Add a node

LIVE DEMO

default: extensions: Behat\MinkExtension: sessions: javascript: selenium2: wd_host: "http://127.0.0.1:4444/wb/hub" capabilities: version: ""

ConfigurationBecause magic...

LIVE DEMO

http://127.0.0.1:4444/wb/hub

Complex UsersDealing with very complex user states

Feature: Show relevant promotions to non-paying activated customers

Scenario: Show promotion pricing to referred clients from the EUGiven I create a user with: | email-activated | | is-EU-member | | is-referred-client | | NOT has-made-purchase |Then ...

SetupWhat attributes does our user have?

interface AttributeInterface {// Get the required attributes to have this attributepublic function getDependencies();// Does this user have this attribute?public function has(MinkContext $context);// Allocate this attribute to the userpublic function allocate(MinkContext $context);// Attempt to remove this attribute from the userpublic function remove(MinkContext $context);

}

AttributesAttributeInterface.php

class IsEUMember extends AttributeInterface {public function getDependencies() { return ["email-activated"]; }

public function has(MinkContext $context) {$context->visit("/profile");$field = $context->getSession()

->getPage()->findField("country");

return in_array($field->getValue(), $this->eu_countries);}

}

AttributesIsEUMember.php

// ...public function allocate(MinkContext $context) {

$context->visit("/profile");$context->selectOption("country", "Netherlands");$context->pressButton("Update");

}

public function remove(MinkContext $context) {$context->visit("/profile");$context->selectOption("country", "UK"); // Future-proofing$context->pressButton("Update");

}

AttributesIsEUMember.php

// This class must handle dependency conflicts by examining the// dependencies of each with/without attributeclass Request {

protected $attributes = [];public function with(AttributeInterface $feature);public function without(AttributeInterface $feature);// List of attributes (including dependents) requiredpublic function getWith();// List of attributes (including dependents) that must be removedpublic function getWithout();

}

RequestRequest.php

class User {public function __construct(MinkContext $context, $request) {

$this->createNewUser($context);foreach ($request->getWith() as $attr) {

$attr->allocate($content);}foreach ($request->getWithout() as $attr) {

$attr->remove($content);}

}public function canSupport($request);

}

UserUser.php

use Behat\Gherkin\Node\TableNode;class FeatureContext … {

/** * @Given /^(?:I )create a user with:$/ */public function iCreateAUser($type, TableNode $table) {

$attributes = $table->getRowsHash();$request = new Request();// Build using $request->with(...) & $request->without(...);$user = new User($context, $request);

}}

TableNodeHandling a table

Some days everything is made of glass

Common Gotchas

PermutationsThe reason why testing is painful

Expect breakagesAnd that’s a good thing

Speed vs CoverageFind the right balance

Keep Selenium updatedBrowsers change faster than fashion trends

Behat documentshttp://docs.behat.org points to v2.5 docs but

doesn’t tell you.Use http://docs.behat.org/en/v3.0/

http://docs.behat.org

http://docs.behat.org/en/v3.0/

Questions?or ask me later via @thomas_shone

Feature: Administration of talk submissions In order to be able to manage a conference, as an admin, I should be able to manage talks

Background:Given there is a speaker registered as "[email protected]" with a password "secrets"And User "[email protected]" has admin rightsAnd I login as "[email protected]" with password "secrets"

Scenario: Approve a submitted paper Scenario: Decline a submitted paper

Final TaskApply what you know

HintAdmin rights are granted by a CLI too

@javascriptFeature: Admin In order to be able to manage a conference, as an admin, I should be able to manage talks

Background: Given there is a speaker registered as "[email protected]" with a password "secrets" And User "[email protected]" has admin rights And I login as "[email protected]" with password "secrets"

Scenario: Approve a submitted paper Given I create a talk called "New Talk" And I am on "/admin/talks" And I click on element ".js-talk-select" Then I should see an ".check-select--selected" element

Scenario: Reject a submitted paper Given I am on "/admin/talks" And I click on element ".check-select--selected" Then I should not see an ".check-select--selected" element

Model? Answer


* @Given User :email has admin rights */

public function userHasAdminRights($email){

exec("php -f bin/opencfp admin:promote " . escapeshellarg($email));}

/** * @Given I click on element :selector */

public function iClickOnElement($selector) { $element = $this->getSession()->getPage()->find("css", $selector); if (!is_object($element)) { throw new LogicException("Element $selector not found"); } $this->getSession()->evaluateScript('$("' . $selector . '").click();');

}}

Model? Answer


* @Given I create a talk called :title */

public function iCreateATalkCalled($title) { $this->visit("/dashboard"); $this->clickLink("Submit a talk"); $this->fillField("title", $title); $this->fillField("description", "Awesome"); $this->fillField("type", "regular"); $this->fillField("category", "testing"); $this->fillField("level", "mid"); $this->checkOption("desired"); $this->pressButton("Submit my talk!");}

}

Model? Answer

Thank youPhoto from Flickr by John Morey, TrojanRat, Gerry Machen, USFS Region 5,

Peregrina Tyss and Thomas Hawk

Technology

I put on my mink and wizard behat (tutorial)