Automated Testing in WordPress, Really?!

Automated Testing in

WordPress,Really?!

Rate this talk: https://joind.in/10115

#dc4d - Automated Testing in WordPress with @ptahdunbar

Ptah (Pirate) Dunbar

● Started with WordPress and PHP in ‘05

● Contributing developer toWordPress, BuddyPress, bbPress

● Full stack Web Developer

● Architect at LiveNinja.com

● WPMIA co-organizer and SoFloPHP member

☠ Became Pirate Dunbar

● Understand automated testing concepts,ideas and best practices.

● Learn PHPUnit basics and the WordPress testsuite.

● Resources and homework

Agenda

In one hour

WordPress powers

1 in 5 websites

source: http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites

WordPress community

28,5102,177

source: http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites

“The result is that a lot of the plugins are written in poor code and turn out to be poorly compatible with other plugins”

— Yoast

http://yoast.com/plugin-future/

● WP_DEBUG

● var_dump();

● print_r();

● error_log();

● debug_backtrace();

Manual Testing

Pull out the tools

● WP_DEBUG

● var_dump();

● print_r();

● error_log();

Manual Testing

Pull out the tools

Ad-hoc & Temporary

● WP_DEBUG

● var_dump();

● print_r();

● error_log();

Manual Testing

SLOW & Error Prone

Pull out the tools

● WP_DEBUG

● var_dump();

● print_r();

● error_log();

Pull out the tools

Manual Testing

Doesn’t scale

A scripted process that invokes your app to test

features and compares the outcome with expected

results.

Automated Testing

Persistent var_dumps();

Automated Testing

Better than checking the logs

Automated Testing

The Bigger Picture

Automated Testing

Continuous Integration

Continuous Delivery

Continuous InspectionReleasing early, releasing often

Phingvagrant

Getting started

Automate Testing

There are so many

Frameworks

CHOOSE YOUR FRAMEWORK

PHPUnithttp://phpunit.de/manual/

Sebastian Bergmann

vim composer.json && composer update

{ "require-dev": {

"phpunit/phpunit": "3.7.*",

"phpunit/phpunit-selenium" : "*",

http://getcomposer.org

$>./vendor/bin/phpunit

PHPUnit

Terminology

● Test CaseA set of conditions that you set up in order to assert expected outcome.

PHPUnit

Terminology

PHPUnit

Terminology

● Test ClassA collection of test cases, extends PHPUnit

PHPUnit

Terminology

● Test ClassA collection of test cases, extends PHPUnit

● Test SuiteA collection of test classes

<?php// test classclass CalTest extends PHPUnit_Framework_TestCase{

// test case public function testAddReturnsSumOfTwoPositiveIntegers() { // assert stuff. }}

PHPUnit

TEST CLASS

<?php// test classclass CalTest extends PHPUnit_Framework_TestCase{

// test case public function testAddReturnsSumOfTwoPositiveIntegers() { // assert stuff. }}

PHPUnit

TEST CLASS

plugin/loader.phpincludes/

admin.phpapi.php…

phpunit.xmltests/

adminTest.phpApiTest.php…

PHPUnit

plugin/loader.phpincludes/

admin.phpfunctions.php…

phpunit.xmltests/

integration/…

acceptance/…

PHPUnit

<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="vendor/autoload.php"> <testsuites> <testsuite name="tests"> <directory suffix="Test.php">tests/</directory> </testsuite> </testsuites></phpunit>

PHPUnit

phpunit.xml - configuration file for PHPUnit

PHPUnit

phpunit.xml

Configure your test suite location

<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="vendor/autoload.php"> <testsuites> <testsuite name="integration"> <directory suffix="Test.php">tests/integration</directory> </testsuite> <testsuite name="acceptance"> <directory suffix="Test.php">tests/acceptance</directory> </testsuite> </testsuites></phpunit>

PHPUnit

phpunit.xml

Configure your test suite location

PHPUnit

phpunit.xml

Bootstrap file is included before any tests run

Assertions

PHPUnit

Explicitly check expected outcomeagaisnt actual outcome.

Assertions

PHPUnit

Explicitly check expected outcomeagaisnt actual outcome.

$this->assertTrue(condition);

Arrange, Act, Assert

PHPUnit

function testThatItsTestingTime(){

PHPUnit

3. Assert (check for the expected behavior)

PHPUnit

2. Act (call the method/trigger the action)

PHPUnit

1. Arrange (the context/dependencies)

2. Act (call the method/trigger the action)

PHPUnit

Example

PHPUnit

<?phpclass CalTest extends PHPUnit_Framework_TestCase{ public function testAddReturnsSumOfTwoPositiveIntegers() { // Arrange

// Act

// Assert

PHPUnit

plugin/tests/unit/calTest.php

// Act

// Assert

PHPUnit

// Act

// Assert $this->assertEquals(3, $result);

PHPUnit

// Act $result = $calculator->add(1,2);

PHPUnit

$calculator = new Calculator();

PHPUnit

./vendor/bin/phpunit

PHPUnit

Time: 148ms, Memory: 2.75Mb

OK: (1 test, 1 assertions)

PHPUnit

Failed asserting that 4 equals 3

<?phpclass UserServiceTest extends PHPUnit_Framework_TestCase{ public function testPersistWithValidUserdataReturnsUserObject() {

// Arrange

// Act

// Assert

//…}

PHPUnit

plugin/tests/unit/LiveNinja/User/ServiceTest.php

// Arrange

// Act

// Assert$this->assertTrue($user instanceof ‘\LiveNinja\User\Entity’);

//…}

PHPUnit

// Arrange

// Act$user = $service->persist($validUserdata);

//…}

PHPUnit

// Arrange$service = new \LiveNinja\User\Service;$validUserdata = [...];

//…}

PHPUnit

//…}

PHPUnit

//…}

PHPUnit

//…}

PHPUnit

// Assert$this->assertTrue($user instanceof ‘\LiveRacoons\User\Entity’);

//…}

PHPUnit

//…}

PHPUnit

//…}

PHPUnit

Failed asserting that false equals true

ASSERTIONSAppendix: http://phpunit.de/manual/3.7/en/appendixes.assertions.html

PHPUnit

Use the most specific assertion possible

● assertTrue();

● assertEquals();

● assertContains();

● assertGreaterThan();

● assertNotNull();

● assertFalse();

● assertNotEquals();

● assertContainsOnly();

● assertLessThan();

● assertSame();

PHPUnit

There was 1 failure:

1) Tests_Basic::test_readmereadme.html's version needs to be updated to 3.9.Failed asserting that '3.8' matches expected '3.9'.

/private/tmp/wordpress-tests/tests/phpunit/tests/basic.php:29

PHPUnit

1) Tests_User_Author::test_get_the_authorFailed asserting that two objects are equal.--- Expected+++ Actual@@ @@ WP_User Object ( 'data' => stdClass Object (- 'ID' => '3'- 'user_login' => 'User 1'- 'user_pass' => '$P$BpIqOzMNRGZNy9qxKL/3cCDCMe85o2.'- 'user_nicename' => 'user-1'- 'user_email' => 'user_2@example.org'+ 'ID' => '2'+ 'user_login' => 'test_author'+ 'user_pass' => '$P$BUdMebxEjJ23.6LbH9ujvVUFBsUuZv/'+ 'user_nicename' => 'test_author'+ 'user_email' => 'user_1@example.org' 'user_url' => '' 'user_registered' => '2013-12-20 15:31:01' 'user_activation_key' => '' 'user_status' => '0'- 'display_name' => 'User 1'+ 'display_name' => 'test_author' )- 'ID' => 3+ 'ID' => 2 'caps' => Array (- 'subscriber' => true+ 'author' => true ) 'cap_key' => 'wptests_capabilities' 'roles' => Array (- 0 => 'subscriber'+ 0 => 'author' ) 'allcaps' => Array ( 'read' => true 'level_0' => true- 'subscriber' => true+ 'upload_files' => true+ 'edit_posts' => true+ 'edit_published_posts' => true+ 'publish_posts' => true+ 'level_2' => true+ 'level_1' => true+ 'delete_posts' => true+ 'delete_published_posts' => true+ 'author' => true ) 'filter' => null )

/private/tmp/wordpress-tests/tests/phpunit/tests/user/author.php:50

PHPUnit

1) Tests_User_Author::test_get_the_authorFailed asserting that two objects are equal.--- Expected+++ Actual@@ @@ WP_User Object ( 'data' => stdClass Object (- 'ID' => '3'- 'user_login' => 'User 1'- 'user_pass' => '$P$BpIqOzMNRGZNy9qxKL/3cCDCMe85o2.'- 'user_nicename' => 'user-1'- 'user_email' => 'user_2@example.org'+ 'ID' => '2'+ 'user_login' => 'test_author'+ 'user_pass' => '$P$BUdMebxEjJ23.6LbH9ujvVUFBsUuZv/'+ 'user_nicename' => 'test_author'+ 'user_email' => 'user_1@example.org' 'user_url' => '' 'user_registered' => '2013-12-20 15:31:01' 'user_activation_key' => '' 'user_status' => '0'- 'display_name' => 'User 1'+ 'display_name' => 'test_author' )- 'ID' => 3+ 'ID' => 2 'caps' => Array (- 'subscriber' => true+ 'author' => true ) 'cap_key' => 'wptests_capabilities' 'roles' => Array (- 0 => 'subscriber'+ 0 => 'author' ) 'allcaps' => Array ( 'read' => true 'level_0' => true- 'subscriber' => true+ 'upload_files' => true+ 'edit_posts' => true+ 'edit_published_posts' => true+ 'publish_posts' => true+ 'level_2' => true+ 'level_1' => true+ 'delete_posts' => true+ 'delete_published_posts' => true+ 'author' => true ) 'filter' => null )

/private/tmp/wordpress-tests/tests/phpunit/tests/user/author.php:50

// Assert$this->assertTrue($user instanceof ‘\LiveRacoon\User\Entity’);

//…}

PHPUnit

// Assert$this->assertInstanceOf(‘\LiveNinja\User\Entity’, $user);

//…}

PHPUnit

//…}

PHPUnit

//…}

PHPUnit

<?phpclass UserServiceTest extends PHPUnit_Framework_TestCase{ public function testPersistWithInvalidUserdataReturnsWPError() {

// Arrange$service = new \LiveNinja\User\Service;$invalidUserdata = [];

// Act$user = $service->persist($invalidUserdata);

// Assert$this->assertInstanceOf(‘WP_Error’, $user);

//…}

PHPUnit

WordPress Testsuite

WordPress with Testshttp://develop.svn.wordpress.org/trunk/

1858 Tests, 8611 Assertions, 2.59 minutes

WordPress Testsuite

Getting started

WordPress Testsuite

<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="tests/bootstrap-wp.php"> <testsuites> <testsuite name="tests"> <directory suffix="Test.php">tests/</directory> </testsuite> </testsuites></phpunit>

WordPress Testsuite

<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="tests/bootstrap-wp.php"> <testsuites> <testsuite name="tests"> <directory suffix="Test.php">tests/</directory> </testsuite>

<testsuite name="integration"> <directory suffix="Test.php">integration/</directory> </testsuite> </testsuites></phpunit>

plugin/tests/integration/PluginTest.php

<?phpclass PluginTest extends PHPUnit_Framework_TestCase{ // test cases...}

WordPress Testsuite

plugin/tests/integration/PluginTest.php

<?phpclass PluginTest extends WP_UnitTestCase{ // test cases...}

WordPress Testsuite

$>./vendor/bin/phpunit

WordPress Testsuite

Run Testsinside of an isolated

WordPress Environment

WordPress Testsuite

ConfigureWordPressOptions

$GLOBALS['wp_tests_options'] = ['active_plugins' => [

'hello.php',...

],'current_theme' => 'kubrick',...

bootstrap.php

WordPress Testsuite

ConfigureWordPress

Includesfunction __muplugins_loaded(){

// code and stuff.require_once 'env-debug.php';

}tests_add_filter('muplugins_loaded', '__muplugins_loaded');

bootstrap.php

WordPress Testsuite

● Navigate to site URL (Updates globals)$this->get_url($url);

● Test WP_Query for Conditionals (is_page, is_single, is_404)$this->assertQueryTrue($arg1, $arg2, ...);

● Test for Errors$this->assertWPError($thing);

● Genereate WordPress data fixtures$this->factory->post->create_and_get();

$this->factory->comment->create_post_comments($pid, 100);

$this->factory->user->create_many(5);

$this->factory->blog->create();

and more…

WordPress Testsuite

plugin/tests/integration/WPCustomizationTest.php

WordPress Testsuite

<?phpclass WPCustomizationTest extends WP_UnitTestCase{

// test casesfunction testRedirectForDateBasedPermalinks(){

// Arrange

// Act

// Assert

WordPress Testsuite

// Arrange

// Act

// Assert$this->assertQueryTrue( 'is_404' );

WordPress Testsuite

// Arrange$customWP = new WPCustomization;$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);

// Act$customWP->deprecate_unused_pages();$this->go_to('/2007/');

WordPress Testsuite

/** * @dataProvider getRequiredPlugins */function testAllRequiredPluginsAreActive($plugin){

// Assert$this->assertTrue( is_plugin_active($plugin),

sprintf('%s is not activated.', $plugin) );}

function getRequiredPlugins(){

return [[‘hello.php’],

WordPress Testsuite

/** * @dataProvider getWPOptions */function testWPOptionSettingsAreConfigured($option_name, $option_value){

// Assert$this->assertSame($option_value, get_option($option_name));

function getWPOptions(){

return [[‘home’, ‘http://example.org/wp/’],[‘siteurl’, ‘http://example.org/’],

WordPress Testsuite

http://www.seleniumhq.org/

plugin/tests/acceptance/ConnectTest.php

Acceptance Testing

<?phpclass ConnectTest extends PHPUnit_Framework_TestCase{ protected function setUp() { }

public function testUserCanLogInViaTwitter() { }}

Acceptance Testing

<?phpclass ConnectTest extends PHPUnit_Extensions_Selenium2TestCase{ protected function setUp() { }

Acceptance Testing

<?phpclass ConnectTest extends PHPUnit_Extensions_Selenium2TestCase{ protected function setUp() { $this->setBrowser("*chrome"); $this->setBrowserUrl("https://wpss.dev/"); }

Acceptance Testing

public function testUserCanLogInViaTwitter() { $this->open("/");

Acceptance Testing

public function testUserCanLogInViaTwitter() { $this->open("/"); $this->click("link=Log in"); $this->waitForPageToLoad("30000");

Acceptance Testing

public function testUserCanLogInViaTwitter() { $this->open("/"); $this->click("link=Log in"); $this->waitForPageToLoad("30000"); $this->click("css=img[alt=\"Twitter\"]"); $this->waitForPageToLoad("30000");

Acceptance Testing

public function testUserCanLogInViaTwitter() { $this->open("/"); $this->click("link=Log in"); $this->waitForPageToLoad("30000"); $this->click("css=img[alt=\"Twitter\"]"); $this->waitForPageToLoad("30000"); $this->assertContains( ‘dashboard’, $this->title() ); }}

Acceptance Testing

public function testUserCanLogInViaTwitter() { $this->open("/"); $this->click("link=Log in"); $this->waitForPageToLoad("30000"); $this->click("css=img[alt=\"Twitter\"]"); $this->waitForPageToLoad("30000"); $this->assertContains( ‘dashboard’, $this->title() ); }}

Acceptance

Selenium IDE Plugin

● Visually navigate throughout your site and generate a PHPUnittest case.

● Download Extension○ http://www.seleniumhq.

org/projects/ide/

● Download PHPUnit Formatter○ https://addons.mozilla.org/en-

US/firefox/addon/selenium-ide-php-formatters/

Acceptance

Selenium IDE Plugin

● Visually navigate throughout your site and generate a PHPUnittest case.

● Download Extension○ http://www.seleniumhq.

org/projects/ide/

● Download PHPUnit Formatter○ https://addons.mozilla.org/en-

US/firefox/addon/selenium-ide-php-formatters/

How can we be confident that our tests

cover everything?

Testing boundaries

● (User) Acceptance Testing

○ Verify that all features are done done.

○ Black-box testing, no knowledge of internals.

Testing boundaries

● Integration Testing

○ Test WordPress settings/configuration;

○ Compatibility between plugins and themes.

Testing boundaries

○ Test WordPress settings/configuration,

○ Compatibility between plugins and themes

● Unit Testing

○ Test class methods and functions in isolation, zero dependencies

○ Does one “behavoir”

Testing boundaries

Verify that all features are done done,

black-box testing, no knowledge of

internals

Test WordPress settings/configuration,

compatibility between plugins and

themes

● Unit Testing

Test class methods and functions in

isolation, zero dependencies,

does one “behavoir”.

Testing boundaries

AcceptanceTesting

Integration Testing

Unit Testing

internals

themes

● Unit Testing

Testing boundaries

AcceptanceTesting

Integration Testing

Unit Testing

internals

themes

● Unit Testing

Testing boundaries

Integration Testing

Unit Testing

AcceptanceTesting

#ATWP // Automated Testing in WordPress // @ptahdunbar

What to tests?

● Test plugin works in various WordPress setups

○ Does it work under multisite?

○ What about a custom content directory?

● Test all code paths in functions and methods

● Test compatiblity between most popular plugins

● Test that default pages exists

What to tests?

● Test for theme support

● Test that post formats contain property elements

● Test any required assets that need to be loaded in

templates

● Test for required elements on a page

● Verify search results template displays search term

● Verify SEO meta tags

1. WordPress APIs

What to not tests?

1. WordPress APIs

2. PHP language features

What to not tests?

1. WordPress APIs

2. PHP language features

3. Third party vendor code

Getting into the groove

● Build out templates

● Build out templates○ Create HTML/CSS

● Build out templates○ Create HTML/CSS○ Identify dynamic elements and their data

structure

● Build out templates○ Create HTML/CSS○ Identify dynamic elements and their data

structure○ Label them and fill them with dummy data

○ Verbally state your trying to do

○ Verbally explain what the code does

○ Do this alone or with a fellow dev :)

What’s Next?

● Download WP Skeleton Family

○ https://github.com/ptahdunbar/wp-skeleton-site

○ https://github.com/ptahdunbar/wp-skeleton-plugin

○ https://github.com/ptahdunbar/wp-skeleton-theme

Get started

“A Walking Skeleton is a tiny implementation of the thinnest possible slice of real functionality that we can automatically

build, deploy and test end-to-end.”

● Art of Unit Testing (.NET)

○ https://leanpub.com/u/royosherove

○ Udemy Five day course

● #GOOS Book (Java)

● XUnit Test Patterns (Java)

● Grumpy Books (PHP)

○ https://leanpub.com/u/chartjes

● Misko Hevery

Resources

● Learn moar PHPUnit features

○ data providers,

○ mocks and stubs

○ wordpress testsuite

● Goal: Write at least 100 assertions!

Homework!

Automated Testing

Automated Testingincreases your productivity

Automated Testingfacilitates more shipping

Automated Testingscales with you

Automated Testingis your professional duty

as a developer

Thank youAutomated Testing in WordPress

Pirate Dunbar@ptahdunbaryarr@piratedunbar.com

Rate this talk:https://joind.in/10115

Automated Testing in WordPress, Really?!

Technology

Automated Testing with SilkTest: Strategies That Really Work

WordPress – Un site utilisant WordPress

SearchBlox WordPress Plugin 0d2fco3ozzrfhhd.cloudfront.net/wordpress/WordPress...SearchBlox now provides new and enhanced version of SearchBlox WordPress plugin for WordPress, one

Strategic traffic assignment model for automated driving ...stad.tudelft.nl/wordpress/wp-content/uploads/2019/05/SP6-Presentat… · 24 Autonome voertuigen op busbanen casus Brainport

| Stockton Wordpress · | Stockton Wordpress

Automated Fraction Re-Analysis Does it really make … · Automated Fraction Re-Analysis Does it ... • 1995 PHD in organic chemistry from the University ... •Possible errors in

Automated WordPress to Joomla Extension: How It Works

CMS2CMS: Automated vBulletin to WordPress Migration Plugin

Really Awesome WordPress Plugins You Should Know About

Automated MediaWiki to WordPress Migration Plugin: How It Works

Logging in - HealthWest Partnershiphealthwest.org.au/.../2014/10/WordPress-Training-February-2015.d… · Web viewHealthWest has an automated booking form. You should select HWP

Wordpress. Wordpress… Open Source – GNU General Public License Wordpress…

Making Simple Things Really Complicated - … Simple Things Really Complicated High Availability for WordPress (or just about anything else) David Smith @dsXLII ... Level-Setting This

Who WordPress Users Really Are

U.S. ARMY COMBAT CAPABILITIES DEVELOPMENT ... - WordPress… · Automated real-time feedback for teams with little or no experience and objective, validated performance assessment

WordPress Automation - 2018 WordCamp Jacksonville · Automated Task Manager IFTTT ifttt.com/wordpress Zapier zapier.com/apps/wordpress/integra tions

Can automated vehicles really make our cities better?endofdriving.org/wp-content/uploads/2017/05/Grush-Niles...motorized, automated vehicles and fleets must subscribe. Bikes are not

DE│EN│FR│IT IS YOUR FACTORY REALLY LEAN? · CPS ®RFID Kanban systems, ... IS YOUR FACTORY REALLY LEAN? ... fully automated production system. Purchasing volume

WordPress Custom Theme Designer, WordPress Customization, WordPress SEO

A WordPress Thesis Theme Review - Is it Really Worth It?