Clean & Dirty Acceptance Tests with Cucumber & Watir

���2

iRedeem Testing Automation Strategy

���3

Levels of Automated Testing

���4

Levels of Automated Testing

���4

Levels of Automated Testing

Unit Tests

���4

Levels of Automated Testing

Unit Tests

System Integration Tests

���4

Levels of Automated Testing

Unit Tests

System Integration Tests

Acceptance Criteria Tests

���4

Levels of Automated Testing

Unit Tests

System Integration Tests

Acceptance Criteria Tests

Regression Testing

���4

Levels of Automated Testing

Unit Tests PHP Unit

System Integration Tests

Acceptance Criteria Tests

Regression Testing

���4

Levels of Automated Testing

Unit Tests PHP Unit

System Integration Tests ??

Acceptance Criteria Tests

Regression Testing

���4

Levels of Automated Testing

Unit Tests PHP Unit

System Integration Tests ??

Acceptance Criteria Tests Cucumber & Watir

Regression Testing

���4

Levels of Automated Testing

Unit Tests PHP Unit

System Integration Tests ??

Acceptance Criteria Tests Cucumber & Watir

Regression Testing [Everything]

���5

Disadvantages of UI Automation using Selenium

or Watir

���6

Disadvantages of UI Automation using Selenium or Watir

���6

Disadvantages of UI Automation using Selenium or Watir

Brittle

���6

Disadvantages of UI Automation using Selenium or Watir

Brittle

Time-consuming to Write and Maintain

���6

Disadvantages of UI Automation using Selenium or Watir

Brittle

Time-consuming to Write and Maintain

Time-consuming to Execute

���7

An Aside: Why does execution time matter?

���7

An Aside: Why does execution time matter?

Sagar checks in code and the tests run

���7

An Aside: Why does execution time matter?

Sagar checks in code and the tests run

Bogdan checks in code an hour later. The tests run

���7

An Aside: Why does execution time matter?

Sagar checks in code and the tests run

Bogdan checks in code an hour later. The tests run

Two hours later, the build has broken.

���7

An Aside: Why does execution time matter?

Sagar checks in code and the tests run

Bogdan checks in code an hour later. The tests run

Two hours later, the build has broken.

Who broke it?

���8

Someone would have to bother checking

���9

Disadvantages of UI Automation using Selenium or Watir

Brittle

Time-consuming to Write and Maintain

Time-consuming to Execute

���10

How do we minimize these disadvantages?

���11

How do we minimize these?

���11

How do we minimize these?

1. Reduce the number of tests.

���11

How do we minimize these?

1. Reduce the number of tests.

2. Reduce the coupling between tests and data (or make our own data)

���11

How do we minimize these?

1. Reduce the number of tests.

2. Reduce the coupling between tests and data (or make our own data)

3. Reduce the time it takes to write and modify tests.

���12

1. Business Acceptance Criteria

���13

Business Acceptance Criteria

���13

Business Acceptance Criteria

Core User Paths

���13

Business Acceptance Criteria

Core User Paths

Business-critical Stuff

���13

Business Acceptance Criteria

Core User Paths

Business-critical Stuff

Cannot be allowed to fail

���13

Business Acceptance Criteria

Core User Paths

Business-critical Stuff

Cannot be allowed to fail

Run on CI with every commit

���13

Business Acceptance Criteria

Core User Paths

Business-critical Stuff

Cannot be allowed to fail

Run on CI with every commit

Small and very maintainable

���14

2. Test Data

���15

Test Data

���15

Test Data

Defaults Stored in YAML files

���15

Test Data

Defaults Stored in YAML files

Koinos and Megento Web APIs used

���15

Test Data

Defaults Stored in YAML files

Koinos and Megento Web APIs used

Scenario Independence

���16

3. Abstractions and Data Models

���16

3. Abstractions and Data Models

Data from YAML Files easily available

���17

���18

���19

3. Abstractions and Data Models

Data from YAML Files easily available

���19

3. Abstractions and Data Models

Data from YAML Files easily available

Data Models to maintain state between steps

���20

���21

3. Abstractions and Data Models

Data from YAML Files easily available

Data Models to maintain state between steps

���21

3. Abstractions and Data Models

Data from YAML Files easily available

Data Models to maintain state between steps

And for Comparison

���22

���23

3. Abstractions and Data Models

Data from YAML Files easily available

Data Models to maintain state between steps

And for Comparison

���23

3. Abstractions and Data Models

Data from YAML Files easily available

Data Models to maintain state between steps

And for Comparison

UI automation for often-used Model-tasks

���24

���25

���26

3. Abstractions and Data Models

Data from YAML Files easily available

Data Models to maintain state between steps

And for Comparison

UI automation for often-used Model-tasks

���26

3. Abstractions and Data Models

Data from YAML Files easily available

Data Models to maintain state between steps

And for Comparison

UI automation for often-used Model-tasks

Page Objects to represent pages under test

���27

���28

Exploratory Testing with Cucumber

���29

Exploratory Testing with Cucumber

���29

Exploratory Testing with Cucumber

Needs to be quick

���30

Exploratory Testing with Cucumber

So it will probably be dirty

���31

Separating Concerns

Clean Code Dirty Code

Must always Pass

Business AC

Small Suite

Can switch off

Bugs / Fine-grained

Larger Suite

���32

Exploratory Testing with Cucumber

���33

Given a Bug…

���34

Given a bug…

���34

Given a bug…

1. Write a quick (and brittle) feature file

���35

���36

Given a bug…

1. Write a quick (and brittle) feature file

���36

Given a bug…

1. Write a quick (and brittle) feature file

2. Write some quick (and dirty) step definitions

���37

���38

Given a bug…

1. Write a quick (and brittle) feature file

2. Write some quick (and dirty) step definitions

���38

Given a bug…

1. Write a quick (and brittle) feature file

2. Write some quick (and dirty) step definitions

3. Check it Fails correctly

���38

Given a bug…

1. Write a quick (and brittle) feature file

2. Write some quick (and dirty) step definitions

3. Check it Fails correctly

4. Commit and assign ticket to Devs

���39

When it comes back to FT?

���39

When it comes back to FT?

$ rake t @jira-ird-787

���40

Regression Testing

���41

If a Dirty Test Fails?

���41

If a Dirty Test Fails?

Is it the test, or the product?

���41

If a Dirty Test Fails?

Is it the test, or the product?

Fix the test

���41

If a Dirty Test Fails?

Is it the test, or the product?

Fix the test

Or turn it off!

���42

Separating Concerns

Business AC Given… When… Then…

Dirty ScenariosYAML Data

Data Models EnvConfig Product Category Member

Page Objects DashboardPage MemberLoginPage

ProductDetailsPage etc…

Step Definitions Step Definitions

Dirty Page Objects

���43

Dirty > Clean

���44

Questions?

���45

That’s It