Acceptance Test Driven Development (ATDD) · ATDD Lineage ATDD comes from: Example driven...

Acceptance

Test Driven

Development

(ATDD)Lean-Agile Acceptance

Test-Driven Development (

Ken Pugh, 2011

ATDD

Acceptance tests are

developed collaboratively

between the :

Customer

Developer

tester

Lean-Agile Acceptance Test-Driven Development (2011) Ken Pugh

Acceptance test driven development – involves developing

requirements tests prior to implementation

Chapter 2

ATDD Lineage

ATDD comes from:

Example driven development (EDD)

Behavior-driven development (BDD)

Story-driven development (SDD)

Domain-driven Design (DDD)

Executable acceptance test-driven

development (EATDD)

Lean-Agile Acceptance Test-Driven Development (2011) Ken Pugh

Testing Strategy –

Chapter 3

Testing Matrix Chapter 3

Where

Tests

Run

Positive & Negative Testing

Positive testing – ensure the program works

as expected

Negative testing – check to see that

program does not create unexpected

results

Acceptance Testing –

Chapter 4

Requirements & Tests are

Linked

Tests clarify and amplify requirements

Can’t have a requirement without a test or

vice versa

Tests written after the code is written is a

new requirement or a new detail on an

existing requirement

Chapter 4

Requirement Examples

Examples of requirements clarify the

requirements and can be used as tests

Requirement Examples -

Example

Requirement Examples -

Example

Testing

Tests of business rules can be executed in at

least four ways:

1. Through the user interface of a

transaction that invokes the business rule

2. Development of a user interface that

directly invokes the business rule

3. Unit test implemented in a language’s

unit testing framework

4. Automated test that communicates with

the business rule module

Example project –

Chapter 5

Charter & Features

Charter objectives - project acceptance tests

Features (high level requirements) - feature acceptance criteria

ATDD Process Chapter 5

CharterVision - Describes the end-state of the result of the project, why

project should be done.

Mission - Path that will be taken to get to the vision.

Scope – boundaries of the project and key issues that the

project addresses (optional)

Deliverable – tangible outputs (optional)

Objectives - Measurable criteria for the success of the project

Principles - Statement of values that a team uses to inform

decision making.

CharterVision - Describes the end-state of the result of the project, why project should be done.

Mission - Path that will be taken to get to the vision.

Scope – boundaries of the project and key issues that the project addresses (optional)

Deliverable – tangible outputs (optional)

Objectives - Measurable criteria for the success of the project -project acceptance tests

Principles - Statement of values that a team uses to inform decision making.

Objectives – Acceptance

Tests

Objectives must be visible by external observation

SMART :

Specific

Measureable

Achievable

Relevant

Time boxed

Video

store

rental

example

Releases

Possibly create mini-charters for each project release

ATDD Process

High Level Requirements

High level requirements or features

Create a list of possible features.

Select those that accomplish the objectives.

Estimate if the selected features can be implemented

to achieve the objectives.

If not, reevaluate the objectives.

Video Rentals Brain Storm

Features to Accomplish

Objectives

Feature Acceptance Criteria

Sequence & Schedule Features

User Stories – Chapter 6

User stories - Story acceptance criteria

ATDD Process

User Stories Chapter 6

Break “features” into smaller “user stories” (also

called “requirement story”).

Only breakdown enough features to fill the

team’s time until the next feature breakdown.

Constraint stories also belong here .

Example – software must be developed using

open source software.

User Story Process

While creating user stories , list one or more

acceptance criteria for each story

These acceptance criteria are expanded to

specific acceptance tests just before the story

is develop

Make stories brief. Details are gathered just

prior to, or during, the story’s implementation

Sprint PlanningEstimate the business value and effort for each

story – ratio gives a rough “Return On

Investment” (ROI) – use ROI to determine what

belongs in next sprintAs ROI flattens

may terminate

project for

other projects

with higher ROI

If iterations involve approximately

the same effort, the slope of the

business value curse is roughly the

ROI

User Story Breakdowns

Break large stories into smaller – but business

value for a story is not achieved until all

needed stories are developed

Developer stories don’t give customer direct

value, but may be needed

Developer Stories

Some customer stories require framework development, translating to many developer stories. Example -development organization develops common modules or components

Developer stories add technical value and make it easier to track units of work

Acceptance criteria for these defined by the developers, not the customers

If considerable framework development is needed, - have an embedded technical project and a “technical product owner”, who can break a customer stories into technical customer stories, each with acceptance criteria, is needed

Roles

Role – “hats” people wear. One person can perform different roles.

Role attributes: Frequency of use

Domain expertiseComputer expertise

General goal

Can create “persona” for each role.

Persona – imaginary person described with lots of detail.

User Stories

As a <role>, I want to <do something> so

that <reason>.

Want stories to satisfy INVEST

I – independent

N – negotiable

V – valuable

E – estimatable

S – small

T – testable

User Stories

Story

Accept

ance

Criteria

Story Estimates

Estimates needed:

Business value – relative worth of a story

to the business

Effort - include all work required to deliver

the store including implementing the

code, testing it, and any other work

involved.

Business Value

Business value:

Increased revenue (sales, royalties, fees)

Decreased expenses

Using fewer resources

More efficient use of resources

Customer satisfaction

Product promoters/satisfiers/detractors

Staying in business

Avoiding risk

Estimating Business Value

Order the stories and assign values :

Fibonacci series (1, 2, 3, 5, 8, 13, …)

power of 2 (1, 2, 4, 8, …)

Collaborating on

Scenarios – Chapter 7

Use cases (scenarios) - acceptance tests

ATDD Process

Use Case

Use case – describes a sequence of actions

and reactions between the user and the

software

Use cases can be highly details, fairly

lightweight, or anywhere in between

Don’t create use cases for stories until they are

needed, helping to avoid “analysis paralysis”

Sample Use Case

As the clerk, I want to check out a CD for a customer

so that I can keep track of who has rented it.

Use Case Template

Name – brief name given to user story

Description – can be the user story

Actor – almost always the role

Sample Use Case

As the clerk, I

want to check

out a CD for a

customer so

that I can keep

track of who

has rented it.

Sample Use Case - Exceptions

As the clerk, I want to check out a CD for a customer

so that I can keep track of who has rented it.

Sample Use Case - Alternatives

As the clerk, I want to check out a CD for a customer

so that I can keep track of who has rented it.

Sample Use Case - Tests

As the clerk, I want to check out a CD for a customer

so that I can keep track of who has rented it.

Outline acceptance tests – at least one for the main

course, each exception, and each alternative

Alternative Ways to Generate

Scenarios

Scenarios can be generated :

• Story map

• Conceptual flow

Story

Map

Conceptual Flow

Test Anatomy – Chapter 8

Finally the tests

ATDD Process

Developing Tests

Customer, developer and tester develop tests together

Tester may envision extra tests

New tests may be discovered during implementation, but if this occurs frequently, the team can investigate the root cause of missing tests

Test Structure

Six Tests Similar Tests

Test Flow – Check Out CD

Sample

Test

…

Test CasesTest case is a test scenario with actual data.

Data can be placed in tables

Tables can be useful for expressing tests:. Three common types of tables:

Calculation

Data

Action

Calculation Table

Format:

? Indicates output

Examples

Data Table

Data tables can be used for input values or

expected values

Examples

Can include “data” in the name. If data

meets certain criteria, specify it after the

name

Action Table

First column: enter, press and check.

Examples

Use Tables or Not

Tests Needed

At least one test is needed for:

1. Every scenario (happy path)

2. Every alternate path

3. Every exception

more later….

Scenario Tests– Chapter 9

Cross-Story/Scenario Tests

Cross-Story – stories that have issues with other stories

Example:

Scenario for monitor that expects response from each disk each second or reports a failure

Scenario that turns a disk off if it isn’t accessed after 60 seconds.

Tests for each scenario may work, but need one or more tests for the interactions (integration testing)

Example – Late Rental & CD

Limit Reached

Multi-Level Tests

Run tests at multiple levels

Example – scenario that involves business

rules. Business rule tests are usually simpler

than UI tests (business rules don’t require

much setup). Test separately. Test business

rules, and later test UI.

UI Tests

Create a working system early and test

against objectives. (Could create a

prototype.)

Alternate Interfaces

Tests Needed

At least one test is needed for:

1. Every scenario (happy path)

2. Every alternate path

3. Every exception

4. Cross story tests

5. Business rules (multi-level)

6. UI tests

User Story Breakup –

Chapter 10

Boundary Conditions

Consider boundary conditions and if more

tests are needed.

System Boundary –

Chapter 11

External Interfaces

Credit Card Processor

Bank Transfer

Component Tests

Component Tests - continued

Component Tests - continued

Explore ways the credit card could be

denied and ask the customer what to do in

each case. Tests can be built for each.

Development Review –

Chapter 12

Workflow Tests

Need to test the entire workflow to make

sure there are no issues between related

stories and that the entire flow is usable.

Tests NeededAt least one test is needed for:

1. Every scenario (happy path)

2. Every alternate path

3. Every exception

4. Cross story tests

5. Business rules (multi-level)

6. UI tests

7. External interfaces

8. Component

9. Usability testing

10. Quality attribute testing

Revisit Chapter 1

Test Conditions (Chapter 1)

Tests should:

1. Fail for a well defined reason (the reason

the test was created)

2. No other test should fail for the same

reason, (otherwise you have a

redundant test)

3. A test should not fail for any other reason

Test Doesn’t Fail for Other

Reasons

A test should not fail for any other reason –

hard to achieve.

Tests have setup, action/trigger and

expected result. A test may fail because the

setup did not work or the action/trigger did

not function properly.