Acceptance Test Driven Development at StarWest 2014

ATDDAcceptanceTest Driven Development

J a r e d R i c h a r d s o n A g i l e A r t i s a n s

Who are you?

Jared Richardson

Author Ship It! and Career 2.0

Screencast editor PragProg.com

2nd public signatory of the Agile Manifesto

Started AgileRTP in 2007!

Agile coach

AgileArtisans.com

Who are you?

What would you say you do here?

Roles?

Responsibilities?

Why are you here?

What are your tests for this session?

Objectives

Understand ATDD in context of SDLC

Learn how to turn requirements into tests

What is a good acceptance test?

ATDD as a communication vehicle

Outline

Terms and definitions

Requirements and tests

Software development

ATDD examples

Anatomy of a test

Tables

System boundary

Breaks

Halfway (90 minutes)

One Rule

There are exceptions to every statement

Except this one

What is an Acceptance test?

External view of the system

Examines visible effects

Inputs

Outputs

State changes

External interfaces

Definitions

Acceptance criteria -> General idea

Acceptance test -> specific. Pass/fail.

Implementation independent

Triad -> Customer, developer, tester

Fast Car

Who wants one?

Criteria

Closed course. Measure acceleration.

Test

Accelerate 0 to 60 in X seconds

Fast Car

0 to 60 in 30 seconds

0 to 60 in 5 seconds. Top speed is 61.

0 to 60 in 5. Top speed is 120. 60 to 120 in 5 min

0 to 60 in 5. 60 to 120 in 20 sec. Range of 1/4 mile.

Range of 200 miles. Cost of 7 billion.

Why?Rework down from 60% to 20%

Workflows work 1st time

Little room miscommunication

Saves time

Getting business rules RIGHT

Game changing

Tighter cross functional team integration

Crisp visible story completion criteria

Automation yields reduced testing time

Requirements and Tests

Requirement

Needed by a user to solve a problem

Needed by a system to satisfy contract, standard, spec, regulation...

Requirements Include

Functions (what software does)

Constraints (limits)

Requirements Should not

Include

Implementation details

Specific manifestation

A Brownie Spec

Ingredient        Parts  by  weight

Sugar                                                              23.0  Flour                                                              21.0  Shortening                                        16.8    Nuts                                                                16.0  whole  eggs                                        13.0  Cocoa                                                            5.5  Dextrose,  anhydrous      4.4  Salt      

Brownie Spec

a.  Whip  eggs  in  large  bowl  on  high  speed  un3l  light  and  fluffy.  b.  Combine  sugars,  cocoa,  salt,  and  leavening;  add  to  beaten  eggs,  and  whip  on  high  speed  un3l  thick.  c.  Add  shortening  slowly  while  mixing  on  low  speed.  d.  Scrape  bowl  and  whip  on  high  speed  un3l  thick.  e.  Mix  flour,  nuts,  and  flavors  together  and  fold  into  baDer;  mix  un3l  uniform.  f.  Pour  baDer  into  pan  at  a  rate  that  will  yield  uncoated  brownies  which,  when  cut  such  as  to  meet  the  dimension  requirements  specified  in  3.4f,  will  weigh  approximately  35  grams  each.    g.  Bake  at  3500F  un3l  done  (30  to  45  minutes).  

Requirements and tests

Inter-related

Can't have one w/o the other

Failing test...

... is a requirement

Passing test...

... valid system spec

ATDD as a Prism

Requirements

Tests

ATDD

Software Development

Exercise What Issues?

Current software dev process

Write down your own

Groups of 3 to 8

Pass to front

Developers Testers

Customers

The Team

Customer Unit

Product owner

Business analysts

Users

QA

Developer/Tester Unit

Programmers

Testers

QA

Developers Testers

Customers

The Triad

Traditional Value Stream

Elicit&Requirements&

Analyze&Requirements& Design&&

Code& Test&

Why go back?

Deploy&

Mistakes?

Misunderstandings

Missed requirements

Mis other

Mistakes

Feedback corrects misunderstandings

Quick feedback better than slow feedback

Desired Actual

Slow Feedback

Actual

Fast Feedback

Agile Value Stream

Elicit&Requirements&

Analyze&Requirements&With&Tests&

Design&&

Code&With&Tests& Deploy&

Move Testing Forward

Two Types of Testing

To find defects

To prevent defects

Finding Defects

Waste

Identifying mistakes

flickr.com/photos/mag3737/444557606

preventing Defects

Saves development time

Saves QA time

Saves money

Steps

Author tests (write)

customer, tester, developer together

Connect to the system (automate)

developer

Run the tests (execute)

developer, testers, customers, CI system

Test% write% bind% run%

Two Ways

Code%write% run%bind%

Code%write%

run%

bind%

Test First

Write the test first

Write the code after

One molds the other

First in wins

Testing Strategies

Meszaros, XUNIT TEST PATTERNS: REFACTORING TEST CODE, Fig 6.1 "Purpose of Tests" p. 51, © 2007 Pearson Education, Inc

Testing Focus

Verification vs

ValidationVerification Are the requirements right Focus of acceptance tests

Validation Have we got the right requirements? Customer really do it this way? How do you test this?

Acceptance Testing

Applica'on*/**Middle*'er*

Core*Business*Logic**

UI*

Persistence*

Acceptance Tests

Tests all the way down

Requirement* Requirement**

Test* Test*

Requirement*

Test*

Test*

Service*Implementa4on*

Test*

Customer Acceptance Tests

Developer Acceptance Tests

Service*Implementa4on*

Requires services to implement

Acceptance Test Examples

Temperature

Input in Celsius

Output in Fahrenheit

What tests?

Temperature

Test Binning

Celsius' Fahrenheit' Notes'0" 32"100" 212" Needed?"

Celsius' Fahrenheit' Notes'*273.15" *459.67" Precision"

Formula Tests

Precision Tests

Celsius' Fahrenheit' Notes'*273.15" *459.67" 0"Kelvin""*273.151" Error" Below"0"Kelvin""500" 932" Maximum"–"Needed?"

Limit Tests

Is This A Good Spec?

Business Rules

If a Customer Rating is Good and the Order Total is less than or equal $10.00, Then give no discount Otherwise give a 1% discount

If Customer Rating is Excellent, Then give a discount of 1% for any order

If the Order Total is greater than $50.00, Then give a discount of 5%

Rule Table => Test

Another Business Rule

Email coupon codes

If a customer enters the code, provide a 20% discount

Discount an item or the order?

Implementation

Testing script

Program interface

Xunit framework

ATDD framework

Testing Script

Tools for testers

(Usually) GUI based

Testing Script

Log in as a Customer who is rated GoodStart an OrderAdd items totally $10.01Complete order Verify $0.10 discount

Repeat for 10 other similar cases

Program Interface

Code something (CLI or GUI)

Screen

Discount Method

Discount Percentage Screen Customer Type: Good Order Total: 10.01 Percentage: 1 %

C:>DiscountPercentage Good 10.01 Percentage: 1%

Xunit Test

class TestCase { testDiscountPercentageForCustomer() { SomeClass o = new SomeClass()

assertEquals(0, o.computeDiscount(10.0, Good)); assertEquals(1, o.computeDiscount(10.01, Good)); assertEquals(1, o.computeDiscount(50.01, Good)); assertEquals(1, o.computeDiscount(.01, Excellent)); assertEquals(1, o.computeDiscount(50.0, Excellent));

assertEquals(5, o.computeDiscount(50.01, Excellent));

} }

Fit Test

Tables as Requirements and Tests

Discount Rule Customer Rating Order Total Discount Percentage

Good <= $10.00 0% Otherwise 1%

Excellent Any 1% > $50.00 5%

Discount))Test)Order)total) Customer)ra3ng) Discount)percentage?)$10.00% Good% 0%%$10.01% Good% 1%%$50.01% Good% 1%%$.01% Excellent% 1%%$50.00% Excellent% 1%%$50.01% Excellent% 5%%

Requirement

Test

Context

Context Diagram

Process'

User

Reports

Commands Views

Another Context Diagram

Process'

User

Another System Shared Database

Commands

Views

Requests

Responses

Stores

Retrievals

Engine Controller

Engine&Controller&

Engine Rotation Speed

Gasoline Flow Control

Spark Timing

Acceleration / De-acceleration

Air Flow Control

Embedded Contexts

Sub$ System Two

Input Output

System One

Internal I/O

Car Context

Drive&sha*&&

Acceleration De-acceleration Movement

Trans-mission&

Engine&& Wheels&

&

Software Context

Core or API

Developer Acceptance

Test

Customer Acceptance

Test

Middle-tier

GUI

Use Cases and Tests

Use Case Template

Name—Iden3fier  to  easily  reference  it  by    DescripGon—Brief  note    Actor—Who  ini3ates  the  use  case  Pre-­‐condiGons—What  must  be  true  before  the  use  case  is  ini3ated    Post-­‐condiGons—What  is  true  if  the  use  case  successfully  executes  Main  course—Steps  that  show  the  sequence  of  interac3ons

Example: Part One

Name—Check  Out    CD.  DescripGon—Check  out  a  CD  for  a  customer.  Actor—Clerk.    Pre-­‐condiGons—The  customer  has  an  iden3fica3on.  The  CD  has  an  iden3ty.    

Post-­‐condiGons—The  CD  is  recorded  as  rented.  The  rental  contract  is  printed.  

Main  Course:    1.  The  clerk  enters  the  customer  iden3fica3on  and  CD  iden3fier  into  the  system.  

2.  The  system  records  the  informa3on.  3.    The  system  prints  a  contract  that  the  customer  signs.

Example: Part Two

Alterna3ves:    flow  that  allows  the  use  case  to  be  successful  even  if  some  condi3on  occurs  

Excep3ons:  condi3ons  can  occur  that  may  not  allow  it  to  reach  its  post-­‐condi3ons.  

Business  Rules:  rules  that  must  be  followed  regardless  of  technology    

Note:  Alterna3ves  and  Excep3ons  could  become  separate  story(ies)  if  requires  significant  effort  

Excep3ons  and  Alterna3ves  numbered  with  reference  to  Main  Course  steps

Example: Part ThreeExcepGons:  1a.   Customer  iden3fica3on  is  not  recognized.  Clerk  repeats  step  1.    

1b.  The  customer  violates  the  CD  Rental  Limit  business  rule.     The  clerk  no3fies  the  customer  of  the  viola3on.     The  use  case  is  abandoned.    

Business  Rule:  CD  Rental  Limit       A  customer  can  rent  only  three  CDs  at  any  one  3me.    

AlternaGves       3a.  The  printer  jams.      The  clerk  fills  out  the  contract  by  hand.        The  use  case  exits.

Acceptance Testing

The Triad create acceptance tests for stories

Tests from general (expected path) ...

... to specific (exceptions)

Examples

Rent a CD (expected path)

Bad Customer ID (Enter it wrong)

CD Rental Limit (already has 3. Try 4.)

Print jam (simulate by removing paper)

Test Anatomy

Test Flow

Setup

Use

Verify

Test Flow

Ini$al'System'State'

Test'

Setup'(Given))'

Trigger'(When)'

Assert'(Then)'

Final'System'State'and'Output'

Action or Event

ExpectedSystem'State'and'Output'

Given/When/Then

Given  =  Use  case  pre-­‐condi3ons    When  =  Main  course  (or  excep3onal  course)  Then  =  Use  case  post-­‐condi3ons  

Given/when/Then

Given  (Setup)     Customer  has  ID  (ini3al  system  state)     CD  has  ID  (ini3al  system  state)     CD  is  not  currently  rented  (ini3al  system  state)  When  (Trigger)     Clerk  checks  out  CD  (ac3on)    Then    (Verify)     CD  recorded  as  rented  (final  system  state)     Rental  contract  printed  (output)  

Context Diagram

Internal or External Persistence

Output is response to change address

Change address Send mail

External repository for address

Input for send mail

Internal or External Persistence

How  to  test  internal  persistence?    Through  another  ac3on    Through  a  “see-­‐through”  method      

Change address

Internal persistence of address (state)

Send mail

Addressed mail

ExerciseWrite  ques3ons  (tests)  for:     “copying  a  file  from  one  directory  to  another”  Describe  in  terms  of:  Given:  some  state  When:  copy  (file,  source_directory,  des3na3on_directory)  

Then:  expected  state  

Source'Directory'Contents'

Des/na/on'Directory'Contents'

Copy'File''

Des/na/on'Directory'Contents?'

Anything'else?''

X' X' X'

Given When Then (Expected)

Scenario

B

A X

Copy%

B X

A X

Example

Source'Directory'Contents'

Des/na/on'Directory'Contents'

Copy'File''

Des/na/on'Directory''Contents?'

Source'Directory'Contents?'

Content'Comparison?'

Notes'

X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#

Standard#copy##

X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#

X#overwri?en#

X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Des-na-on.X)##

X#not#overwri?en#

What about modification date and other attributes? Make more columns

Given When Then (Expected)

Tables

Tables

Table-­‐driven  requirements  can  reduce  ambiguity  over  free  text  

Tables  can  be  the  tests  Discovery    Filling  out  tables  may  bring  to  light  suppressed  premises    Unstated  requirements  or  assump7ons  

Table Types

Calculation

Action

Data

Calculation Table

Calc Table Example

Discount Calculation

Item Total Customer Rating Discount Percentage?

$10.00 Good 0%

Given

When

Then

Data

!  Exists&(or&should&exist)&–&for&Given&and&Then&

Title Data

Value Name 1 Value Name 2

Value for 1 Value for 2

Another value for 1 Another value for 2

Data Table Example

Customer Data

Name ID

James 007

Maxwell 86

Customer Data Name Begins with="’J"

Name ID

James 007

Action Table

Enter  enters  data  into  an  entry  field  Press  ini3ates  a  process,  such  as  a  Submit  buDon  Check  sees  if  a  result  is  equal  to  an  expected  value

Action Table Example

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Press Rent

Check Rented True

Can be just “When” or a stand-alone Given-When-Then

Given

When Then

Copy File ExampleCopy%a%file%(happy%path)%%!  Given&a&source&&!  and&target&directory&&&&&&&

File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F

Copy File

Enter Source Directory A

Enter Target Directory B

Enter File name X

Press Copy

!  When&copy&request&is&made&

&&&&&&

!  Then&file&is&copied&&&&&&&

File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F,X

The Action

!  Can$drive$a$GUI$

!  Or$a$method$$CopyFile(Directory source, Directory target, Filename file)

Copy File ExampleNon$existent*file***Given&a&source&and&target&directory&

File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F

Copy File

Enter Source Directory A

Enter Target Directory B

Enter File name Q

Press Copy Error_file_does_not_exist

!  When&copy&request&is&made&for&a&non;existent&file&,&an&error&is&generated&&&&&

!  Anything&else?&&&&&

Given/When/Then revsited

Given  (Setup)     Customer  has  ID  (ini3al  system  state)     CD  has  ID  (ini3al  system  state)     CD  is  not  currently  rented  (ini3al  system  state)  When  (Trigger)     Clerk  checks  out  CD  (ac3on)    Then    (Verify)     CD  recorded  as  rented  (final  system  state)     Rental  contract  printed  (output)  

Check Out A CD!  Given&Customer&has&ID&&&&&&&and&CD&has&ID&&&&&&&and&CD&is&not&currently&rented&&

Customer Data

Name ID

James 007

CD Data

ID Title Rented

CD2 Beatles Greatest Hits No

CHECKING OUT CONTINUED

! When%a%clerk%checks%out%a%CD:%

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Press Rent

CONTINUED FURTHER...

!  Then%the%CD%is%recorded%as%rented%and%a%rental%contract%is%printed:%%

CD Data

ID Title Rented Customer ID

CD2 Beatles Greatest Hits Yes 007

Rental Contract

Customer ID Customer Name CD ID CD Title

007 James CD2 Beatles Greatest Hits

THE ACTION

!  Can$drive$a$GUI$

!  Or$a$method$$Rent(CustomerID aCustomer, CDID aCD);

ASSUMPTIONS

Assume...- No other data changes- Can select customers (or CDs) from a list

Could have tests for these assumptions...

Test ExampleCheck&that&no&other&elements&changed&on&check1out&

!  Given&Customers&and&CDs&&&&&&&& Customer Data

Name ID James 007 Maxwell 86

CD Data

ID Title Rented Customer ID

CD2 Beatles Greatest Hits No

CD3 Lucy&Michelle&Hits No

CD4 Janet&Jackson&Hits Yes 86

Test Example 1Check&that&no&other&elements&changed&on&check1out&

!  Given&Customers&and&CDs&&&&&&&& Customer Data

Name ID James 007 Maxwell 86

CD Data

ID Title Rented Customer ID

CD2 Beatles Greatest Hits No

CD3 Lucy&Michelle&Hits No

CD4 Janet&Jackson&Hits Yes 86

Test Example 2

! When%a%clerk%checks%out%a%CD:%

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Press Rent

Test Example 3

!  Then%only%the%rented%CD%is%affected%%Customer Data Name ID James 007 Maxwell 86

CD Data

ID Title Rented Customer ID

CD2 Beatles Greatest Hits Yes 007

CD3 Lucy%Michelle%Hits No

CD4 Janet%Jackson%Hits Yes 86

Selection ExampleCheck&that&a&customer&can&be&selected&&

!  Given&Customers&&&&&&&

Customer Data Name ID James 007 Maxwell 86

Check Out CD

Enter Customer ID 86

Press Select

!  When&a&Customer&is&selected&&

&&&&&&

!  Then&the&correct&customer&is&selected&&&&&&&

Selected Customer Name ID Maxwell 86

Business Rules Test

Usually many

One test per rule

Example

Rates&

Type& Days& Cost?&Regular( 2( $2(Golden(Oldie( 3( $1.50(Hot(Stuff( 6( $12(Hot(Stuff( 50(( IGBTYOT((

CD(Rental(Rates(Regular(($2(/(2(days(plus($1(/(day((Golden(Oldie($1(/(2(days(plus($(.50(/(day(Hot(Stuff(($4(/(2(days(plus($2(/(day(

UI & Business Rules

Email&Valida)on&(Middle/Tier)&

Value&& Valid?&&manager@sam.com, yes,someone@@somewhere.com, no,

!#$%&*+7/=?^_,{|}~@sam.com, yes,

Email

Ajax call to middle-tier

Submit,

Middle-tier test

Alternatives

Can use "script" in tables or separately

Copy A File (Happy Path)

Given directory A with files X, Y, & Z

And directory B with files D, E, F

When file X is copied from A to B

Then directory A has files X, Y, & Z

and directory B has files D, E, F, & X

CD Rental (happy Path)

Given customer James with ID 007

And CD ID CD2 w/title "Beatle's Greatest Hits" is not rented

When customer 007 rents CD2

And a rental contract is printed with: name James, customer ID 007, CD ID CD2, and CD title "Beatle's Greatest Hits"

Business Rule Script Test

Rates&

Type& Days& Cost?&Regular( 2( $2(Golden(Oldie( 3( $1.50(Hot(Stuff( 6( $12(Hot(Stuff( 50(( IGBTYOT((

When(a(CD(of(type(Regular(is(rented(for(2(days(Then(the(cost(is($2(When(a(CD(of(type(Golden(Oldie(is(rented(for(3(days((Then(the(cost(is($1.50((When(a(CD(of(type(Hot(Stuff(is(rented(for(6(days(Then(the(cost(is($12((

System Boundaries

External Interfaces

!  Connec&ons(to(external(systems(need(to(have(test(doubles((mocks)(

!  Random(events(may(need(to(be(simulated((!  Test(doubles(give(repeatability(and(speed((

External system, device, service

Random events

Time

External Interface Example

CD Rental System

Credit Card Processor

Sam’s Bank

Charge Transfer To

Confirmation or Denial Bank

Statement

Example of a Credit Card Test Double

CD Rental System

Credit Card Processor Test Double

Charge

Confirmation or Denial

Example of a Email Test Double

CD Rental System

Email Server Test Double

Email

Confirmation or Denial

Test verifies that email sent to proper address with proper contents

Complex Business Rule and

Separation

Complex Business Rule

Field&One& Field&Two& Field&Three& Field&Four& Result&?&>&20& <&50&or&blank& >=100& Y& Allow&Otherwise&or&blank&

DNC& DNC& DNC& Disallow&

DNC& Otherwise& DNC& DNC& Disallow&DNC& DNC& Otherwise&or&

blank&DNC& Disallow&

DNC& DNC& DNC& N&or&blank& Disallow&

A business rule determines whether a user is allowed to perform a certain operation Fields may contain values or be blank

Complex Business Rule

Field&One& Result?&>&20& Allow&Blank& Disallow&Otherwise& Disallow&

Field&Two& Result?&<&50& Allow&Blank& Allow&Otherwise& Disallow&

Field&Three& Result&?&>=100& Allow&Blank& Disallow&Otherwise& Disallow&

Field&Four& Result?&&Y& Allow&N& Disallow&Blank& Disallow&Otherwise& ??&

Complex Business Rule

Field&One& Field&Two& Field&Three& Field&Four& Result&?&Allow& Allow& Allow& Allow& Allow&Disallow& DNC& DNC& DNC& Disallow&DNC& Disallow& DNC& DNC& Disallow&DNC& DNC& Disallow& DNC& Disallow&DNC& DNC& DNC& Disallow& Disallow&

Business Rule & UI

Two  business  rule  results  Allow  Disallow  

Alternate  UI  manifesta3ons  of  the  business  rule  Disable  the  buDon  if  disallowed  Hide  the  buDon  if  disallowed.  Display  dialog  box  if  disallowed  and  they  push  the  buDon    

To  cut  down  tes3ng,  expose  business  rule  test    5  +  2  tests,  rather  than  5  *  2  tests  

Separate UI from Business Rule

Business'Rule'Display'Requirement''

Business'Rule'Result' UI'Display?' Notes''

Disallow' Disable'Bu9on'

Allow' Enable'Bu9on'' Take'to'some'entry'screen'

End@to@end'Business'Rule'Test'

Sample'user'ID' Business'Rule'Result' UI'Display?'

10345' Allow' Enable'Bu9on'

5555' Disallow'' Disable'Bu9on'

Third'column'will'change'if'Business'Rule'Display'Requirement'changes''

Separate the Model from the View

Rental Contract

Customer ID Customer Name CD ID CD Title Rental Due

007 James CD2 Beatles Greatest Hits 1/23/2011

Rental Contract Template The customer named <Customer Name> with the ID <Customer ID>, hereafter referred to as the Renter, has rented the CD identified by <CD ID> with the title "<CD Title>," hereafter referred to as the Rented CD, from Sam's Lawn Mower Repair and CD Rental Store, hereafter referred to as the Rentee. The Renter promises to return the Rented CD to the Rentee by <Rental Due>. If said Renter exceeds .. blah…blah…blah

Rental Contract Printout The customer named James with the ID 007, hereafter referred to as the Renter, has rented the CD identified by CD2 with the title "Beatle's Greatest Hits," hereafter referred to as the Rented CD, from Sam's Lawn Mower Repair and CD Rental Store, hereafter referred to as the Rentee. The Renter promises to return the Rented CD to the Rentee by 1/23/2011. If said Renter exceeds .. blah…blah…blah

Separate the Event from the Response

Logon%Sequence%%

Username% Password% Result?% Notes%

Sam$ 123$ Bad_password$ 1st$bad$

George$ 1123$ No_user$ 2nd$bad$

Sam$ 1234$ Bad_password$ 3rd$bad$

Same$$ 12345$ Security_viola?on$ 4th$bad$

Logon%Ac9ons%%

Result% Ac9on%

Bad_password$ Display$“password$bad”$

No_user$$ Display$“no$user”$

Security_viola?on$ Send$enforcement$team$

Google Maps

What can you test w/o a UI?

Google Maps

No UI?

Search phrase => List of places

Place => Position

Zoom Level => List of maps

Position => List what's there

Test Evaluation

WARNING!!

Acceptances tests do NOT replace interactive communication!

They provide focus for the communication

Test Evaluation

Remove  redundant  redundancy  Business  domain  terms    Shared  between  customer  unit  and  developer/tester  units  

Avoid  lots  of  input  and  output  columns  Break  into  smaller  tables  (example  follows)

Big Table to Smaller OneBig$Data$Test$

F1$ F2$ F3$ F4$ …$ Valid?$$ Reason$

1$ 2$ 3$ 6$ Y$

1$ 3$ 3$ 6$ N$ F2$+$F3$>$5$

1$ 3$ 1$ 6$ N$ F2$+$F3$<$5$$

…$Lots$more$$for$other$rules$

Big$Data$Test$for$F2EF3$$

F2$ F3$ Valid?$$ Reason$

2$ 3$ Y$

3$ 3$ N$ F2$+$F3$>$5$

3$ 1$ N$ F2$+$F3$<$5$$

If can apply default values to F1, F4 etc., then could be:

Guidelines

Develop tests & automation separately First, understand Second, automate (if appropriate)

Automate tests for regression Run in continuous integration

Cover 100% of function (if practical)

Separation of Concerns

Separate  business  rules  from  how  results  of  business  rules  are  displayed      

Separate  calcula3on  of  a  business  rule  (such  as  a  customer  ra3ng)  from  the  use  of  that  business  rule  (such  as  to  give  a  discount)        

Separate  each  use  case  or  step  in  a  workflow    Separate  out  valida3on  of  an  en3ty  from  use  of  en3ty  

Separation and Abstraction

One  story  together:  When  the  build  fails,  send  an  email  to  the  administrator    

Separate  and  abstracted:    When  the  build  fails,  generate  an  alert    When  an  alert  is  generated,  no3fy  the  responsible  individual    

When  the  responsible  individual  is  no3fied,  use  that  individual’s  no3fica3on  preference  (email)    

Test  the  no3fica3on  itself    works  (e.g.  email  works)

What is a Good Test?

Fails reliably (for a known reason)

No false positives

No other test covers the same area

NOTE: Tests at the "top" often forced to assume business rules... aim for 1 failure at each level

Use Alternate Tables

Sudoku Puzzle 1 4 7

2 5 8 3 6 9

4 7 1 5 8 2

6 9 3 7 1 4

8 2 5 9 3 6

Sudoku Solution 1 6 5 4 9 8 7 3 2 9 2 4 3 5 7 6 8 1 8 7 3 2 1 6 5 4 9 4 9 8 7 3 2 1 6 5 3 5 7 6 8 1 9 2 4 2 1 6 5 4 9 8 7 3 7 3 2 1 6 5 4 9 8 6 8 1 9 2 4 3 5 7 5 4 9 8 7 3 2 1 6

Create tables in alternative forms, if standard not appropriate

Required(Performance((

Puzzle(Type( Time(to(Solve?( Notes(

Easy% .1%seconds%% See%easy%determina2on%

Impossible% 1%second% No%solu2on%%

Story Tests Not Enough

Disk%Monitor%

Report'Error'If'Disk'Does'Not'Respond'Within'1'Second

Power%Saver%%

Spin'Down'Disk'If'Not'Accessed'In'Past'Minute

Other uses for Tests

Estimation Aids

Degree of Doneness

Process note

Requirement Testable Requirement

Developable Requirement

At least one example for each test

Not schedulable Preferably all examples for all tests

process Note

Code%write%

run%

bind%

Code%write%

run%

bind%

Write previous iteration

Write within iteration

Not the End.... The Beginning!

Objective Review

Understand ATDD in context of SDLC

Learn how to turn requirements into tests

What is a good acceptance test?

ATDD as a communication vehicle

Our OutlineTerms and definitions

Requirements and tests

Software development

ATDD examples

Anatomy of a test

Tables

System boundary

Recap

Primary  goals  Discover  ambiguous  requirements  and  gaps  in  requirements  early  on.  

Create  a  record  of  business/development  understanding.  Give  feedback  on  quality  .  

Secondary  goals  Use  acceptance  tests  as  an  executable  regression  test.  Measure    your  progress  towards  "done”  completeness.  Measure  the  complexity  of  requirements.  Use  the  tests  as  a  basis  for  user  documenta3on.

Exercise

You listed your development issues...

Would ATDD help, hurt, or be neutral

Why?

Write down your answer and pass it up

Agility Responding to Change

You're my customers

You provided your requirements

Now play the user/tester

Were your requirements met?

What else needs to be done?