PS3: Acceptance Test

AN EXAMPLE

PS3: Acceptance Test

Based on Testing Extreme Programming

By Lisa Crispin, Tip House

http://www.informit.com/safari/author_bio.asp@ISBN=0321113551

http://www.informit.com/safari/author_bio.asp@ISBN=0321113551

Goal

Check the functionality of the entire system as specified by the project's customer. Acceptance tests should not include specific

knowledge about the internals of the system and are said to be "black box" tests.

These tests touch the system only at predefined APIs or GUIs.

To be effective, system-level tests often require large amounts of domain-specific data.

The customer's involvement in creating this data and specifying system behavior is critical.

2

Software Engineering Spring 2011

Example Application: Seller Reputation

3


Rate the transaction

Rate a transaction

Select a seller to rate from a list of sellers

Accurate description

Select a transaction to rate

Communication

Quick shipping

Reasonable shipping charges

4


Major phases

High level Acceptance Tests Write stories Define some acceptance tests for each story

Find Hidden Assumptions in the stories Customer’s view; User’s view; Programmer’s view Add High level Acceptance Tests based on these

assumptionsObtain Data for testing

Bad, sad, happy scenariosWrite executable tests

Write code for each acceptance test defined Bridge the code to the application

5


High Level Acceptance Test

Story I:Rate Transaction Story The user can provide a rating for a transaction he took

part in, expressing the degree to which he is satisfied with it in the following properties: Accurate description, Communication, Quick shipping, Reasonable shipping charges, and textual comment.

Define high-level acceptance tests for the Story.Expected Results

Data Action

Success: the rating for the transaction is updated

An unrated transaction that exists in the database

Rate the seller by Accurate description, Communication, Quick shipping, Reasonable shipping charges, and add a textual comment.

6


Obtaining hidden Assumptions

Story

User’s view

Programmer’s view

Customer’s view

7


Step1 – Customer view

I'm the customer. We need a way to track performance of sellers in our system in

order to compute the reputation of this seller. For this purpose we need the users that interacted with this seller

to rate the seller by 4 different aspects. We don’t want users that did not have experience with the seller to

rate it – so only user who interacted with the seller may provide ratings for him.

We don’t want the user to provide multiple good ratings or bad ratings – so we allow only one rating per transaction.

What business problem is it solving? How does it solve it? Users would like to know a seller reputation before initiating a

transaction with him. By collecting users rating we use some Trust/Reputation algorithm

to infer the seller’s reputation.

8


Step 2: User view

I'm a user, I want to provide a rating for a transaction I made What are the worst and best things that can happen? What would really irritate me?

Worst thing: I can't locate the transaction I want to rate. It's in the system, but I can't find it. Two rating properties appear to have a similar meaning —I can't tell the

difference. Best thing:

I can see all the transactions I have performed in the system sorted by date, seller rating –state (rated/unrated).

I can see the rating I gave for the rated ones . I can also select an unrated transaction form my list and rate it.

How can I screw up? How should the system respond? Can I accidentally rate the same transaction more than once? Can I rate a seller that I didn’t interact with? Can I change my opinion – change the rating I already gave a seller upon

completion of a transaction?

9


Step 3: Programmer view

I'm the programmer. What's the simplest implementation that could

possibly work? A list of all transactions each indicated as rated or not. Click on an unrated transaction to select it and to rate it. Click on a rated transaction impossible.

OR?

We need some way to identify a transaction Do we? by a number?

10


Step 4: Tester’s view

How likely is the implementation to solve the business problem? Fairly likely.( under honest and cooperating users

assumptions)How likely is the implementation to solve the

related problems? Fairly likely.

How likely is the implementation to avoid irritating the user? We need a way to indicate the user that

no more than one rating is allowed per transaction There is no option to modify a rating already provided The value they can provide is within a predefined range

11


Hidden assumptions

Identify problematic hidden assumptions in the story The system provides a list of the user’s transactions The user can view rated transactions. The user cannot edit rated transaction. All four properties are required fields. The rating values range within a known scale. Textual comments are optional ?

12


Additional tests (based on the assumptions)

What other acceptance test can be added according to the assumptions?

Expected Results Data Action

Success: the rating for the transaction is updated

An unrated transaction that exists in the database

1.1.Rate the seller by the four properties.

Failure: should not be able to enter numbrs out of range

An unrated transaction, exceed maximum values for the four properties .

1.2. Rate the seller by the four properties

…

Success: The list of transactions the user had with the seller is not empty

A user , a seller, existing transactions of the two with “closed” field =No

2.1. find the List of unrated transactions the user had with a seller.

Fail: The list of transactions the user had with the seller is empty

A user , a seller, no transactions of the two with “closed” field =No

2.2. find the List of all transactions the user had with a seller.

…

13


Writing Executable Acceptance tests

The rateTransStory. High-level acceptance tests for this story:

Rating an unrated transactions using correct values: succeeds – the transaction properties are updated, transaction is closed

Rating an unrated transactions using out-of –range values: fails– the transaction properties are not updated, transaction remains opened.

Data: Users : Bob, Clair Sellers: AutoShop, Buy4Cheap Transactions: Bob, AutoShop - unrated transaction 1223,

Clair, Buy4Cheap– rated transaction 1226 This captures the essence of the test but leaves out the details. The

details can go into an executable test, as follows: assertTrue(rateTrans(1223,1,2,3,4,”bad experience”) ) assertTrue(rateTrans(1223,1,2,3,4,””) ) assertFalse(rateTrans(1223,7,2,3,4,”Called right on time”) ) assertFalse(rateTrans(1226,3,2,3,4,”unreliable”) ) assertFalse(rateTrans (1227,1,2,3,4,”blabla”) - Can blabla cause the test failure?

14


Sampled Test Data Table 16.3

Defined with the customerHappy, sad, bad…

Exxpected Results

Comment Costs Shipping Communications

Accurate Trans ID

Opened Characteristics

Transaction successfully updated

bad experience

3 2 2 1 1223 Y Valid

Transaction successfully updated

4 5 2 1 1223 Y Valid

Rating score is out of range

THE WORST!!

1 2 1 0 1223 Y Invalid

Rating score is out of range

Nice but not proffesional

1 4 2 9 1223 Y Invalid

Transaction is already rated

Very polite 3 5 2 1 1226 N Invalid

Invalid Transaction ID

Bla bla 3 2 2 1 1227 Invalid

15


Testing Method per Action

For each action of the high level acceptance test– write a testing method:

public class rateTransStoryTest {public void testRateTrans() {// Test data and assumptions for action Rate go here}public void testListTrans() {// Test data and assumptions for action List go here}

}

16


Testing Method per Action

public class rateTransStoryTest extends SellerReputationTest {public rateTransStoryTest (String name) {super(name); } \\ A pass-

thru constructor

public void testRateTrans() {assertTrue(rateTrans(“1223”,1,2,3,4,”blabla”) ) ;assertTrue(rateTrans(“1223”,1,2,3,4,””) ) ;assertFalse(rateTrans(“1223”,7,2,3,4,”blabla”) ) ;assertFalse(rateTrans(“1226”,3,2,3,4,”blabla”) ) ;assertFalse(rateTrans (“1227”,1,2,3,4,”blabla”);

}public void testListTrans() {…}

}

rateTrans(<T_ID ,><score1>, <score2>, <score3>,

<score4><comment>) should be bridged to the real application

17


import junit.framework.*; //Define an application-test class that extends JUnit' s TestCase

import SellerReputation.*;

public class SellerReputationTest extends TestCase {// bridge classpublic SellerReputationTest(String name) {super(name); }

public void assertFalse( boolean condition) {assertTrue(!condition);

}// Make every story acceptance test a sub-class of the application test class.

Public boolean rateTrans(String tId, int att1, int att2, int att3, int att4, string comment){SellerReputationTrans trans= new SellerReputationTrans();return trans.rate(tId, att1,att2,att3,att4, comment);

}public boolean listTrans( String uId, String sId) {

…….}

}

A Bridge to the rate operation as implemented in the application

18


Bridge by inheritance


The SellerReputationTrans class in the SellerReputation system represents a transaction.

The SellerReputationTrans has a rate method that sets the transaction rating criteria: Accurate description, Communication, Quick shipping, Reasonable shipping charges and comment. If the specified values are all valid, it then attempts to update the transaction, if the

update is successful it returns true otherwise - if the update fails or if any of the specified values is invalid, it returns

false

The SellerReputationUser class in the SellerReputation system represents a user.

The SellerReputationUser has a transList method that searches for a list of transactions the user had with a specified seller If the user had interactions with the specified seller, it produces the list of

transactions, if the list is nonempty it returns true otherwise - if the list is empty or if seller id is invalid, it returns false

19



TestCase (junit )• assertTrue• assertEqual• …

SellerReputationTest• assertFalse• rateTrans • listTrans• …

rateTransStoryTest• testRateTrans• testListTrans• …

Functionality provided by testing framework, e.g. junit

Project Test Defines the bridge to

the application

Actual tests

20


Bridge by Composition


TestCase (junit )• setUp()• assertTrue• assertEqual• …

SellerReputationTest• setUp()• assertFalse• rateTrans { ..bridge..} • listTrans { ..bridge..}• …

rateTransStoryTest• testRateTrans

{..rateTrans..}• testListTrans{..listTrans

..}• …

<<Interface >>

SellerReputationBridge

• rateT()• listT()• …….

Real Bridge

• rateT()• listT()• ……..

Proxy Bridge


bridge

real

implements implements

21


How to avoid recompile the tests when replacing bridges ?• SellerReputationTest class receives the

bridge as parameter (setter and/or in constructor).

• SellerReputationTest class reads configuration file to select the right bridge (in setUp()).

• SellerReputationTest class uses external class to receive the bridge (next slide).

• Any other methods you can come up with.Software Engineering Spring 2011

22



TestCase (junit )• setUp()• assertTrue• assertEqual• …

SellerReputationTest• setUp()• assertFalse• rateTrans { ..bridge..} • listTrans { ..bridge..}• …

rateTransStoryTest• testRateTrans

{..rateTrans..}• testListTrans{..listTrans

..}• …

<<Interface >>SellerReputationBridge

• rateT()• listT()• …….

Real Bridge


Proxy Bridge


bridge

real

implements implements

23

Driver

getBridge()

<<use>><<use>>

protected void setUp() throws Exception {

super.setUp();…

this.bridge=Driver.getBridge();…

}

<<use>>

public static SellerReputationBridge getBridge() {ProxyBridge bridge=new ProxyBridge (); // add when real bridge is readybridge.real = new RealBridge (); return bridge;

}

Documents

PS3: Acceptance Test