Embed Size (px)
DESCRIPTION
Agile Testing Days 2012 Presentation from Mike Scott & Tom Roden
Citation preview
SQS Group Limited
Mike Scott & Tom Roden
@MikeAScott @TomMRoden
Agile Testing Days
20 November 2012
Potsdam, Germany
Building a lean, mean, BDD
automation machine
Caption Competition
2
Caption Competition
A story about building an executable documentation system
• Large hedge fund COTS implementation
• Crippling regression test burden
Born out of necessity
• 6-8 month cycle daily release
• 80 times defect reduction Results:
• and again
• Same patterns automation platform
Then did it again…
Safia– Archetypes and Templates, a voyage in automating acceptance
3
The problem quantified
Real data from the field
40
67.5
100
78
23 15
12
5 3 0
28 64
151
217
311 371
393
479
545
601
0 0
25
106
284
371 393
479
545
601
0
100
200
300
400
500
600
700
0
20
40
60
80
100
120
February April August November February April May June July August
# T
es
t S
uit
es
Ma
n D
ays
Eff
ort
Month
Time taken for Release Regression testing
Man Days Effort for Regression Test # Test Suites # Automated Suites
4
The confidence problem!
Where it came from
First Approach to Acceptance Testing:
UAT in a phase, after a period of development
5
6
What it felt like…
7
Becoming acceptance driven…
Building Integrity in
8
Develop
Ready User
Story
Customer
Verification
Specify tests
/ examples
Stakeholder(s) & BA
Testers
Developers
Backlog
Demo /
Showcase
Automate
acceptance
tests
Session
Based
Testing
The initial effect of automated acceptance testing
Real data from the field
40
67.5
100
78
23 15
12
5 3 0
28 64
151
217
311 371
393
479
545
601
0 0
25
106
284
371 393
479
545
601
0
100
200
300
400
500
600
700
0
20
40
60
80
100
120
February April August November February April May June July August
# T
es
t S
uit
es
Ma
n D
ays
Eff
ort
Month
Time taken for Release Regression testing
Man Days Effort for Regression Test # Test Suites # Automated Suites
9
Knowledge and experience to design a framework
• binding tests to investment apps
A set of design patterns
• from business specialists
Inject knowledge
• easy to customise
Vanilla acceptance tests
• tests as specifications
Living documentation
What we gained from this
10
Illogical Architecture Diagram
How Safia Works
House
Adaptor
System
Adaptor
Trading system
Maps SAFIA terminology to
system terminology
e.g. Calypso, Beauchamps,
Summit
SAFIA automates at the
server level via the API
Safia fixtures
AP
I Fitnesse fixtures
Maps investment house
terminology to SAFIA
terminology e.g.
Trade Capture = Trade Entry
11
e.g.
FX, Bond, Equities
IRS, FRA, CDS
e.g.
Run a P&L report
Update a legal entity
Amend SDI
Amend fees
Creating automated test cases using Safia
How Safia Works
Test Archetype
“Test Function”
Create a trade
Instrument
Template
IRS template
Test
Create a valid IRS trade
Apply Template &
Archetype
12
Let’s put it to the test
Demo
13
It brings benefits on different levels
Why use a framework like this
14
Benefits
Increased collaboration
Accelerating implementation
& learning
Anyone can interact with it
Teaching & coaching aid
Reduced Cost of Ownership
It was easy, right?
In retrospect…
15
The path to faster release
Real data from the field
40
67.5
100
78
23
15 12
5 3 0
28 64
151
217
311 371
393
479
545
601
0 0
25
106
284
371 393
479
545
601
0
100
200
300
400
500
600
700
0
20
40
60
80
100
120
February April August November February April May June July August
# T
es
t S
uit
es
Ma
n D
ays
Eff
ort
Month
Time taken for Release Regression testing
Man Days Effort for Regression Test # Test Suites # Automated Suites
16
Tests as specs, the principles of test design
How we wrote the tests
17
What makes a good acceptance
test?
Self-documenting
About business intentions
Concise and granular
A specification not a script
Anyone can understand it
Based on testable statements
~One third is system understanding
Reducing the Cost of Ownership
18
Points to consider when automating
19
Theory ‘aint practice
Start early & iterate frequently
Distil and simplify
Tests or Documentation?
DON’T PANIC !!!
“Brace yourself Daphne…”
20
Questions
Thank you for listening
21
Please evaluate our
presentation by using the
evaluation booklets
which you can find in
your conference bag.
Thank you!
SQS Group Limited
Mike Scott & Tom Roden
SQS Group Limited
7-11 Moorgate, London EC2R 6AF
Internet: www.sqs.com
