View
5
Download
0
Category
Preview:
Citation preview
P R E S E N T A T I O N
International Conference On Software Testing Analysis & Review
October 29-November 2, 2001 San Jose, CA USA
Thursday, November 1, 2001 10:30 AM
THEORY AND PRACTICE IN TEST PROCESS IMPROVEMENT AT
BARCLAYS BANK
Kath Harrison Barclays Bank Pic
and
Martin Pol POLTEQ IT Services B.V.
T5
Presentation Bio
Theory and Practice in Test Process Improvement
Kath HarrisonMartin Pol
Objectives
To highlight practical learning points from developing
and implementing a Test Process Improvement project
in a large IT development organisation
• What will this presentation tell me ?
• Who is this presentation for ?
• What this presentation is not
Agenda
• Introduction to TPI
• Process of change
– Theory
– Barclays Experience
• Lessons learnt
TQM
SPI
TPI
Positioning and Scope of TPI
Test Process Improvement ?
Testing is uncontrollable
Testing costs too much
Testing takes too much time
Testing doesn’t help, defects left!
“Continuous improvement of the quality and the
efficiency of the testing process, related to the
output of the total software process”
Complaints
Test Process Improvement!
• Quality Timeliness CoverageInsight Control
• Costs Cheaper
• Lead Time Faster
“Continuous improvement of the quality and the
efficiency of the testing process, related to the
output of the total software process”
Improvement Life-cycle in General
• Goal and scope
• Current situation
• Desired situation
• Implementing changes
Required:
reference model
Key areas
Levels Test MaturityMatrix
Improvement suggestionsCheckpoints
The TPI® Model
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Process of Change
Change Project: Requirements
Knowledge of:
– Structured testing
– Test processes
– Organisation
– Management of change
– The TPI model
A Team!
Time
Sponsors
Social skills
Agenda
• Introduction to TPI
• Process of change
– Theory
– Barclays Experience
• Lessons learnt
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Process of Change
Awareness
• Necessity of change
• Commitment on
– Problem
– Approach
– Goals
Continuous activity !
Target, Scope, Approach
Target
– Time
– Money
– Quality
Scope
– One or more test levels
– Project / organisation
– Key areas / Levels
Approach
– Change process
– Organisation
– Costs of change
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Barclays Process of Change
Barclays
• UK based financial services organisation with operations in 60 countries
• Incorporates Barclaycard - largest credit card group in Europe
• Profits for year to Dec 2000 of £3,456 million
• Facts
– UK Branches - 2129
– Overseas Branches - 624
– ATM’s - 3,800
– Current Accounts - 8.3 million
– Telephone Banking customers 1.2 million
– Internet Banking customers - 3 million
Barclays IT Development Organisation
General
• Development staff of approximately 1,800
• About 200 of these work in testing
• Split across 2 major sites and some minor ones
• Delivers around 500 projects each year
• IT Development budget of £100 million
Testing
• Centralised testing capability
• Testing Services currently delivers 50% of all testing (Approx. 130 staff)
Background Testing
• A familiar story - improve speed to market, cost, quality of products
• Some high profile issues that could be exploited
• Created a vision of where we wanted to go - the horizon
TPI Approach
• TPI is first and foremost a Business Change project (Transition project)
• Scope was carefully managed
• Management and stakeholder buy-in secured
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Process of Change
Assessment
“If you don't know where you are,
a map won't help” (Watts Humphrey)
• Preparation
• Collecting information
• Analysis (using the TPI model)
• Reporting
Improvement actions
• In synchronisation with targets!
• Levels of TPI model
• Other criteria:
– Fast, visible results
– Low costs
– Easiest actions first
– Acceptance level
– Best cost / profit ratio
– Decrease highest risk
– …..
Plan
How to steer the change process into the right
direction?
What ?
Who ?
When ?
How ?
How much ?
.. … ? Test specific activities and
Change specific activities
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Barclays Process of Change
TPI Approach
• Approach and planning thorough and linked to vision
• Used TPI model as a vehicle to take us where we were going
• Transition project recognised most of the things that would impact the project
Areas of focus
• Lifecycle and techniques
– Lifecycle model
– Strategy technique
– Specification techniques
– Defect management
– Reporting
• Tools
• Environments
• Management of change
Management of change
• Project management
• Communication
• Metrics
• Training
• Pilots
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Process of Change
Implementation
• Execute the plan
• Measure (self-assessments)
• Consolidate
Communicate !
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Barclays Process of Change
Areas for Focus
Managementof Change Lifecycle
& Techniques
Tools
Environments
More Managementof Change More Lifecycle
& Techniques
More Tools
More Environments
• Detail takes over
• Vision is not static
Below the Water - Life Cycle & Techniques
• Fit with development lifecycle tool set
• Legacy test products
Below the Water - Tools
• Historically problematic
• High negative impact on testing
Below the Water - Environments
• Communication
• Metrics
• Support
• Project demands
• Culture
Below the Water - Management of Change
• Process driven
• Date driven
• Project focus
• Resistance
Below the Water - Culture in Barclays
• SPI
• Process V tools
• Integration V complexity
• Resistance
Below the Water - Culture in Development
• BAU V transition
• 3rd Party suppliers
• Programme
• Resistance
Below the Water - Culture in Testing Services
• Skills V ability
• Comfort zone
• Resistance
Below the Water - Culture of Testing Professionals
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Process of Change
Evaluation
• Success ?
• Targets met ?
• Next step
– Assessment
– Adjust improvement actions
– New targets or areas of consideration
– Next improvement process
– Stop
performevaluation
formulateplan
implementimprovement
actions
commitment
executeassessment
defineimprovement
actions
objectives scope
approach
Barclays Process of Change
• Testing (Testing Services) on the map
• Master Test Plan: integral risk and test control
• First steps in automation
• Improved morale
• Roles, career path and training available
• Baseline established
• Gaining experience
• Reduced costs ?
• Reduced timescales ?
Achieved Benefits
Agenda
• Introduction to TPI
• Process of change
– Theory
– Barclays Experience
• Lessons learnt
Lessons learnt
• Implementation requires competent testing and project management professionals, and sound leadership
• Accept that resistance is an integral part of any business change project
• Don’t over engineer the solution - plan for incremental change over time
• Pay attention to all 4 cornerstones of testing
• Tools can only support the lifecycle and techniques
Stay focussed on Testing!
Thank you !
Martin Pol Martin Pol has more than 25 years experience in the information business. Since 1983 he's completely dedicated to Testing. He's gained experience by managing testing processes and implementing structured testing in many organizations in different branches. With exceptional insight and experience in practical testing issues, Martin is a regarded presenter at conferences and training sessions throughout Europe, the Americas and Asia. As Manager R&D of IQUIP Informatica B.V. he was responsible for the creation of the structured testing approach TMap(r) and the Test Process Improvement model, TPI(r). Martin Pol is the co-author of three (Dutch) books on TMap(r) and TPI(r) which have been translated in English and German. A Japanese version is due in 2001. Under his management TMap(r) became a de facto testing standard. Now the approach is used in more than 400 companies Europe and the USA. The international publication of the TPI(r) book in English and German caused world wide application by many key-players in the IT-industry, e.g. NOKIA, Deutsche Telekom, AXA, Dell, Compaq, Intel, Novell, ABN-AMRO bank, Barclays bank, etc. Since the model is complementary to Software Process Improvement models such as the CMM, TPI(r) is frequently used within Software Engineering Process Groups. Martin Pol was Programme Chair of the EuroSTAR conference in 1996 and 1997 and is chairman of the Dutch Special Interest Group in Software Testing, TestNet. He recently received the "European Testing Excellence Award" for his contribution to the field of testing across Europe. Through his company Polteq IT Services B.V. he's offering international test consultancy services now. TMap(r) and TPI(r) are registered trademarks of IQUIP Informatica B.V. in the Netherlands. Martin Pol was the former manager R&D of IQUIP Software Control and responsible for the development of both the TMap(r) approach and the TPI(r) model.
Recommended