2015 06-03 ti4 agile presented at ncs

logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png

R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen


Test Improvement 4 Agile


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

The Evolution of IT and Testing

Pioneering

Structuring

Op>mizing


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Prototyping Waterfall Agile/Scrum V-‐model Spiral model DEVOPS Itera>ve RAD TDD ATDD BDD W-‐model XP Con>nuous Integra>on/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB TMap-‐Next Packages SOA Devices Social media Virtualiza>on Cloud Mobile Internet of Things Web Localiza>on Big Data

The Evolution of IT and Testing


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools Working software over comprehensive documentation

Customer collaboration over contract negotiation Responding to change over following a plan

That is, while there is value in the items on

the right, we value the items on the left more.


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Test Improvement 4 Agile roadmap

Quick wins, good

prac>ces Goals, scope

Interviews, mee>ngs

Assessment

Awareness, commitment,

buy-‐in Implement and evaluate


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Test Improvement 4 Agile roadmap

Cost


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Agile testing maturity levels Forming Norming Performing


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Agile testing maturity levels Forming Norming Performing

Set the basis and the first steps towards working in

an Agile manner

Adopt a process that facilitates the Agile view

on working

Continuously improve the way

you work by living the Agile

way


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Assessment model

Key area Forming Norming Performing 1 Key area 1 1 2 3 4 1 2 3 4 1 2 3 4

2 Key area 2 1 2 3 4 1 2 3 4 1 2 3 4

3 Key area 3 1 2 3 4 1 2 3 4 1 2 3

… …… … … … … … … … … … … … …

n Key area n 1 2 3 1 2 3 1 2 3 4

•  Where am I? •  Where do I want to be? •  How do I get there?


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Test process

•  What and when to test? •  Finish what you start •  Fully integrated •  Everybody tests

Forming Norming Performing

TI4Agile

Risk based testing

Testing in and over sprints

Everybody tests


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

•  Automated tests: unit ß à acceptance •  Non-functional testing •  Test data •  Regression testing •  Fit for purpose & context


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Test management

•  People management •  Generic test approach / strategy •  Risk analysis •  Release planning •  Birds eye view:

Cross teams – sprints – projects


TI4Agile

Generic process

Support the process

Support the people


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Generic test approach / strategy

Keys to success

Risks Maturity

Skills Product

Size Time

Culture

Ban

dwid

th


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

The wheel of risk

Core

Layer 1

Layer 2

Core

Layer 1

Layer 2

Core

Layer 1

Layer 2

Core

Core

Core


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Planning & Estimation

•  Work breakdown & plan for all activities –  Preparation, Review, Execution, Retest, Unit test, Bug

fixing •  It is a team effort!


TI4Agile

Know what to plan and estimate

Measurable, small parts

Time is fixed, scope is flexible


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

System

Item Item Item Item

Item Item Item Item

Item Item Item Item

Keys to success

13 8

20

13


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Defect management

•  Use one system for all defects and ensure traceability •  Decide what and when to log


TI4Agile

Defects are logged

Not all defects are logged

Focus on working software


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

•  Take into account: –  Co-location –  Maturity of team –  Number of defects –  Organizational influence –  Drive for metrics –  Size of development –  Complexity of development


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Test environment

•  Self controlled •  Production like •  Version controlled •  Continuous integration


TI4Agile

There is an environment

The team is stakeholder

The environment is

flexible


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

Virtualization Version control

1

4

9

2

3

6

7

5

8

T1

T2 10

Tags

Trunks

Branches Merges

Discontinued development

branch


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Test automation

•  At every level •  Important part of the sprint •  Risk based •  Maintainable


TI4Agile

Test automation

provides added value

Test automation at different test

levels

Maintainable and continuous


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

Maintainable Continuous integration

Testware Test scripts – Test cases – Test data – Test

environment

System Under Test Software – Service – Platform - Infrastructure

Management Reporting

Projects Changes

Patches Fixes

Test Automation


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Regression & E2E testing

•  Focus wider than current sprint –  Probably wider than the team

•  Assign where and when •  Business value


TI4Agile

Regression and E2E in

focus

In and over iteration

Proactively addressed


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

•  Understandable, visualized, clear E2E information

•  Optional: –  Hardening Sprint focused on improving the product

•  E2E, Regression, Loose ends, Shortcuts, Non-functionals


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

People


TI4Agile

People are motivated

People embrace

changes in the product

People initiate process changes


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

Feedback •  State something positive related to the subject •  State your criticism objectively •  Don’t use the word “but” •  State the effect •  Suggest an improvement

Thank you for delivering a lot of information in the meeting. I noticed that you were talking a lot, this provided me little room to give my opinion. You might

want to ask others for their input in the future.


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

•  Know the context •  Assist others

•  Know your own strengths •  Leave your comfort zone

30


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Interaction

•  Actively share information •  Open and honest •  Use meetings for their purpose

–  Understandable for all attendees


TI4Agile

Traceable decisions

Focussed meetings

Ad-hoc, when needed


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

Communication = essential •  SCRUM-board

32


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Teamwork

•  Respect and trust each other •  Commitment as a team

–  Work towards a team goal •  Help each other where possible

–  Multi disciplined


TI4Agile

The team executes a

task

The team is committed

The team is self managing


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Keys to success

•  Keep the goals visible

•  Team development

Bruce W. Tuckman: Stages of group development

34


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Test profession

•  Test skills •  Soft skills •  Technical skills (drivers, stubs, logging, …) •  Share test knowledge

–  Within the discipline, over teams –  Within team, over disciplines


TI4Agile

Proficient in testing

Focus on soft and technical

skills

Profession includes more

than just testing


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Key to success

•  Charters, SBTM, … •  Exploratory testing

–  Simultaneously learn, plan, design, execute and report


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Stakeholder commitment

•  Deliver acceptance criteria and share the context •  Recognise the value of the team and of each role •  Prioritise backlog, risks and defects •  Create an effective environment •  Participate in acceptance •  Allow freedom of choice


TI4Agile

Stakeholders start projects

Stakeholders help projects

Stakeholders participate in

projects


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

The path to commitment

Contact Awareness

Understanding

Posi2ve percep2on

Adop2on

Ins2tu2onali-‐za2on

Internaliza2on

Level of com

mitm

ent


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Assessment model

Key area Forming Norming Performing 1 Stakeholder commitment 1 2 3 4 1 2 3 4 1 2 3

2 Planning & Estimation 1 2 3 4 1 2 3 4 1 2 3 4

3 People 1 2 3 4 1 2 3 4 1 2 3

4 Interaction 1 2 3 4 1 2 3 4 1 2 3 4

5 Teamwork 1 2 3 1 2 3 4 1 2 3 4

6 Test process 1 2 3 1 2 3 4 1 2 3

7 Test management 1 2 3 4 1 2 3 1 2 3

8 Test profession 1 2 3 4 1 2 3 4 1 2 3

9 Test automation 1 2 3 1 2 3 4 1 2 3 4

10 Regression & E2E testing 1 2 3 4 1 2 3 1 2 3 4

11 Defect management 1 2 3 1 2 3 4 1 2 3

12 Test environment 1 2 3 1 2 3 1 2 3 4


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Transitioning from traditional to agile Process Waterfall

Development Iterative &

Incremental Agile

Development

Measure of Success

Management Culture

Requirements & Design

Coding & Implementation

Test & Quality Assurance

Planning & Scheduling

Conformance to plan

Response to change, working code

Command & control

Leadership collaborative, empowerment of teams

Big & upfront documentation

Continuous / emergent / just in time elaboration

Code all features in parallel. Test later

Code & unit test, deliver serially

Big, planned / test late Continuous & concurrent testing starting early

detailed / fixed scope, estimate time & resources

Two-level plan / fix date, estimate scope

(Scaling Software Agility: Best Practices for large Organisations – DeanLeffingwell 2007)


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

Key area Forming Norming Performing 1 Stakeholder commitment 1 2 3 4 1 2 3 4 1 2 3

2 Planning & Estimation 1 2 3 4 1 2 3 4 1 2 3 4

3 People 1 2 3 4 1 2 3 4 1 2 3

4 Interaction 1 2 3 4 1 2 3 4 1 2 3 4

5 Teamwork 1 2 3 1 2 3 4 1 2 3 4

6 Test process 1 2 3 1 2 3 4 1 2 3

7 Test management 1 2 3 4 1 2 3 1 2 3

8 Test profession 1 2 3 4 1 2 3 4 1 2 3

9 Test automation 1 2 3 1 2 3 4 1 2 3 4

10 Regression & E2E testing 1 2 3 4 1 2 3 1 2 3 4

11 Defect management 1 2 3 1 2 3 4 1 2 3

12 Test environment 1 2 3 1 2 3 1 2 3 4

Questions?


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen


Test Improvement 4 Agile

Ruud Teunissen Senior Test Consultant

Polteq Test Services BV [email protected] – @RuudTeunissen


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

01. Stakeholder commitment – Forming

1.  The principal stakeholder is defined and known by the testers

2.  Stakeholders deliver the committed resources

3.  Stakeholders actively acquire information on the progress of the project

4.  Stakeholders are willing to adapt their way of working to the test process


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

01. Stakeholder commitment – Norming

1  The stakeholders provide a mandated representative (PO) to the team

2  The stakeholders define business value and provide a prioritized product backlog

3  The stakeholders define acceptance criteria for the items on the product backlog

4  Stakeholders attend the review meeting


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

01. Stakeholder commitment – Performing

1  The product owner is actively participating in the team activities

2  Stakeholders trust and value the team(s)

3  Stakeholders share responsibility for the quality of the product


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

02. Planning & Estimation – Forming

1.  Each test activity is planned (prepare - define - execute)

2.  Test levels are defined and overlap is minimized (UT – ST - AT)

3.  A technique for test effort estimation is applied

4.  Planning is agreed with the stakeholders


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

02. Planning & Estimation – Norming 1  Estimation is used to estimate team effort (cross

functional)

2  The planning meeting is used to elaborate on user stories

3  Testing work is broken down into measurable, small tasks (preferably 8hrs or less)

4  Burn-down (or up) charts are used to keep track of progress


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

02. Planning & Estimation – Performing 1  Scope is adjusted when estimation seems to be

incorrect, since time boxes are fixed

2  Changes on user stories are fit into the planning wherever possible

3  The team(s) decide what is delivered in an iteration

4  Team velocity is used in planning


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

03. People – Forming

1.  People are well trained and/or experienced in their functions

2.  People are willing to put in extra effort when needed (commitment)

3.  People can explain their value in the project context

4.  People take full responsibility for their work


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

03. People – Norming

1  People understand SCRUM terminology and know the purpose of the different SCRUM meetings

2  SCRUM master keeps track of the process

3  SCRUM master removes roadblocks outside the team

4  People have a positive attitude towards change


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

03. People – Performing

1  People proactively provide feedback

2  People know how to handle feedback and use the feedback to improve

3  People are able to help with tasks outside their main area of expertise (T-shaping)


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

04. Interaction – Forming

1.  It is possible to trace back points of action, agreements and decisions

2.  Progress, product quality and risks are communicated

3.  Meetings are only attended by relevant people

4.  Tooling to support the necessary communication within the project is supplied


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

04. Interaction – Norming

1  The stand-up meeting is used to actively monitor progress and adjust the planning accordingly

2  The planning meeting is used to discuss and identify risks

3  The retrospective deals with positive points and improvement suggestions

4  Documentation reflects the current state of the product


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

04. Interaction – Performing

1  People use ad-hoc meetings to obtain or communicate relevant information

2  Testers share their test ideas as soon as possible

3  Documentation is fit for purpose (just enough)

4  Good practices are shared over the teams (and projects)


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

05. Teamwork – Forming

1.  The team is co-located

2.  Each team member is involved in important decisions

3.  The team understands its goal


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

05. Teamwork – Norming

1  Testing is an integrated task of a multi disciplinary team

2  The team commits to the scope in a sprint

3  SCRUM coaching is provided to improve the teamwork

4  The team contains all skills necessary to deliver a working product


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

05. Teamwork – Performing

1  Team members know and value each other

2  Specialists share their expertise to improve the knowledge in the team

3  The team is responsible for a sufficient quality product

4  Team members can take on different roles instead of being limited to their function


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

06. Test process – Forming

1.  The effort for test tasks is based on product risks

2.  Test design techniques are applied when creating test cases

3.  Testing considers different quality characteristics


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

06. Test process – Norming

1  Non-functionals are addressed in and over sprints

2  Pairing is used to improve the test quality

3  The Definition of Done includes items related to testing

4  Developers use simple techniques for unit testing


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

06. Test process – Performing

1  Testing debt is monitored and managed

2  User acceptance is fully integrated in the process

3  All team members have a responsibility for the test process


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

07. Test management – Forming

1.  Test management allocates the proper test staff

2.  A generic test approach is available

3.  Test management provides birds eye view on quality and risks in the project

4.  Test management contributes to the release planning by adding a quality view to high level estimation


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

07. Test management – Norming

1  Test management initiates risk identification at product backlog level

2  Test management supports the product owner(s) to identify dependencies between user stories prior to finalizing the release planning

3  Test management supports the SCRUM master in removal of test roadblocks


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

07. Test management - Performing

1  Test management facilitates test meetings to share the testing pitfalls and lessons learned over teams (and projects)

2  Test management provides mentoring to the teams to improve the quality of testing

3  Test management increases awareness and understanding of Agile testing within the organization


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

08. Test profession – Forming

1.  Testers have received specific test training and/or have sufficient experience in the field of structured testing

2.  Testers can explain the rationale behind chosen techniques that have been applied

3.  Test functions are part of the organizations career development

4.  Testers can explain their added value


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

08. Test profession – Norming

1  Testers have received training(s) regarding soft skills

2  Technical activities are part of the testing job (e.g. working with drivers/stubs, code reviews)

3  Testers provide support to testing activities of other team members

4  Testers use exploratory testing in addition to using structured techniques


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

08. Test profession – Performing

1  Testers are proficient in at least one programming language

2  Testers are proactively gathering relevant information to be able to execute their tasks and to minimize waiting time

3  Testers strive to continuously improve their testing


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

09. Test automation – Forming

1.  The organization acknowledges the importance of test tooling and allocates sufficient budget

2.  Test cases are automatable by describing them using ‘inputs - actions - expected results’

3.  Decisions on what to automate are based on risks and return on investment


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

09. Test automation – Norming

1  Automated unit testing is part of the development process

2  Tests are automated to save time for manual testing

3  Test cases are designed for automation

4  Test automation is seen and treated as software development


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

09. Test automation – Performing

1  Automation is modular and maintainable

2  Requirements are set up to be executable

3  Teams are allowed to try different tooling to reach their own optimal setting

4  Test automation is used to support continuous integration


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

10. Regression & E2E testing – Forming

1.  There is a basic strategy for regression testing

2.  There is a basic strategy for E2E testing

3.  Test sets are maintained and updated

4.  Responsibilities for E2E and regression testing are clear


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

10. Regression & E2E testing – Norming

1  Regression testing is part of every iteration

2  Regression and E2E testing is applied over multiple iterations and when needed over multiple teams and multiple projects

3  The user stories contain information on relevant E2E functionality


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

10. Regression & E2E testing – Performing

1  The team is aware of the value of and need for E2E and regression testing

2  All testing is used to address current business value

3  Regression test sets can be easily scoped to address specific areas of functionality

4  E2E test possibilities are utilized continuously


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

11. Defect management – Forming

1.  All persons involved in logging and/or tracking defects use the same defect tracking tool

2.  Defects have a common set of characteristics (e.g. unique id, description, status)

3.  All raised defects are followed up


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

11. Defect management – Norming

1  There are simple criteria for deciding when to log a defect

2  Defects that can not be fixed within the sprint have to be added to the product backlog

3  Defects, either logged or not logged, are communicated with the complete team

4  Impact of defects on the product, but also on the sprint needs to be taken into account


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

11. Defect management – Performing

1  The team strives to fix defects as soon as possible (<1 day)

2  The person that logged the defect helps to solve the defect

3  Defects are logged with the minimal set of information needed to follow up the defects


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

12. Test environment – Forming

1.  The environment is managed (setup, maintenance, version management, etc.)

2.  The environment is sufficiently representative for the test

3.  The data used in the environment can easily be refreshed


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

12. Test environment – Norming

1  The test environment is always available to the team

2  Changes to the environment are communicated to the involved teams

3  The version management system used for the environment supports branching


R G B

35 30 96 Blauw

232 62 38 Rood

108 174 68 Groen

12. Test environment – Performing

1  Testing takes place on all relevant environments

2  The environment supports the use of continuous integration

3  The team manages its own environments

4  A mechanism to efficiently support testing in different configurations is in place (e.g. virtualization)

Technology

2015 06-03 ti4 agile presented at ncs