Archana Joshi Testing in agile is it easier said than done

3rd Asia Pacific Software Testing Conference

Testing in Agile

Easier said than done?

Archana Joshi Wipro Consulting Services

archana.p.joshi@gmail.com

© 2009 Wipro Ltd - Confidential 2 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 2

Who am I?

About Myself - Archana Joshi:

•Manager - Agile practice at Wipro Consulting Services • Certified Scrum Practitioner with 9+ years in software development, management and consulting • Involved as a scrum master, process consultant, agile coach on various teams

About Wipro Consulting Services:

• Wipro Consulting Services (WCS) is a division of Wipro Ltd (NYSE: WIT), a $5.5bn global enterprise. • WCS has 1200+ consultants across the globe providing unique solutions to our clients worldwide. • The Agile initiative started in 2004 and agile is now one of the mainstream processes at Wipro. • WCS has a dedicated agile practice which consults teams and clients on agile implementation.

© 2009 Wipro Ltd - Confidential 3 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 3

Program Landscape

Development program for a new web based application in wealth management space

• Technology J2EE / Web 2.0 / Flex • Involves integration with 3rd party components like – SFDC, SiteMinder, LDAP AD, Akamai

Drivers for agile • Respond to changing needs of the business • Improve the relationship among the technology and business partners • Gain competitive edge in dynamic financial market through reducing time to market

Team

• Program divided across 3 agile tracks - based on functionality • Each track was a cross-functional distributed team involving

•Developers (vendor 1 & customer IT) •Testers (vendor2), BA (vendor3) • User Experience personnel (vendor 4) • Scrum Master (vendor1), Product Owner (customer business)

• Shared resources included business, legal, architecture, security and compliance teams

Agile Awareness • Excluding the scrum masters, entire team was new to agile • Team had undergone 2 days class room training on agile

© 2009 Wipro Ltd - Confidential 4 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 4

• New Product development

– Integration with other 3rd party applications

– Integration across multiple tracks

• Test Team belonged to different hierarchy within the organization

– Forming cross-functional teams

– Work together as a cohesive team

• Compliance to financial and security regulations

– Testing to consider these requirements too

• Team new to agile way of working

– Documentation needed for everything to start work

– All manual testing, minimal automation

• Team distributed across locations

– Multiple location, time zone challenges

Challenges for testing

© 2009 Wipro Ltd - Confidential 5 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 5

Overall Process and Release Cycle

Stabilization 2 week Iterations of Development with Testing

Production Live Interim integration points

2 wks 1 wk

2 wks 2 wks

2 wks 1 wk 2 wks 1 wk

2 wks 2 wks 2 wks 2 wks 2 wks 2 wks 5 wks

2 wks 2 wks 2 wks 2 wks 2 wks 2 wks 5 wks

2 wks 1 wk 2 wks 1 wk 2 wks 2 wks

2 wks

2 wks

Track 1

Track 2

Track 3

Release Cycle

Initiation Stabilization Iteration Release and Deployment

Process

© 2009 Wipro Ltd - Confidential 6 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 6

Key Lessons Learnt from the program

#1 Team Involvement - Sooner the better

#2 Divide and rule

#3 Work together as a team

#4 Code will change. Accept it.

#5 Readiness checks are necessary

#6 Do not forget to measure

#7 Establish governance mechanism at start

© 2009 Wipro Ltd - Confidential 7 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 7

#1 Team Involvement - Sooner the better

• The test team was involved right from the initiation & planning stage • Test strategy formed during the initiation stage.

• It contained the high level approach on how system shall be tested • It also contained the responsibilities of the testers within the team at different stages of the project

• For estimations and planning along with the business and development teams, testers were also involved • Early involvement helped testers in understanding business perspective and dependencies better

© 2009 Wipro Ltd - Confidential 8 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 8

#2 Divide and rule

• Instead of trying to fit all types of tests within an iteration, division based on types was done • Testing within an iteration

• Feedback provided by business, UX team & testers • Unit testing & Integration testing (by developers) • Functional testing (by testers with guidance from BA) • Regression testing (by testers)

• Testing outside an iteration (stabilization)

• Performance testing • Security (ethical hack) testing • Final UAT & Regression testing • Exploratory testing by business and legal teams

© 2009 Wipro Ltd - Confidential 9 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 9

#3 Work together as a team

• Though there were multiple vendors spread across different locations and time zones, the cross-functional team worked as a cohesive team • The scrum master had to be on a constantly look out for any team behaviors which can result into group-ism and arrest the behavior at the start • Collaboration tools like net meeting, VCON, TCON were extensively used • Some of the key team members in the team – team leads, sr. developers, testers, BA came together at same physical location for planning and initial iteration of the program

© 2009 Wipro Ltd - Confidential 10 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 10

#4 Code will change. Accept it.

• Testers typically test on a code which is fairly stable • When an iteration is 2 weeks long, there are code changes till the last day of the iteration • There were challenges in changing the mindset of the testers that the code undergoing test is going to change frequently during development of the iteration • During iteration

• At end of the day, developers provided links to their code from the development environment which the testers used to provide feedback on. • This was considered as feedback & not defects as the iteration was work in progress • At end of the iteration which ever feedback was not addressed was entered as defects in the Quality Center by the test team • Test team was responsible for collating the feedback from business and UX teams as well

C

h

a

n

g

e

© 2009 Wipro Ltd - Confidential 11 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 11

#5 Readiness checks are necessary

• Agile does not mean that the checks or sign-off are no longer to be taken by the test team • Within an iteration

• Review were done by the BA and business on the test cases and test plan to ensure coverage • “Done” ness criteria was established for tracking the iteration completeness. • “Done” ness criteria also included the compliance and security checks

• Outside an iteration • Readiness criteria to move from one environment (integration -> staging -> production) was established by the test team

Demo Env. /

Dev. Env.

Integration

Env

Staging

Env.

Production

Env.

Test Environment during

stabilization Test Environment during iterations

© 2009 Wipro Ltd - Confidential 12 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 12

#6 Do not forget to measure

• Monitoring of metrics is important. Especially for a large program • HP Quality Center was used as the test management tool • Following metrics were captured

• Within an iteration • Defects which are not addressed and carried forward in an iteration • Test cases and scripts that were completed and on which environment

• Out side iteration • Defect spread based on priority • Test scripts % execution • Test scripts % passing

747

61

583

4685Total Pending Cases/Scripts

Total PassTotal Fail

Total Percentage Passed

Total Scripts Blocked/ Descoped

Execution Status

Total Test Cases/Scripts Planned

Total Executed

© 2009 Wipro Ltd - Confidential 13 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 13

• The teams – business, BA, test, developer belonged to different groups within the organization

• There was a need to establish a strong governance structure which was agreeable to various stakeholders involved

• This was not finalized by the time the program started and hence there were issues seen during the initial iterations

• The PMO was made aware of the process to be followed. They provided the by-in on the process and artifacts that were produced as part of the agile process

• Learning from retrospective of each track was circulated at all levels thus ensuring management support

#7 Establish governance mechanism at start

Level 1

Level 2

Level 3 Key stakeholders from IT, business, legal, security &

compliance, PMO

Scrum Master, IT Program

Manager, IT shared teams

Track – 1: Scrum Master and Team

Track - 2: Scrum Masters and

Team

Track - 3: Scrum Masters and

Team

Weekly monitoring, Program risks, Budget, Costs, Change Management

Weekly monitoring, Metrics, Inter & Intra team dependencies IT risks and mitigation for program

Daily tracking, Project Quality, Resource Capacity

© 2009 Wipro Ltd - Confidential 14 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 14

Benefits

• Fewer number of defects found during final UAT

• Hardly any issues seen after the application went live

• Less number of testing resources required compared to the traditional approach

• Better team morale – Team feeling agile is the way to go !

• Increased appreciation between business, developer & test teams for their respective work

© 2009 Wipro Ltd - Confidential 15 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 15

In Conclusion….

• Start with simple practices that are easy to adopt

• Implement a piece meal approach

• Advanced practices like extensive Automation, Test Driven Development can come in later

• Allow the team time to adapt to the agile way of working

………..And you will realize instead of being

“Easier Said Than Done”

it will be “Easier Said And Done”

© 2009 Wipro Ltd - Confidential 16 Testing in Agile – Easier Said Than Done. © 2010 Wipro Ltd - Confidential 16

Thank you for your time !

Questions ?