Better Together: Group Exploratory Testing

W10 Session 10/26/2016 1:30:00 PM

Better Together: Group Exploratory Testing

Presented by:

Jeff Abshoff

ANSYS Canada Ltd

Brought to you by:

350 Corporate Way, Suite 400, Orange Park, FL 32073 888---268---8770 ·· 904---278---0524 - info@techwell.com - http://www.starcanada.techwell.com/

Jeff Abshoff ANSYS Canada Ltd Jeff Abshoff is a testing manager at ANSYS, Inc., a global developer of engineering simulation software. At ANSYS for more than eleven years, Jeff has worked on projects ranging from small, single team features to large cross-business-unit feature integration. With a strong emphasis on exploratory testing, he still regularly participates in testing new features. Jeff leads an international team of ten testers who are responsible for certification, performance, automated regression, integration, and exploratory testing. Jeff works to empower his team members to significantly impact product quality through early engagement in the design process; collaboration with peers, developers and other stakeholders; and exploration of the software under test.

10/24/2016

1

Better Together:Better Together: Group Exploratory Testing

Jeff Abshoff – ANSYS Incjeff.abshoff@ansys.com

STARCanada 2016STARCanada ‐ 2016

Have you ever…

• Had your team size triple?

• Had testers distributed at various sites?

• Had a team with different backgrounds/skills?

• Worked on a product with integration problems?

• Found too many defects late in the cycle?

• Sped up the product release cycle?• Sped up the product release cycle?

• Been invisible?

10/24/2016

2

I have!

Bigger, distributed team

32 2

11

1

12

10/24/2016

3

Different backgrounds and skills

Automation Interactive

Intermediate (1)

Senior (2)‐focused ‐focused

Strong Physics Knowledge

Weak Physics Knowledge

TestingJunior (7) 4 Teams

10/24/2016

4

Late defects, integration issues

22 %

Defect entry rate per release phase Defects found by core testers

Increased release cadence

Release 16.0 – Dec 2014

Release 16.1 – Release 16.2 – Release 17.0 –Mar 2015 Jul 2015 Dec 2015

10/24/2016

5

Testing an afterthought

How to…

• Facilitate team collaboration?

• Cross‐train team?

• Improve our contribution to quality?

• Give quicker feedback?

• Provide more value?

10/24/2016

6

Group Exploratory Testing

Ideal

10/24/2016

7

Virtual

1

1

3

22

1

Group Exploratory Testing

10/24/2016

8

Schedule

• 2 hour session

• Pick a date

Schedule

• Gather 3‐6 testers

• Invite other stakeholders

• TIP ‐ Take care to not overload testers

10/24/2016

9

Plan

• Start with  aquick huddle

• Plan charter

• TIP ‐ Use a template

Test

• Share screens

• TIPS:• Use two monitors

• Limit distractions

10/24/2016

10

Test

How’d you Will this also

Is this already 

reported?

ydo that?

also affect…?

Does this make 

sense to you?

Can someone reproduce 

this?

Is there a work‐

around?

Learn

• Testers learn from each other

10/24/2016

11

Learn

Record coverage

10/24/2016

12

Record issues

Follow‐up

10/24/2016

13

Record issues

• Tidy‐up loose ends

Follow‐up

10/24/2016

14

Innovation over time300

Number of GET sessions vs Time

150

200

250

Number of Sessions

0

50

100

J‐14

F‐14

M‐14

A‐14

M‐14

J‐14

J‐14

A‐14

S‐14

O‐14

N‐14

D‐14

J‐15

F‐15

M‐15

A‐15

M‐15

J‐15

J‐15

A‐15

S‐15

O‐15

N‐15

D‐15

J‐16

F‐16

M‐16

A‐16

M‐16

J‐16

J‐16

A‐16

S‐16

Other things we’ve learned

• Part of development process

• Replaces bug bash

• Doesn’t fix everything

• Beware of narrow focus

• Avoids group think

• Time ‘sink’ but quick turn around• Time  sink  but quick turn‐around

• Variety of uses

10/24/2016

15

Further growth

• Better use test coverage

• Grow testers leading sessions

• Promote use in other parts of ANSYSparts of ANSYS

How to…

• Facilitate team collaboration?

• Cross‐train team?

• Improve our contribution to quality?

• Give quicker feedback?

• Provide more value?

10/24/2016

16

Now What?

Have you ever…

• Had your team size triple?

• Had testers distributed at various sites?

• Had a team with different backgrounds/skills?

• Worked on a product with integration problems?

• Found too many defects late in the cycle?

• Sped up the product release cycle?• Sped up the product release cycle?

• Been invisible?

10/24/2016

17

How to…

• Facilitate team collaboration?

• Cross‐train team?

• Improve our contribution to quality?

• Give quicker feedback?

• Provide more value?

Group Exploratory Testing

10/24/2016

18

Thanks!