Software Quality Assurance in Agile World

Arto Stenberg

Co-Creating What’s Next 2

Multi-National locations to support your BPO/ITO and R&D Innovations

About Us

PEOPLE

18,000+

United States 4,500

China 5,500

Philippines 7,500

Europe & South America 800

GEOGRAPHIC MIX

ANNUAL SALES

$440M

Telecom 20%

Technology 25%

Financial Services 15%

Consumer, Retail, F&B 10%

INDUSTRY MIX

Media & Entertainment 25%

LOCATIONS

34

Customer Care & Support

SERVICE MIX

Sales Management & CRM

Help & Technical Support

Application Services

Product Engineering & R&D

Quality Assurance & Testing

Globalization

Data & Analytics

Infrastructure & DevOps

Co-Creating What’s Next 3

Symbio Services & Solutions

Technology

Platforms

Managed

Services

• App Services

• Cloud Services

• Full Stack

Development

• Maintenance

• Connected Car

• WiSh Station

• IoT solutions

• Azure solutions

• Full end-to-end

solutions

• Mobile platforms

• Realtime OS’s

• High performance

• Various chipsets

• Fast bootup needs

• Long battery life

• Optimization

• Embedded Dev

• Device SW

Maintenance

• Training

• TotalQA

• Testing Services

• Test Automation

• Device Testing

• I18n Engineering

• L10n Testing

• Deployment &

Local Support

• China Market

Entry

• SeLion

• App. Intake

Moderation

• Mobile Test

Automation

• App. Intake

Automation

• SeLion

• RATA

App & Cloud Embedded GlobalizationQA & Testing

Customer Experience Management

• Training

Enablement

• Contact Center

Apps

• 24/7 Help Desk &

Tier 1 -3 Services

• Customer Care

• Tech Support

• Sales

• Global Coverage

Contact Center

Co-Creating What’s Next 4

Co-Creating What’s Next 5

Quality

Assurance

Co-Creating What’s Next 6

The Nature of SW Development

Development Testing Accepted

Development Testing AcceptedFault

fixTesting

HappyCustomer

UnhappyCustomer

Planned:

Reality:

Dev

Iteration

Reqs

Regs

Co-Creating What’s Next 7

› Defect prevention

› Are you building the right thing in the right way?

› Process oriented (e.g. ISO 9001 - 2015, CMMI, SPICE)

› There exists an ideal process, which is described in the

standards, that every company needs to implement and

follow.

Quality Assurance (QA):

Co-Creating What’s Next 8

› Quality Control (QC):

- Product oriented (e.g. ISO 25010 ”SQuaRE”)

- Are the results of what you have build, what you expected?

› Testing:

- Isn't assuring quality, it's monitoring and controlling it

- Defect detection

Quality Control and Testing

Co-Creating What’s Next 9

USA SW Quality Metrics per Function Point (C.Jones)

OriginDefect Potential

Best / Average / PoorRemoval Efficiency

Best / Average / PoorDelivered Defects

Best / Average / Poor

Reqs 0.40 1.00 1.50 85 % 77 % 50 % 0.08 0.23 0.75

Design 0.60 1.25 2.20 97 % 85 % 50 % 0.02 0.19 1.10

Code 1.00 1.75 2.50 99 % 95 % 80 % 0.01 0.09 0.50

Docs 0.40 0.60 1.00 98 % 80 % 70 % 0.01 0.12 0.30

Fixes 0.10 0.40 0.80 95 % 70 % 50 % 0.01 0.12 0.40

TOTAL 2.50 5.00 8.00 96 % 85 % 62 % 0.13 0.75 3.05

Co-Creating What’s Next 10

› Technical or Structural quality

› Process quality

› Usage quality

› Service quality

› Aesthetic quality

› Standards quality

Quality Has Many Faces

Co-Creating What’s Next 11

1. Low defect potentials

2. Effective defect prevention methods

3. High defect detection efficiency (DDE)

4. High defect removal efficiency (DRE)

5. Use of pretest inspections

6. Use of pretest static analysis

7. Use of formal test case design

8. Good ease of learning

9. Good ease of use

10. Good technical support

11. High user satisfaction

12. Good warranty

The 12 Most Effective Software Quality Factors

Co-Creating What’s Next 12

Quality Assurance

In

Agile World

Co-Creating What’s Next 13

If the computer program in question is

being developed for the first time, arrange

matters so that the version finally

delivered to the customer for operational

deployment is actually the second version

insofar as critical design/operations areas

are concerned.

“Managing the Development of Large Software

Systems” by Winston Royce 1970

Co-Creating What’s Next 14

Birth of Scrum

Harvard Business Review: January-February 1986

Co-Creating What’s Next 15

› Leading companies show six characteristics in managing

their new product development processes:

1. Built-in instability

2. Self-organizing project teams

3. Overlapping development phases

4. "Multilearning”

5. Subtle control

6. Organizational transfer of learning

Moving the Scrum Downfield

Co-Creating What’s Next 16

› The lack of a standard and well-understood structure

opens up the team to the chance of power struggles

and disruptive social conflicts.

› Scaling Agile up from small applications to large

systems with multiple teams in multiple locations has

proven to be complicated and difficult.

› Poor measurement practices associated with Agile and

with many self-organizing teams give the method the

aura of a cult rather than of an engineering discipline.

Caution About Self-Organizing Teams (C.Jones)

Co-Creating What’s Next 17

QA

Product Owner

Team

Co-Creating What’s Next 18

› Product & release & requirement & defect management

› Stakeholder / customer collaboration

› Product backlog, prioritization & grooming

› Definition of Done(s)

› Define & verify acceptance criteria & tests

› Collaboration with the team

› Participate in Sprint Review & Retrospective

Product Owner

Co-Creating What’s Next 19

› Communication with Product Owner

› Communication inside the team

› Managing technical debt

› Sprint Planning, requirements & stories

› Estimations (story points = dev time + test time + DoD)

› Integration & testing & defect management

› Definition of Done(s)

Team

Co-Creating What’s Next 20

The appearance of SAFe and DAD.

Multiple Teams at Enterprise Level

Co-Creating What’s Next 21

Co-Creating What’s Next 22

› Janet Gregory & Lisa Crispin:

- More Agile Testing

› Capers Jones:

- Software Engineering Best Practices

› Kenneth S. Rubin:

- Essential Scrum

› Stefan Wagner:

- Software Product Quality Control

References

Co-Creating What’s Next

Thank you!