Software Quality Assurance in Agile World
Arto Stenberg
Co-Creating What’s Next 2
Multi-National locations to support your BPO/ITO and R&D Innovations
About Us
PEOPLE
18,000+
United States 4,500
China 5,500
Philippines 7,500
Europe & South America 800
GEOGRAPHIC MIX
ANNUAL SALES
$440M
Telecom 20%
Technology 25%
Financial Services 15%
Consumer, Retail, F&B 10%
INDUSTRY MIX
Media & Entertainment 25%
LOCATIONS
34
Customer Care & Support
SERVICE MIX
Sales Management & CRM
Help & Technical Support
Application Services
Product Engineering & R&D
Quality Assurance & Testing
Globalization
Data & Analytics
Infrastructure & DevOps
Co-Creating What’s Next 3
Symbio Services & Solutions
Technology
Platforms
Managed
Services
• App Services
• Cloud Services
• Full Stack
Development
• Maintenance
• Connected Car
• WiSh Station
• IoT solutions
• Azure solutions
• Full end-to-end
solutions
• Mobile platforms
• Realtime OS’s
• High performance
• Various chipsets
• Fast bootup needs
• Long battery life
• Optimization
• Embedded Dev
• Device SW
Maintenance
• Training
• TotalQA
• Testing Services
• Test Automation
• Device Testing
• I18n Engineering
• L10n Testing
• Deployment &
Local Support
• China Market
Entry
• SeLion
• App. Intake
Moderation
• Mobile Test
Automation
• App. Intake
Automation
• SeLion
• RATA
App & Cloud Embedded GlobalizationQA & Testing
Customer Experience Management
• Training
Enablement
• Contact Center
Apps
• 24/7 Help Desk &
Tier 1 -3 Services
• Customer Care
• Tech Support
• Sales
• Global Coverage
Contact Center
Co-Creating What’s Next 4
Co-Creating What’s Next 5
Quality
Assurance
Co-Creating What’s Next 6
The Nature of SW Development
Development Testing Accepted
Development Testing AcceptedFault
fixTesting
HappyCustomer
UnhappyCustomer
Planned:
Reality:
Dev
Iteration
Reqs
Regs
Co-Creating What’s Next 7
› Defect prevention
› Are you building the right thing in the right way?
› Process oriented (e.g. ISO 9001 - 2015, CMMI, SPICE)
› There exists an ideal process, which is described in the
standards, that every company needs to implement and
follow.
Quality Assurance (QA):
Co-Creating What’s Next 8
› Quality Control (QC):
- Product oriented (e.g. ISO 25010 ”SQuaRE”)
- Are the results of what you have build, what you expected?
› Testing:
- Isn't assuring quality, it's monitoring and controlling it
- Defect detection
Quality Control and Testing
Co-Creating What’s Next 9
USA SW Quality Metrics per Function Point (C.Jones)
OriginDefect Potential
Best / Average / PoorRemoval Efficiency
Best / Average / PoorDelivered Defects
Best / Average / Poor
Reqs 0.40 1.00 1.50 85 % 77 % 50 % 0.08 0.23 0.75
Design 0.60 1.25 2.20 97 % 85 % 50 % 0.02 0.19 1.10
Code 1.00 1.75 2.50 99 % 95 % 80 % 0.01 0.09 0.50
Docs 0.40 0.60 1.00 98 % 80 % 70 % 0.01 0.12 0.30
Fixes 0.10 0.40 0.80 95 % 70 % 50 % 0.01 0.12 0.40
TOTAL 2.50 5.00 8.00 96 % 85 % 62 % 0.13 0.75 3.05
Co-Creating What’s Next 10
› Technical or Structural quality
› Process quality
› Usage quality
› Service quality
› Aesthetic quality
› Standards quality
Quality Has Many Faces
Co-Creating What’s Next 11
1. Low defect potentials
2. Effective defect prevention methods
3. High defect detection efficiency (DDE)
4. High defect removal efficiency (DRE)
5. Use of pretest inspections
6. Use of pretest static analysis
7. Use of formal test case design
8. Good ease of learning
9. Good ease of use
10. Good technical support
11. High user satisfaction
12. Good warranty
The 12 Most Effective Software Quality Factors
Co-Creating What’s Next 12
Quality Assurance
In
Agile World
Co-Creating What’s Next 13
If the computer program in question is
being developed for the first time, arrange
matters so that the version finally
delivered to the customer for operational
deployment is actually the second version
insofar as critical design/operations areas
are concerned.
“Managing the Development of Large Software
Systems” by Winston Royce 1970
Co-Creating What’s Next 14
Birth of Scrum
Harvard Business Review: January-February 1986
Co-Creating What’s Next 15
› Leading companies show six characteristics in managing
their new product development processes:
1. Built-in instability
2. Self-organizing project teams
3. Overlapping development phases
4. "Multilearning”
5. Subtle control
6. Organizational transfer of learning
Moving the Scrum Downfield
Co-Creating What’s Next 16
› The lack of a standard and well-understood structure
opens up the team to the chance of power struggles
and disruptive social conflicts.
› Scaling Agile up from small applications to large
systems with multiple teams in multiple locations has
proven to be complicated and difficult.
› Poor measurement practices associated with Agile and
with many self-organizing teams give the method the
aura of a cult rather than of an engineering discipline.
Caution About Self-Organizing Teams (C.Jones)
Co-Creating What’s Next 17
QA
Product Owner
Team
Co-Creating What’s Next 18
› Product & release & requirement & defect management
› Stakeholder / customer collaboration
› Product backlog, prioritization & grooming
› Definition of Done(s)
› Define & verify acceptance criteria & tests
› Collaboration with the team
› Participate in Sprint Review & Retrospective
Product Owner
Co-Creating What’s Next 19
› Communication with Product Owner
› Communication inside the team
› Managing technical debt
› Sprint Planning, requirements & stories
› Estimations (story points = dev time + test time + DoD)
› Integration & testing & defect management
› Definition of Done(s)
Team
Co-Creating What’s Next 20
The appearance of SAFe and DAD.
Multiple Teams at Enterprise Level
Co-Creating What’s Next 21
Co-Creating What’s Next 22
› Janet Gregory & Lisa Crispin:
- More Agile Testing
› Capers Jones:
- Software Engineering Best Practices
› Kenneth S. Rubin:
- Essential Scrum
› Stefan Wagner:
- Software Product Quality Control
References
Co-Creating What’s Next
Thank you!