Embed Size (px)
DESCRIPTION
This case study will show you how the Social Security Administration (SSA) used HP Quality Center to implement a test center of excellence (TCOE), and how you can benefit from a similar framework. You’ll learn how SSA created standards and promoted reusable templates, artifacts, methodologies, tools, standards, and processes to reduce cost, decrease time to market, and improve quality. The presenters will also explain how they provide infrastructure support for test environments and use data and standard metrics throughout the organization to produce quality software. In addition, you’ll learn how they developed the Structured Method for Automated Regression Testing (SMART) architecture built on HP QuickTest Professional and integrated it with Quality Center to establish a TCoE. The presenters took the solution to the next level by developing and integrating test utilities that both manual and automated test engineers can use to improve productivity. They’ll explain how these utilities saved time across a variety of tasks, and how the extended architecture enables end-to-end testing of web, Web services, database, and mainframe applications and automates other testing tasks.
Citation preview
1 ©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
TCoE + SMART Architecture = SMART TCoE
David Fern – Social Security Administration
Dimple Doshi – Lockheed Martin
Opening Quotation
2
“We are what we repeatedly do.
Excellence, then, is not an act,
but a habit.”
Aristotle (384 BC – 322 BC)
“We are the Center of Excellence.”
Agenda
• What is the Test Center of Excellence (TCoE)?
• How is SSA Achieving the TCoE?
• The Structured Method of Automated Regression Testing (SMART) Architecture
• How is SSA Achieving the SMART TCoE?
• Next Steps
3
“We are the Center of Excellence.”
Quick Exercise
4
In the next 15 seconds build a paper airplane.
Time remaining:
25
151314111210976854231Time is up!
Fly your plane!
“We are the Center of Excellence.”
Quick Exercise
5
What did we learn from this exercise?
Some are better than others
Many different types
Many different materials
Some didn’t finish
“We are the Center of Excellence.”
Why?
What is a TCoE?
6
Project Team A
Project Team B
Project Team C
Project Team D
Project Team A
Project Team B
Project Team C
Project Team D
Role-based Applications
Shared Processes and
Expertise
Centralized Infrastructure
Non-TCoE
TCoE
“We are the Center of Excellence.”
Approach
What is a TCoE?
7
“We are the Center of Excellence.”
Governance
Centralization
Main Concepts
What is a TCoE?
8
“We are the Center of Excellence.”
Benefits of a TCoE Drives best practices
Established standards and processes
Efficient test management
Increased quality and productivity
Maximizes return on investment (ROI)
What is a TCoE?
9
“We are the Center of Excellence.”
Key Functions of a TCoE Implement standards, methodologies and templates
using a centralized repository
Provide infrastructure support
Optimize resources
Create actionable metrics
Deploy and maintain defect management system
What is a TCoE?
10
People Process
Technology
“We are the Center of Excellence.”
Test Center of
Excellence
The Components of a TCoE
What is a TCoE?
11
“We are the Center of Excellence.”
People
Roles and responsibilities
Specific skill sets
Organizational alignment
“We are the Center of Excellence.”
What is a TCoE?
12
Process Best Practices/Methodologies
Manual Test Processes
Automated Test Processes
Inspection
Quality Control
Quality Assurance
“We are the Center of Excellence.”
What is a TCoE?
13
Technology
Tools
Techniques
What is a TCoE?
14
Technology - Tools
Foundation
Shared Data Repository
Central Administration
Work Flows Open APIs
Functional Test Tools
WinRunnerQuickTest
ProfessionalLoadRunner
Business Process Test
Service Test WebInspect
Quality Center
Requirements Test Plan Test Lab DefectsRelease
Management
“We are the Center of Excellence.”
How is SSA Achieving the TCoE?
15
“We are the Center of Excellence.”
Who is SSA? The Social Security Administration’s (SSA)
mission is to deliver Social Security services that meet the changing needs of the public. These services include but are not limited to the issuance of Retirement Benefits, Disability Benefits and Social Security Cards.
We test the front-end and backend of our Internet, voice, and Web Service applications using automated solutions.
A CMM level 3 organization.
How is SSA Achieving the TCoE?
16
“We are the Center of Excellence.”
Maturity Stages
2001 2002 2008-Present 2005-20072003-2004
Center of Excellence
~1% HP Customers
Service Utility
6% HP Customers
Product Utility
14% HP Customers
Project Testing
80% HP Customers
Manual Testing
80% of IT Organizations
Introduction of
SMART/QTP, BPT,
Telephone
Automation and
other types of
automated testing
Establish the
Factory Floor and
Centralized Test
Management using
TD/QC
Develop
Methodology &
Templates
Standard metrics
and reports
Build the
Organization and
Initial Processes
using Pilot
Application
How is SSA Achieving the TCoE?
17
“We are the Center of Excellence.”
People Roles and Responsibilities
Management
Team/Technical leaders
Test engineers
Skills Test bed environment management
Manual/Automation testing
Backend/Legacy systems testing
End–to-End systems testing
Specialized testing
Organizational alignment
Manager
Lead 1
Tester 1 Tester 2
Lead 2
Tester 3 Tester 4
How is SSA Achieving the TCoE?
18
TCoE
TestTeam
TestTeam
TestTeam
TestTeam
TestTeam
TestTeam
P1
P2
P3
P4
P5
P6
P7P8
P9
P10
P11
P12
P13
P14
P15
P16
P18
P19
“We are the Center of Excellence.”
People P17
How is SSA Achieving the TCoE?
19
• Project Scope Agreement (PSA)
• Planning Meetings
• Standard Master Test Plan
• Detailed Functional Requirement(DFR)
• Automation Agreement
• Requirement Reviews
• Standard QC Setup
• Standard Tests (Sanity, Max Data, Functional Valid and Invalid, Navigation, End-to-End Test, etc.)
• SMART Utilities
• Standard Tests
• SMART Architecture
• Standard Processes (Triage)
RequirementsInitiationManualTesting
AutomatedTesting
Defects
• Standard Wrap up processes
• Package Scripts for Integration
• Post Production Support
Closure
“We are the Center of Excellence.”
Process
Internet Project Lifecycle (IPLC)
How is SSA Achieving the TCoE?
20
Process The Pre-Validation, Validation and Post-Validation Process
SMART QTP New Project Implementation Process
QT
P
Imp
lem
en
tatio
n
Te
am
Au
tom
atio
n
En
gin
ee
rM
an
ge
me
nt
1
Identify Project
Resource
2, 4
Document/Update
Project Scope
3
Review Project
Scope
5
Develop Project
6, 8
Develop/Update
Project
Documentation
7
Conduct Code
Review9
Check-Out Code
and Execute
Tests
10
Document and
Share Results
11
Update Code
and
Documentation
12
Conduct
Project Review
“We are the Center of Excellence.”
How is SSA Achieving the TCoE?
21
Standards - QC
“We are the Center of Excellence.”
How is SSA Achieving the TCoE?
22
Standards - Documentation
“We are the Center of Excellence.”
How is SSA Achieving the TCoE?
23
Technology Tools
Quality Center (QC)
QuickTest Professional (QTP)
LoadRunner
Business Process Test
Service Test
WebInspect
Requirements Center
In house Resource Accounting System (RAS)
Techniques
QC Framework
Integration of QC/QTP
QTP Architecture
Automated End-to-End Testing
Utilities
Environments
How is SSA Achieving the TCoE?
24
TCoE
Foundation
Shared Data Repository
Central Administration
Work Flows Open APIs
Functional Test Tools
WinRunnerQuickTest
ProfessionalLoadRunner
Business Process Test
Service Test WebInspect
Quality Center
Requirements Test Plan Test Lab DefectsRelease
Management
“We are the Center of Excellence.”
QTP
“We are the Center of Excellence.”
How is SSA Achieving the TCoE?
25
Service Test
TCoE
LoadRunner
QC
WebInspect
Requirements CenterTCoE
The SMART Architecture
26
Version 1
Version 2
Version 3
HP Universe 2010
HP Universe 2009
HP Universe 2008
“We are the Center of Excellence.”
Our Evolutionary Journey
The SMART Architecture
27
1
Global Utility Function
environment.qfl
Test Data
data.xls
Test Design
design.xls
Test Results
report.html
Object Repository
project.tsr
Control Script
script.mts
34
Utility Component
script.mts
1
Documentation
documentation.doc
“We are the Center of Excellence.”
The Engine - Process
QC
The SMART Architecture
28
SMART QTP Features
Keyword/Data Driven
QC Integration
Execution
Exception Handler
Custom HTML Results Reports
Programmatic Defect Reporting to QC
“We are the Center of Excellence.”
The SMART Architecture
2929
“We are the Center of Excellence.”
Features of the SMART Architecture:
Keyword/Data Driven Reusability
Easy to learn
Enhanced performance
The SMART Architecture
3030
“We are the Center of Excellence.”
Features of the SMART Architecture:
QC Integration Requirements
Test Plan
Test Lab
Defect Tracking
The SMART Architecture
3131
“We are the Center of Excellence.”
Features of the SMART Architecture:
Execution Local/shared drive or from QC
Single or multiple hosts
Test execution scheduling
The SMART Architecture
3232
“We are the Center of Excellence.”
Features of the SMART Architecture:
Exception Handlers
Recovery scenarios
Preconditions
Exit iteration
Defect reporting
The SMART Architecture
3333
“We are the Center of Excellence.”
Features of the SMART Architecture:
Custom HTML results/reports User friendly
Attaches report in QC
Attaches error screen shot with date and time stamp in QC
The SMART Architecture
3434
“We are the Center of Excellence.”
Features of the SMART Architecture:
Test Results Component Custom HTML reports
The SMART Architecture
3535
“We are the Center of Excellence.”
Features of the SMART Architecture:
Programmatic Defect Reporting to QC Keyword to set predetermined conditions
The SMART Architecture
3636
Submitting SQLConnecting to the
Database123456789
Please enter your personal information.
First Name:
Last Name:
City:
State:
John
Doe
Baltimore
Maryland
Welcome to abc World
User ID:
Password:
ssmith
**********
http://www.myhomepage.com
Database Test Execution from QC
The SMART Architecture
37
“We are the Center of Excellence.”
SMART QTP Utilities: Description
Business Case
Automation conversion of manual process
Accuracy
Novice users
Best ROI
Global Data Table
The SMART Architecture
38
“We are the Center of Excellence.”
SMART QTP Utilities:
Data Table Creation Utility Description
Generates the data table templates
Global Data Table
The SMART Architecture
39
“We are the Center of Excellence.”
SMART QTP Utilities:
File Handler Utility Description
Attaches multiple files to the Test Plan module of QC at same time
Global Data Table
The SMART Architecture
40
“We are the Center of Excellence.”
SMART QTP Utilities:
File Comparison Utility Description
Compares files such as Doc, Txt, Excel, HTML, JPG and BMP and report the differences
Global Data Table
The SMART Architecture
41
“We are the Center of Excellence.”
SMART QTP Utilities:
Import Utility Description
Imports test cases from QC to an excel file
Global Data Table
The SMART Architecture
42
“We are the Center of Excellence.”
SMART QTP Utilities: Automated Requirements Mapper (ARM) Utility
Description
Loads the requirements into QC
Maps requirements to test cases in QC
Global Data Table
The SMART Architecture
43
“We are the Center of Excellence.”
SMART QTP Utilities:
PATHtoSMART (PATS) Utility Description
Converts previous automation architecturetest scenarios and data tables for the SMART architecture
Global Data Table
The SMART Architecture
44
“We are the Center of Excellence.”
SMART QTP Utilities: Automated Manual Test Scripts (AutoMan)
Description
Generates test scripts from the Requirements Traceability Matrix (RTM) following parent child relationship
Global Data Table
The SMART Architecture
45
“We are the Center of Excellence.”
SMART QTP Utilities:
Dashboard Description
Generates consolidated reports across multiple QC projects
Global Data Table
The SMART Architecture
46
SMART QTP Utilities - Dashboard
“We are the Center of Excellence.”
The SMART Architecture
47
“We are the Center of Excellence.”
SMART QTP Utilities:
Get Requirement ID (GetRID) Utility Description
Generates a unique Requirement ID from the RTM
Global Data Table
The SMART Architecture
48
“We are the Center of Excellence.”
SMART QTP Utilities:
Database (DB) Utility Description
Extracts data from a DB2 or ORACLE data bases and fetch the output in an Microsoft Excel spread sheet.
Automated Cleanup and Traceability of Test scripts (AcTT) Utility Description
Gathers and reports the test scripts in QC that are not covered by requirements
The SMART Architecture – The Engine
49
“We are the Center of Excellence.”
I-PLC
How is SSA Achieving the SMART TCoE?
50
ARM
PATH
NEW
Releases
Subsequent
Releases
Quality Center
Load
Requirements
Create
Test
Plan
Execute
Test
Create
ReportDefects?
Yes
WinRunner
No
Automated Tests
Select
Automated
Tests
For Regression
Execute
Test
Integration
Changes
Capture Test Plan
DVTT Tools
IVEN, VDB,
MVASS
Data
Tables
Share test
scripts
Conduct Post-Validation Activities
GUI-LessSMART
QTP
LoadRunner
Baselined Requirements
Stable Test Environment
Accurate Release Notes
Stable Object Names
“We are the Center of Excellence.”
File
Handler ImportDashboard AcTT
File
Compare
GetRID PATSAutoManDB
Conduct Pre-Validation Activities Conduct Validation Activities
80% of HP Customers~1% of HP Customers6% of HP Customers
14% of HP Customers
SMART QTP Utilities SMART QTP Architecture
“We are the Center of Excellence.”
How is SSA Achieving the SMART TCoE?
51
TCoE + SMART Architecture
QTP
Service Test
LoadRunner
QC
WebInspect
Requirements CenterTCoE
= SMART TCoE
Next Steps
Next Steps:
QTP 10
QC 10
VISTA
More Utilities
52
“We are the Center of Excellence.”
“We are the Center of Excellence.”
Questions /Comments
54 ©2010 Hewlett-Packard Development Company, L.P.
To learn more on this topic, and to connect with your peers after
the conference, visit the HP Software Solutions Community:
www.hp.com/go/swcommunity
55
