Embed Size (px)
DESCRIPTION
As organizations mature in their testing processes, the perennial quest to achieve ultimate excellence has led many to establish a Testing Center of Excellence (TCoE). Many of these initiatives have been plagued with issues ranging from partial implementation to complete abandonment midway. Additionally, most TCoE initiatives meet heavy resistance and inertia from testing teams who perceive them as a threat to their independence and way of working. At the heart of these issues lie misalignment with business goals, faulty ROI analysis prior to investing, poor communication, and an incorrect organizational model. Drawing from their experience consulting with organizations on TCoE initiatives, Krishna Iyer and Mukesh Mulchandani share insights, experiences, and lessons learned from both their successes and failures. Learn how to go about creating your own TCoE while overcoming the common—and not so common—challenges you will face along the way. Draw on their experience to troubleshoot some of your unique problems.
Citation preview
Session
Presented by:
Krishna Iyer & Mu ni
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073 888‐2
W13 Concurrent4/9/2014 3:15 PM
“Building a Testing Center of Excellence: Experiences, Insights and Failures”
kesh MulchandaZenTest Labs
300,68‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Krishna Iyer Zen Test Labs
With more than fifteen years of experience in QA, Krishna Iyer is the CEO of Zen Test Labs, a niche software quality assurance firm focused on revolutionizing the way we test software. A prolific speaker, Krishna has delivered multiple presentations, keynotes, and tutorials at global conferences including STARWEST and STAREAST. Krishna balances his time between driving financial growth for Zen Test Labs and devoting himself to social causes.
Mukesh Mulchandani Zen Test Labs
CTO of Zen Test Labs Mukesh Mulchandani is focused on dramatically reducing redundant manual tasks in the software testing cycle. With fourteen years of experience in developing solutions for complex testing challenges, Mukesh is the principal architect behind all of Zen Test Labs’ testing solutions. A frequent presenter at global conferences including the STAR conferences, he spends his time developing cutting edge solutions for Zen Test Labs. Mukesh is passionate about developing the skills of youth at large and, to this end, spends his spare time leading non-profit training initiatives.
2/4/2014
1
BUILDING A TEST CENTRE OF EXCELLENCE: EXPERIENCES, INSIGHTS AND FAILURES
PRESENTED BY:KRISHNA IYER AND MUKESH MULCHANDANI
ABOUTKRISHNAANDMUKESH
Krishna is the CEO of Zen Test Labs‐ a niche software quality assurance firm focussed on revolutionizing the way we test software. Krishna comes with over 15 years of experience in QA and is a prolific speaker He has delivered multiple track presentations keynotes and tutorials at global
KRISHNA IYER
is a prolific speaker. He has delivered multiple track presentations, keynotes and tutorials at global conferences including STARWEST and STAREAST. Krishna balances his time between driving financial growth for Zen Test Labs and devoting himself to social causes.
MUKESH MULCHANDANI
Mukesh is the CTO of Zen Test Labs‐ a niche software quality assurance firm focussed on dramatically reducing redundant manual tasks in the software testing cycle. Mukesh brings with him 14 years of
i i d l i l ti f l t ti h ll d i th i i l hit t
2
experience in developing solutions for complex testing challenges and is the principal architect behind all of Zen Test Labs’ testing solutions. Mukesh has also presented regularly at global conferences including the STAR conferences. Mukesh spends his time developing cutting edge solutions for Zen Test Labs and is passionate about developing the skills of the youth at large. To this end, he spends his spare time leading non‐profit training initiatives.
2/4/2014
2
AGENDA
TEST CENTRE OF EXCELLENCE (TCOE)
INSIGHTS ON SETTING UP A TCOE
BASELINE AND BENCHMARK
IMPLEMENTING REUSABILITY IN TESTING
SHAPING SKILLSETS, MINDSETS AND TOOLSETS
BUILDING QUALITY INTO SOFTWARE
ROI BASED AUTOMATION
INTEGRATING MULTIPLE TOOLS AND METHODOLOGIES UNDER A SINGLE FRAMEWORK
RECRUITING AND RETAINING GOOD TESTERS
CONCLUSION
3
TESTCENTREOFEXCELLENCE
CENTRALIZEDKNOWLEDGEMANAGEMENT
STANDARDIZEDTEST
PROCESSES
EXCELLENT
CENTRALIZEDRESOURCE
MANAGEMENT
MANAGEMENT
TOOLS ANDTECHNOLOGY
TEST CENTRE OFEXCELLENCE
EXCELLENTQUALITY
IMPROVEDTIME TOMARKET
4
REUSABLEARTIFACTS
TECHNOLOGY
LOWER COSTS
INFRASTRUCTURE
2/4/2014
3
INSIGHTS ONSETTINGUPATCOE
1. BASELINEAND
BENCHMARK
2. REUSABILITY7. RECRUITING& R
IN TESTING
3. SKILLSET, MINDSET ANDT
6.INTEGRATINGUNDER ONEF
& RETAININGTALENT
5
TOOLSET
4. BUILDINGQUALITY INTOSOFTWARE
5. ROI BASEDAUTOMATION
FRAMEWORK
1. BASELINEANDBENCHMARK
SIZING TESTING USING TEST POINTS: A SINGLE SIZE METRIC FOR ESTIMATION
TEST COVERAGE METRIC = NUMBER OFTEST CASES PER TEST POINT PER
MODULE
DEFECT DENSITY METRIC = NUMBER OFDEFECTS PER TEST POINT (FOR EACH
MODULE )
TEST PRODUCTIVITY METRIC = NUMBEROF TEST POINTS DELIVERED IN AN HOUR
(FOR EACH MODULE)
6
2/4/2014
4
1. BASELINEANDBENCHMARK
SIMPLE AVERAGE COMPLEX
SIZE ESTIMATION MATRIX
SMALL 1 TP 2.5 TP 3 TP
MEDIUM 4 TP 8 TP 6 TP
7
LARGE 9 TP 10 TP 11 TP
2. IMPLEMENTINGREUSABILITY INTESTING
REUSABLE BUSINESSFUNCTIONS
REUSABLE TESTDATA
AUTOMATICALLYGENERATED TEST CASES
•Create business functions that can be reused to build manual test cases.
•Consider the login module. Once a login business function has been created every
•Create reusable test data for building manual test cases.
• Significant time savings as reusable test data is stored separately and called using a test data id
•Automatically generate descriptive manual test cases (in English) once business function and test data are brought together in a test flow.
8
been created, every manual test case requiring it for that application can reuse the function.
using a test data id (primary key)
2/4/2014
5
3. SHAPINGSKILLSET, MINDSETANDTOOLSET
• EMPOWER TEAMS WITHSTANDARDIZED TRAINING
CREATIVETHINKING
THINKING SKILLSFOR TESTERS
• PROMOTE CONTINUOUSLEARNING AND IMPROVEMENT
• MAINTAIN A TRAINED BENCHTO ENABLE RAPID RAMP‐UPFOR PROJECTS
9
CRITICALTHINKING
COVERAGETHINKING
4. BUILDINGQUALITY INTOSOFTWARE
BUILDING QUALITY INTOSOFTWARE INSTEAD OF TESTING
QUALITY INTO SOFTWARE
10
DEFECT PREVENTION THROUGHDEFECT CAUSAL ANALYSIS
COLLABORATION WITHDEVELOPMENT TEAM
2/4/2014
6
5. ROI BASEDAUTOMATION
INITIATE ALL AUTOMATIONREQUESTS THROUGH TCOE
TCOE TEAM RECEIVES DEMO OFAPPLICATION FUNCTIONALITY, INTERVIEWS KEY STAKEHOLDERS
TCOE TEAM PERFORMS QUICKSMOKE TEST TO CHECK APPLICATION
STABILITY
TCOE TEAM ARRIVES AT A DECISIONTO APPROVE OR REJECTAUTOMATION REQUEST
11
5. ROI BASEDAUTOMATION
SPECIALIZED METRICS TO KEEP TRACK OF AUTOMATION PROJECT
TYPE FORMULA DESCRIPTION
AUTOMATION BY RISK OR
PRIORITY
WEIGHTED TEST CASES AUTOMATED BASED ON
RISK‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
TOTAL WEIGHT OF MANUAL TEST CASES BASED ON
RISK
UNDER THIS METHOD ONE NEEDS TO COMPUTE THE
RISK/PRIORITY OF EACH MANUAL TEST CASE, THENCOMPARE THE SUM OF RISK/PRIORITY OF ALL THE
TEST CASES AGAINST THE TOTAL RISK/PRIORITY OF
TEST CASES AUTOMATED.
AUTOMATION ROI BY TIME
TOTAL EXECUTION TIME FOR TEST CASES
AUTOMATED
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
TO COMPUTE USING THIS METHOD ONE NEEDS TO
CALCULATE TOTAL TIME OF RUNNING ALL THE
MANUAL TEST CASES IN AUTOMATED FASHION VS. THETOTAL MANUAL EXECUTION TIME FOR THESE TESTMANUAL EXECUTION TIME FOR THE TEST CASES
AUTOMATED
TOTAL MANUAL EXECUTION TIME FOR THESE TEST
CASES.
AUTOMATION ROI IN DOLLARSAUTOMATION BENEFITS ACCRUED
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐TOTAL COST OF AUTOMATION
THIS IS STANDARD AUTOMATION CALCULATION IN
DOLLARS OR ANY OTHER CURRENCY.
12
2/4/2014
7
6. INTEGRATINGMULTIPLETOOLS ANDMETHODOLOGIESUNDERASINGLEFRAMEWORK
TCOES CAN DEVELOP CUSTOM SOLUTIONS THAT INTEGRATE MULTIPLE TOOLS AND METHODOLOGIES
FRAMEWORKS CAN COMBINE THEADVANTAGES OF VARIOUS APPROACHES SUCHAS FUNCTIONAL DECOMPOSITION, KEYWORDDRIVEN AND DATA DRIVEN APPROACHES
SUCH FRAMEWORKS CAN BE HIGHLYMAINTAINABLE AS UI, TEST SCRIPTS AND TEST
DATA ARE REUSED
CAN BE A GREAT VALUE ADD FOR NON‐TECHNICAL AND BUSINESS USERS
REDUCED COST OF SCRIPT CREATION. ENABLELARGE SCALE AUTOMATION
13
7. RECRUITINGANDRETAININGGOODTESTERS
A TCOE CREATESCOMPELLING NEWOPPORTUNITIESFOR TESTING
PROFESSIONALS
A TCOE BRINGSTOGETHER THE
BEST TALENT AND C
14
INCREASED JOBSATISFACTION DUETO REMOVAL OFREPETITIVE TASKS
BEST TALENT ANDCREATES TESTSPECIALISTS
CHOOSETHINKING SKILLSOVER TECHNICAL
SKILLS, EDUCATIONALBACKGROUND
AND TOOL SKILLSWHILE HIRING
2/4/2014
8
CONCLUSION
EXCELLENTAND
CONSISTENTQUALITY
TCOE
LOWEREDCOSTS
CAREERALIGNMENT
IMPROVEDTIME TOMARKET
15
CAREERADVANCEMENTOPPORTUNITIES
CULTURE OFTESTING
BETWEENBUSINESS
AND TESTING
VALIDATE
AUTOMATEINNOVATE
Thank You!
Office No.2, 2nd Floor,Abja Pavilion, D.P. Road, Aundh, Pune ‐ 411007,Maharashtra, India.
Contact No.: (+91) ‐ 020 – 66202245Direct: (+91) ‐ 020 – 66202245Blog: www.zentestlabs.com/blog Website: www.zentestlabs.com
2/4/2014
9
ABOUTZENTEST LABSDomains
Banking & Financial Services Banking & other ISV’sMobile and eLearning
BANKING
Technology, Tools and Frameworks
Large scale Test AutomationIP based products‐
ZenFRAME, ZenWRITE, etc.Testing tools Mercury‐ HP,
Test Complete, Selenium (Illustrative only)E ternall a dited ISO
Geographic SpreadUSA operations based
out of Wilmington, DECustomer base that spans
US, EU, APAC and IndiaPartner net ork that
S|One Corporation
BANKING SOFTWARE
Externally audited ISO 27001 processesROI driven engagement
modelsTesting thought leadership‐
STAR conferences, etc.
Partner network that spans US, EU, MEA and APAC
17
OTHERS
