Test Strategy

Program Test Strategy Revision History

Version Date Prepared by Comments

Master Test Plan Approval Signatures

Deliverable Name: Test StrategyVersion Number: 0.8Clarity ID: TBDClarity Project Name: TBD

Title Approver Name Approval (email back with “Approved”)

Date

Table of Contents

1. INTRODUCTION............................................................................................................................................51.1. PURPOSE...................................................................................................................................................... 52. SCOPE........................................................................................................................................................... 62.1. SCOPE ITEMS...............................................................................................................................................62.2. TYPES OF TESTING.....................................................................................................................................63. TEST APPROACH.........................................................................................................................................83.1. TESTING PHASES AND APPROACH..........................................................................................................83.1.1. UNIT TESTING..............................................................................................................................................93.1.2. FUNCTIONAL TESTING.............................................................................................................................103.1.3. SYSTEM INTEGRATION TESTING............................................................................................................103.1.4. REGRESSION TESTING.............................................................................................................................103.1.5. END TO END TESTING...............................................................................................................................103.1.6. USABILITY TESTING..................................................................................................................................113.1.7. OS/BROWSER COMPATIBILITY TESTING...............................................................................................113.1.8. ACCESSIBILITY TESTING.........................................................................................................................123.1.9. DATA MIGRATION TESTING.....................................................................................................................123.1.10. PERFORMANCE TESTING.........................................................................................................................123.1.11. SECURITY TESTING...................................................................................................................................133.1.12. DISASTER RECOVERY / MDHA TESTING................................................................................................133.1.13. CONTENT VALIDATION TESTING............................................................................................................133.1.14. USER ACCEPTANCE TESTING.................................................................................................................133.1.15. CUTOVER TESTING...................................................................................................................................143.1.16. USER ASSESSMENT..................................................................................................................................143.2. TESTING TEAM MODEL IN ITERATIVE DEVELOPMENT........................................................................144. ENTRY AND EXIT CRITERIA.....................................................................................................................164.1. ENTRY AND EXIT CRITERIA FOR STORY TESTING DURING ITERATION:..........................................164.2. ENTRY AND EXIT CRITERIA FOR UAT:...................................................................................................164.3. TEST SUSPENSION CRITERIA..................................................................................................................174.4. TEST RESUMPTION REQUIREMENTS.....................................................................................................175. TESTING ORGANIZATION.........................................................................................................................18

5.1. PROGRAM TEST MANAGEMENT.............................................................................................................185.2. ROLES AND RESPONSIBILITIES..............................................................................................................195.3. PROGRAM TEAMS ROLES AND RESPONSIBILITIES............................................................................226. TRAINING.................................................................................................................................................... 236.1. ON-BOARDING SAPIENT INDIA................................................................................................................236.2. KNOWLEDGE TRANSFER TO PROJECT INDIA......................................................................................236.3. KNOWLEDGE TRANSFER PROCESS TO PROJECT INDIA....................................................................247. TEST ENVIRONMENTS..............................................................................................................................258. TEST TOOLS...............................................................................................................................................278.1. HP QUALITY CENTER................................................................................................................................278.2. HP QUICK TEST PROFESSIONAL............................................................................................................278.3. HP LOADRUNNER......................................................................................................................................278.4. OTHER TOOLS...........................................................................................................................................279. TESTING PROCESSES..............................................................................................................................289.1. DEFECT MANAGEMENT............................................................................................................................289.2. TEST CASE WRITING APPROACH...........................................................................................................299.3. TEST CASE MANAGEMENT......................................................................................................................299.4. TRACEABILITY...........................................................................................................................................2910. COMMUNICATION AND ESCALATION.....................................................................................................3110.1. MEETINGS SCHEDULE..............................................................................................................................3110.2. PURPOSE OF THE MEETINGS..................................................................................................................3211. TEST DELIVERABLES AND REPORTING................................................................................................3311.1. DELIVERABLE LIST...................................................................................................................................3311.2. REPORTING METRICS...............................................................................................................................3312. TEST DATA MANAGEMENT......................................................................................................................3513. TEST AUTOMATION APPROACH.............................................................................................................3614. RISKS ASSUMPTIONS AND DEPENDENCIES.........................................................................................3714.1. RISKS.......................................................................................................................................................... 3714.2. ASSUMPTIONS...........................................................................................................................................3714.3. DEPENDENCIES.........................................................................................................................................37

1. Introduction

1.1. Purpose

The following points are outlined in this document:

Scope of the test Definition of the testing strategy at a high-level Test Control Processes High-level Timeline Test Management Reporting Testing Roles and Responsibilities Testing Tools Test Environments Test Data

2. ScopeThis section covers the Scope of testing and Types of tests that will be done through out the life cycle of the Program.

2.1. Scope Items

2.2. Types of Testing Following is the list of tests that will be done across the lifecycle of the program. The overall approach for these and where these fit in will be covered in the following section of the document.

Unit Testing

Functional Testing

System Integration Testing

Regression Testing

End to End Testing

Usability Testing

OS/Browser Compatibility Testing

Accessibility Testing

Data Migration Testing

Performance Testing

Security Testing

Disaster Recovery Testing

Content Validation Testing

User Acceptance Testing

Cutover Testing

Users Assessment

3. Test ApproachThis section covers the various phases and describes the testing approach for different types of testing across the program.

3.1. Testing Phases and approach

The approach we are following is an early phased approach to testing where the testing activity is embedded within the Development story and starts in parallel with the development effort. No story is deemed complete until the exit criteria for the story as defined in Entry and Exit criteria section below has been met.will be integrated as part of End to End Test stories and testing will be focused on complete business processes.

3.1.1.Unit TestingThe purpose of Unit Testing is to validate that each distinct module of the application is implemented as per the technical and functional specification of the story at the unit level.

Unit tests will be automated where possible using frameworks like JUnit and will be executed throughout the development process across all iterations.

Development teams in the functional tracks will conduct Unit tests as part of Development process and will publish the unit test results as part of the build release to QA.

3.1.2.Functional TestingThe purpose of Functional Testing is to validate that the functionality implemented as part of the story meets the requirements.

3.1.3.System Integration TestingThe key purpose of the System Integration testing is to verify that information flow between different components and systems is in the right format and contains correct data. The testing will include both positive

3.1.4.Regression TestingThe purpose of regression testing is to validate that the code / configuration changes have not broken functionality that was working earlier.

3.1.5.End to End TestingThe purpose of E2E testing is to validate that the entire application, including interfaces, works correctly as a whole when linked together. This will include

3.1.6.Usability TestingUsability Testing is a qualitative methodology used to assess a solution with Project end-users. The purpose is to assess the Usefulness and Usability of the solution - that is how well the solution enables users to accomplish their goals and meet their needs and how easy it is to use.

The Information Architect supports the overall track by serving as dedicated note-taker for the sessions, generating ideas for future state recommendations, and depending on the project, may generate wireframes depicting the recommendations. IA would be allocated for execution and to shape protocol and provide input into analysis and recommendations.

3.1.7.OS/Browser Compatibility Testing

There will be separate stories created for OS/Browser testing and the test team within the Functional tracks will perform these tests for the web pages that are developed over various iterations.

3.1.8.Accessibility TestingPurpose: The purpose is to validate that the web pages conform to W3C AAA Web Content Accessibility will perform these tests for the web pages that are developed over various iterations.

3.1.9.Data Migration TestingThe purpose of Data Migration testing is to validate that all the entities (e.g. Customer records, Products etc) have been migrated from source data into Project data and all the attributes for each of the entities are loaded

correctly.

3.1.10. Performance TestingPerformance tests will be carried out to validate if the Security Testing The purpose of Security test is to validate the application and infrastructure vulnerability.

3.1.11. Disaster Recovery / MDHA Testing Disaster recovery involves testing how the system recovers from disasters, crashes or any catastrophic failures. As part of Disaster Recovery Test, we will bring down one of the environments (TBD) and we will then try and setup an environment that is replica of TBD and perform sanity tests to confirm that everything is working fine functionally and operationally in the new environmentDetailed approach for DR Testing and the ownership is TBD at this stage.

3.1.12. Content Validation Testing

3.1.13. User Acceptance TestingThe purpose of UAT is to allow the business users to validate that the application performs end to end business transactions and meets the criteria outlined within the FSD’s.

3.1.14. Cutover TestingCutover Testing involves conducting a dry run of the cutover process with the intent of validating that the

3.1.15. User AssessmentThis test will be performed as part of training that will be provided to business users in order to get them familiarized with the website and the associated functions. Details will be added as these are finalized.

Organization change vertical will be responsible for doing User Assessment tests.

3.2. Testing Team Model in Iterative Development

4. Entry and Exit CriteriaThis section provides the entry and exit criteria for each of the Iterations and the overall testing

4.1. Entry and Exit Criteria for Story Testing during Iteration:

4.2. Entry and Exit Criteria for UAT:

4.3. Test Suspension Criteria

• High number of Show Stopper or severe defects• Show Stopper or severe defects that affect the ability to effectively run other Test Cases• Number of unresolved Sev 0, Sev 1 and Sev 2 are impacting the Testing progress

significantly• Environmental instability

4.4. Test Resumption Requirements

• Show Stopper and severe defects resolved. The fix verified in development before pushing to the test environment

• Mitigation plan is in place to address large number of unresolved defects• Environmental issues are resolved

5. Testing Organization

5.1. Program Test Management

5.2. Roles and Responsibilities

The anticipated roles for QA team have been identified in the illustration and summarized in the roles and responsibilities table.

5.3. Program Teams Roles and Responsibilities

Team Accountable for Test Phase

Team Responsible for Test Script Creation

Team Responsible for Test Script Execution

Team Responsible for Test Environment Set-up

Unit Testing Development Teams Development Teams

Development Teams

Infrastructure Team

Functional Testing PMO Test Mgmt Team

Functional tracks Functional tracks Infrastructure Team

System Integration Testing

PMO Test Mgmt Team

Functional tracks Functional tracks Infrastructure Team

6. Training

Knowledge Transfer

.

7.

8.

9. Test EnvironmentsThis section covers the list of environments and their usage. Following pictures reflect the list of environments and the code propagation flow:

10. Test ToolsThis section covers the list of tools we will use during testing process.

10.1. HP Quality Center QC customizations, process and standards once these are closed.

10.2. HP Quick Test ProfessionalHP QTP will be used for automated regression testing. Please refer to section on Test Automation framework for QTP usage.

10.3. HP LoadRunnerHP LoadRunner will be used running the performance tests.

10.4. Other tools

11. Testing Processes This section provides details on different testing processes that will be followed by the testing team.

11.1. Defect Management

The defects logged during the testing process will be discussed in Defects triage meetings. A daily Defect triage meeting will be held for each of the Functional tracks. The participants will include Functional Track Lead, Testing Lead, Architect and optionally Tester and Developer.

During these meetings the group will review the new defects logged and the status of outstanding showstopper defects.

Here are the Severity definitions for defects that will be followed on the program

Severity Description0 - Show Stopper There is no workaround and the project will be halted from making further

progress until the defect is resolved1 – Severe The defect prevents further use of this work product until resolved2 – Major The defect will prevent further use of a portion of this work product until it is

resolved3 – Significant The defect should be addressed, but work on future work products may proceed

according to plan4 – Minor The defect has little or no impact on current or future work products

11.2. Test Case Writing ApproachThis section covers the approach we will take for writing test cases. Following are steps that test team will follow:

Requirements Understanding:

• Test Case Creation:

• Traceability: have

• Test Case Reviews: roved by Project business users – that will be one of the entry criteria for End to End test execution.

11.3. Test Case Management

11.4. Traceability Following picture reflects how Traceability will work between Requirements, Test Cases and Test Execution.

:

Vertical -> Functional Track -> Functionality -> Use Case -> Requirements

In the Test Plan module we will break down Functionality into Use Case (s). Each of the Dev stories consisting of requirements will be listed here. All the associated test cases for the Dev story will be listed within the Dev Story folder. These Test Cases will be mapped to Requirements and Use Cases in requirements module. The folder structure in Test Plan module of QC will be as follows:

Vertical -> Functional Track -> Story -> Test Case

In the Test Lab module we will create a folder for the ADR Release no. which will have sub-folders for Iterations. These Iterations will in turn have Test set representing the Dev Story. All the test cases for the scope delivered will be added to these test sets. Folder structure will be as follows:

ADR -> Iteration -> Functional Track -> Story -> Test Case

12. Communication and Escalation

12.1. Meetings ScheduleFollowing picture represents how communication will work across geographies

12.2. Purpose of the Meetings  Reporting

13. Test Deliverables and Reporting

13.1. Deliverable List

Deliverables DescriptionTest Strategy A program high level document will include

Testing phases Metrics R&R Traceability strategy

Test Plan A test plan will be created Scope of testing

Testing Schedule & Logistics Testing Resources Allocation Types of tests planned (including the following)

Data Conversion Test Plan Security Testing Test Plan Performance Test Plan End-To-End Test Plan

Test Scripts A detailed document for each test case including: Description of the test scenario and the purpose of the test. Steps involved in executing the test with expected results. Any special set-up requirements (infrastructure, data, etc.)

Test scripts will be created for testing various stories in QCTraceability Matrix

This contains Mapping of requirements/Business Transactions to test cases to confirm

that testing validates the requirements. This will be stored in QC

Test Results Summary results of the testing phase: Test results documentation as appropriate. List of test scripts executed with their execution status. List of defects recorded and their status These will be recorded in QC

Testing Control Processes

Processes developed establish consistency of testing activities across the project teams

Test Planning covering documentation of test scripts. Test Execution covering documentation of test results. Defect Management Process. Management of testing activities and report progress

Test Training Materials

Training materials used for reviewing the testing strategy, testing control processes and usage of testing tools.

13.2. Reporting Metrics

The metrics that are reported as part of the testing program are intended to provide an accurate view of the testing status and progress. Specifically, the following items will be measured and reported for each team for each testing phase:

Test Scripts created compared with the expected number to be created (tracked by the Functional

Teams or other teams required to create the Test Scripts) Test Scripts executed compared to the total number of test scripts expected to be executed (by

Iteration and team) Number of defects (by severity, track, and iteration) Number of open defects (by severity, team and iteration) The total defect count by severity (by week by iteration and track) Defect severity by cause Defect Aging (by week by iteration and track)

14. Test Data Management

This section describes the approach we will take for test data creation and management.

15. Test Automation Approach

16. Risks Assumptions and Dependencies

16.1. Risks

16.2. Assumptions

16.3. Dependencies