Adaptive Testing Adaptive Testing TechniqueTechnique

Ajay Chhokra and Jitender Ajay Chhokra and Jitender DanwaniDanwani

Adaptive Testing Adaptive Testing TechniqueTechnique

Ajay Chhokra and Jitender Ajay Chhokra and Jitender DanwaniDanwani

www.Test2008.in

2

Trends Business complexities and globalization of business are

impacting

Types of initiatives supported by software delivery

Process of software delivery

Software Testing Today is different than what it was 10 years ago and will be different 10 years from now……..

Facts contributing in Trends are

TechnologySOA, Distributed

Technology, Web

TechnologySOA, Distributed

Technology, Web

MethodologyRapid Development,

Agile

MethodologyRapid Development,

Agile

Customer ExpectationCost, Time, Reliability and Robustness

Customer ExpectationCost, Time, Reliability and Robustness

www.Test2008.in

3

Business Movers and Shakers

Drivers Expectations Effects onTime to Market

Reduce Cycle Time •Revenue•Competitive Edge

Quality Defect Density, Reliability and High Test Coverage

•Customer Satisfaction•Market Share•Revenue

Cost Maximum and Quick ROI

•Pressure on reducing Testing Effort•Quality

Employee Smart Testing •Employee Morale

www.Test2008.in

4

Key Message

- Reduce Cycle Time

- Smart Testing

Test Reduction Techniques

www.Test2008.in

5

Test Reduction – Analysis

Testing effort is 30 – 40% of the complete project development Life cycleTesting effort is 30 – 40% of the complete project development Life cycle

Functional testing is most widely used testing type. Functional testing involves applying various user inputs.

Functional testing is most widely used testing type. Functional testing involves applying various user inputs.

Number of possible Test inputs is very large for complex projects. Testers select limited set of test cases

Number of possible Test inputs is very large for complex projects. Testers select limited set of test cases

Most of functional testing is done in an intuitive mannerMost of functional testing is done in an intuitive manner

Test Reduction is must for regression tests to increase productivityTest Reduction is must for regression tests to increase productivity

www.Test2008.in

6

Test Reduction Techniques

Risk based Testing Pair-wise Testing Cycle Based Testing

www.Test2008.in

7

Risk based Testing

Positives Negatives Focus on the key business areas/critical business processes

Eliminates the major risks

Quick to apply if the tester and business have good exposure to business domain

Identifying the business risks precisely is difficult (Requires clear understanding of applications and business and

detailed knowledge of requirements)

Technique focuses on the key risks and often ignores the low impact areas.

Technique is not foolproof; it is more or less intuitive in nature.

www.Test2008.in

8

Pair-wise Testing

Positives Negatives Reduce the test cases drastically at 2 and 3-level interactions

Helpful for complex systems that involve large number of parameters and their combination

Suitable for regression cases where coverage is to be maintained high with minimal cases count

The defect detection rate versus the applied effort is high.

Chance of undetected defects at lower level interaction is more than higher order interaction

With number of interaction levels increases, the number of cases increases very significantly

Pairwise testing fails when the parameters and levels have not been appropriately identified.

Pairwise testing (primarily focusing on 2 –way implementation) does not cover all input space.

www.Test2008.in

9

Cycle Based Testing

Positives Negatives

Gives good visibility on Testing progress

Helps to define logical milestone for testing. The defect fixes or patches applied are tested in subsequent cycles.

Very helpful when applying Incremental Integration techniques

Technique is discouraged when individual test cycles are too long.

May involve overheads if repeated data setup is required in every subsequent cycle.

Too many cycles in single release delays the process.

Tracking the execution, defects and cross-impact is sometimes difficult when using this technique.

www.Test2008.in

10

Adaptive Testing

A Technique developed to “adapt” to various project needs. Well suited for specific project needs – reducing time or increasing

test coverage etc. Adaptive techniques can be combination of various market

proven techniques such as Risk Based testing, Pairwise testing etc

tailored per project needs.

What is Adaptive Testing ?

www.Test2008.in

11

Why Adaptive Testing ?

1. Cycle based testing fails and proves ‘Overhead’ when too many cycles involved, each requiring fresh data or environmental setups. - Adaptive testing helps control the need of multiple cycles

2. Pairwise testing – It’s a popular test reduction technique but is effective at low level interactions.

- Adaptive technique can help focus on multi-level interaction thereby reducing risk of low coverage

3. Risk based testing often fails to provide complete coverage in large projects.

- Adaptive techniques can ensure that maximum risks are covered giving high coverage

4. Adaptive testing draws a fair balance between the test coverage, the risk and the time to market the quality product.

www.Test2008.in

12

How to Apply Adaptive Techniques ?Steps to Apply Adaptive Techniques

Step 1: Identify the most significant parameters (one’s that have prime impact on business) The various parameters could be

complexity of application defined quality goals business priority and impact time to market delivery timelines system knowledge

Step 2: Analyze the individual techniques that can best fulfill the business needs.

Step 3: Tailor the individual techniques or apply Mix-n-Match to make it adapt to project needs

Step 4: Analyze the results and continue tailoring, if needed

Note : Once you are familiar with how to apply Adaptive techniques, the effort required for tailoring is drastically reduced.

www.Test2008.in

13

How to Apply Adaptive Techniques?

The Mix-n-Match of the individual techniques can help project have different FLAVORS of adaptive testing. Each Case below is project specific and business objective based.

Case 1: Risk Based + Pairwise Test• Define the parameters and levels of interaction • Identify the high impact condition using RBT• Apply Pairwise technique on less/ low impacting parameters

Case 2: Risk Based + Cycle Based Test• Identify the high impact and low impact business condition• Define multiple cycles • Distribute the high impact and low impact functionalities over cycles based on priorities

Case 3: Pairwise Test + Cycle Based Test• Identify parameters and levels • Apply Pairwise technique to arrive at reduced test set• Prioritize the test cases• Distribute the test cases in various cycles – sanity, priority and retest

www.Test2008.in

14

Benefits and Challenges – Adaptive Techniques

BenefitsBenefits

Can be applied to varied project scenarios or type of testing – regression, enhancement, system, integration testing etc

Adaptive techniques are fool proof and help to identify hidden defects

Focus is on high coverage and critical business functions simultaneously

Can be universally applied in product based releases cycles

High probability of defect detection.

ChallengesChallenges

Requires good knowledge of system & market techniques (Pairwise, combinatorial, risk based techniques etc)

Need analytical capabilities in determining how to Mix-n-Match or tailor the techniques

Change sin business process necessitate revision of test cases and re-assess risk associated

www.Test2008.in

15

Case Study

Risk-Risk-Based Based

TechniquTechniquee

Pairwise Pairwise TechniquTechniqu

ee

Adaptive Adaptive TechniquTechniqu

ee

Reduced Test Reduced Test CasesCases

32 16 32

Defects FoundDefects Found 8 5 9

Test Reduction Test Reduction (%)(%)

97.2% 98.6% 97.2%

Defects Comparisons

9

45

8

0

2

4

6

8

10

Release

Defe

ct

Co

un

t

Series1

Series1 9 5 8 4

Adaptive Pairw ise Risk Based Ad-Hoc

Exhaustive Test Case Count = 1152

UHG India Experience – Case Study