Reshaping the Testing Pyramid

1

Reshaping The Testing Pyramid

Kapil Sethikapil.sethi9@gmail.com

2

About Me…

• Quality Analyst

• Automation enthusiast

• Worked with

3

Product Background

• Desktop application• Offline, Online and Network modes of operation

• Has multiple versions in production

• Multiple teams committing to the same repository

4

As a team

5

As a team

6

As a team

7

Observations• Road blocks

• 2 different UI test automation approaches (White and TestComplete)

• Loads of UI automation tests

• Some of the UI automation tests were not part of the build pipelines

• Flaky builds

• 2 weeks of manual regression testing before each release

8

Road Blocks• In Detail

• Number of UI automation tests: ~450

• Running on 2 different modes: ~900 UI automation tests

• Parallel execution using 8 VMS

• Average time taken to execute a test suite: ~35-40 mins

• More than ~400 TestComplete UI automation tests

9

Observations

10

Goal

11

The Headache

12

Experiment• Analysis/Flow

• Analyse existing UI tests

• Check the integration tests coverage

• Identify integration tests candidate and approach

• Share the outcome with the team and finalize the approach

13

Guinea Pig for the Experiment• One of the most flaky UI automation test suite:

Reports

• Number of UI tests: 93

• Time taken to complete execution: ~65 mins

• Going to add 11 new features in Reports

14

Result of the Experiment• Added 5 integration tests in BDD format

15

Outcome of the Experiment• Added 145 integration tests in BDD format

• Execution time: ~ 3 mins

• Deleted 50 UI automation tests

• Execution time for UI automation time: ~25 mins

16

Going Forward…• More collaborative effort

• Review unit tests coverage

• Add BDD format integration tests

• Add UI automation tests only if needed

17

Going Forward…• New Feature: Reports

• Reviewed unit tests coverage

• Added BDD format integration tests: 72 (~2 mins)

• Added UI automation tests: 11 (~4 mins)

• No manual regression testing needed

18

Key Learning• How to conduct an experiment

• Placing the right tests at right level

• Team Buy-In

• Balancing the existing work with rewrite of tests/improvement approach

19

Questions