Deliver Superior Outcomes Using HBT Visualization Tool

Deliver superior outcomes using HBT Quality Visualisation ToolPowered by HBT

T AshokFounder & CEOSTAG Software Private LimitedArchitect - HBT

in.linkedin.com/in/AshokSTAG Ash_Thiru

© 2014 STAG Software Private Limited. All rights reserved.

Delivering superior outcomes requires:

2

good understanding, purposeful strategy, robust test design and rapid execution

Good work -

An excellent orchestration of

Good “mirror” - for a clear & objective feedback rapidly to aid continuous adjustment & adaptation.


The typical “mirror”

3

Requirements traceability, Coverage- Code/Test

#Cycles, TC planned vs executed, Blockers

TC - Pass/fail rate, First pass yieldDefect info- Distribution, Rate, Density, Leakage

!ese are good, but not good enough to be sharply goal focused.

Normally this is done by collecting & analysing typical “extrinsic measures” like:


Good “mirror” needs to tell ...

4

Are test assets good?1

Have we assessed completely?2

How good are the outcomes?3

quality of test cases

quality of execution

quality of product/application


Hypothesis Based Testing - HBT

5

System Under Test

Cleanliness Criteria

Potential Defect Types

Test CasesRequirementstraceability“what to test”

Faulttraceability“test for what”

should satisfy impeded by

© 2014 STAG Software Private Limited. All rights reserved. 6

Quality Levels & PDTs - Powered by HBT

Input cleanliness

Input interface cleanliness

Structural integrity

Behaviour correctness

Environment cleanliness

Attributes met

Flow correctness

Clean Deployment

End user value

L1

L2

L3

L4

L5

L6

L7

L8

L9

!at inputs are handled wellPDTs related to Input data correctness

!at the functional behaviour is correctPDTs related to Functionality

!at the internal structure is robustPDTs related to Structural aspects

!at the user interface is cleanPDTs related to UI

!at end-to-end "ows work correctlyPDTs related to Flow behaviour, Interactions

!at it does not mess up the environmentPDTs related to Resource leaks, Compatibility...

!at the stated a#ributes are metPDTs related to Performance, Load, Volume......

!at it deploys well in the real environmentPDTs related to Compatibility, Migration

!at user expectations are metPDTs related to User "ows, User experience


The W3T

7

QL9 End user value End user value testPDT 57-59End user value test

QL8 Clean deployment Installation test PDT 54-55

Migration testPDT 56Migration test

QL7 A"ributes met LSPS testPDT 44-51

Reliability testPDT 52Reliability testPDT 52

Security testPDT 53

QL6 Environment cleanliness Good citizen testPDT 39-41

Compatibility testPDT 42-43Compatibility testPDT 42-43

QL5 Flow correctness Interaction testPDT 35-38

QL4 Behaviour correctness Functionality testPDT 24-31Functionality test Access control test

PDT 32-34Access control testPDT 32-34

QL3 Structural integrity Structural testPDT 14-23

QL2 Input interface cleanliness API validation testPDT 5-7API validation test GUI validation test

PDT 8-13GUI validation test

QL1 Input cleanliness Input validation testPDT 1-4Input validation test

QL - Quality LevelPDT - Potential Defect Type

Pow

ered

by

HB

T


L1 L2L2 L9

T1 T2 T3 Tn

PDT1-2 PDT3-6 PDT7-10 PDT11-14

CC1 CC2 CC3...

CCn

E1 ✓ ✓...

E2 ✓ ✓ ✓

E3 ✓ ✓

E4 ✓ ✓Wha

t-to

-Tes

t (W

2T)

HBT Baseline - The KEY to Visualisation!

Test-for-What(Expectation)

CC Cleanliness Criteria TfW-E Test-for-What(Expectation)

PDT Potential Defect Type TfW-G Test-for-What(Goal)

T* Test Type WhT What-Test

L* Quality Level WhnT When-Test

List the entities under test (E)(Features, Requirements...) in the order of priority/importance.

Map them to the expectations of the customers.


L1 L2L2 L9

T1 T2 T3 Tn

PDT1-2 PDT3-6 PDT7-10 PDT11-n

CC1 CC2 CC3...

CCn

E1 ✓ ✓...

E2 ✓ ✓ ✓

E3 ✓ ✓

E4 ✓ ✓Wha

t-to

-Tes

t (W

2T)


Test-for-What(Goal)

TfW-E






L1 L2L2 L9

T1 T2 T3 Tn


CC1 CC2 CC3...

CCn

E1 ✓ ✓...

E2 ✓ ✓ ✓

E3 ✓ ✓

E4 ✓ ✓Wha

t-to

-Tes

t (W

2T)


What-Test

TfW-G

TfW-E






L1 L2L2 L9

T1 T2 T3 Tn


CC1 CC2 CC3...

CCn

E1 ✓ ✓...

E2 ✓ ✓ ✓

E3 ✓ ✓

E4 ✓ ✓Wha

t-to

-Tes

t (W

2T)


When-Test

TfW-G

TfW-E

WhT






L1 L2L2 L9

T1 T2 T3 Tn


CC1 CC2 CC3...

CCn

E1 ✓ ✓...

E2 ✓ ✓ ✓

E3 ✓ ✓

E4 ✓ ✓Wha

t-to

-Tes

t (W

2T)


When-What & Test-for-What (W3T)

TfW-G

TfW-E

WhT

WhnT






L1L1 L2L2 L9L9

T1T1 T2T2 T3T3 TnTn

PDT1-2PDT1-2 PDT3-6PDT3-6 PDT7-10PDT7-10 PDT11-14PDT11-14

CC1CC1 CC2CC2 CC3CC3 CC4CC4

E1 #TC-ve

E1 #TCE1 #TC+ve

E1+ve

...

E2 10040

...

500100

E2 10060

500400

E3E3

E4E4E4E4

10060 TOT

#TC-VE

10040

TOT #TC +VE

W2T

W3T

Are test assets good?1quality of test cases

1

2

3

4

Applicable test types considered?1

Any anomalies in #TC wrt importance2

#TC decreasing wrt levels?3

Countable? +ve:-ve ratio ok?4


L1L1 L2L2 L9L9

T1T1 T2T2 T3T3 TnTn



E1 Tot%-ve%

E1 Tot%E1 Tot%+ve%

E1+ve%

...

E2 90%50%

...

85%75%

E2 90%100%

85%85%

E3E3

E4E4E4E4

97%87%

100%100% TOT

%-VE

97%100%

100%100%

TOT% +VE

W2T

W3T

Have we assessed completely?2quality of execution

4

1 2

3

How are we doing wrt test for a entity?1

How are we doing wrt entity as a whole?2

How are we doing wrt a test & level?3

Are we making progress every cycle?4


L1L1 L2L2 L9L9

T1T1 T2T2 T3T3 TnTn



E1 CI(%)-CI%

E1 CI(%)E1 CI(%)+CI%

E1+CI%

...

E2 90%50%

...

85%75%

E2 90%100%

85%85%

E3E3

E4E4E4E4

97%87%

100%100% TOT

%-VE

97%100%

100%100%

TOT% +VE

W2T

W3T

How good are the outcomes?3quality of product/application

4

1 2

3

How clean is the entity wrt CC?1

How clean is the full entity?2

How are the entities f faring wrt a CC?3

How clean is the system as of now?4

If TC are proven complete, then CI (Cleanliness Index) =#TC-Passed/#TC-Total

Note that blocked TC would be considered as potentially failed.

Note we are measuring the “intrinsic cleanliness” here.


Summarising : Good “mirror” needs to tell ...

16

Are test assets good?1

Have we assessed completely?2

How good are the outcomes?3

quality of test cases

quality of execution

quality of product/application


HBT OverviewSIX staged purposeful activities, powered by EIGHT disciplines of thinking

17

D1D2

D4

D3

D5D6

D7

D8

Core Concepts

S1

S2

S3S4

S5

S6SIX Stages of DOINGS1: Understand expectationsS2: Understand contextS3: Formulate hypothesisS4: Devise proofS5: Tooling supportS6: Assess & Analyse

EIGHT Disciplines of ThinkingD1: Business value understandingD2: Defect hypothesisD3: Strategy & PlanningD4: Test designD5: ToolingD6: VisibilityD7: Execution & ReportingD8: Analysis & Management

Uses 32 Core ConceptsFor Problem solving - Techniques, Principles, Guideline

Click here to know more about HBT.http://stagsoftware.com/blog?p=570

http://stagsoftware.com/blog?p=570

http://stagsoftware.com/blog?p=570

http://slidesha.re/qBMNiy

http://slidesha.re/qBMNiy

© 2014 STAG Software Private Limited. All rights reserved. www.stagsoftware.com

Deliver superior outcomes using HBT Quality Visualisation ToolPowered by HBT

Are test assets good? “Quality of assets”1

Have we assessed completely? “Quality of execution”2

How good are the outcomes?“Quality of the product/application”3

HBT is the intellectual property of STAG Software Private Limited.STEMTM is the trademark of STAG Software Private Limited.

@stagsoft

blog.stagsoftware.com

Connect with us...

Thank you.

http://www.stagsoftware.com

http://www.stagsoftware.com

Technology

Deliver Superior Outcomes Using HBT Visualization Tool