Test Tools Test Tools

To Build Test ReportsTo Build Test Reports

Test ToolsTest Tools

Testers use many different tools to Testers use many different tools to help in analyzing the results of help in analyzing the results of testing and to create the information testing and to create the information contained in the test reports.contained in the test reports.

Test ToolsTest Tools

Pareto ChartsPareto Charts Cause Effect DiagramCause Effect Diagram Check SheetCheck Sheet

Pareto ChartsPareto Charts

• Type of Bar chart to view the causes Type of Bar chart to view the causes in order of priority from largest to in order of priority from largest to smallestsmallest

• The Pareto chart provides an The Pareto chart provides an effective tool to graphically show effective tool to graphically show where significant problems and where significant problems and causes are in a processcauses are in a process

Pareto ChartPareto Chart

0

10

20

30

40

50

60

70

80

Characteristics

Cause Effect DiagramCause Effect Diagram

A team tool used to help identify the causes of problems related to processes, products and

services. This technique keeps teams focused on a problem and potential causes. By better

understanding problems within the work processes, teams can reach probable and root causes of a problem.

Cause Effect Cause Effect Diagram(Fishbone Diagram(Fishbone

Diagram)Diagram)

Check SheetCheck Sheet

A check sheet is a technique or tool to record the number of occurrences over a specified interval of time; a data sample to determine the frequency of an event. The recording of data, survey, or sample is to support or validate objectively the significance of the event. This usually follows the Pareto analysis and cause and effect diagram to validate and verify a problem or cause.

Sample Check SheetSample Check Sheet

Reporting Test Reporting Test ResultsResults

Eight Interim Reports:Eight Interim Reports: Functional Testing Status Functional Testing Status  Functions Working TimelineFunctions Working Timeline Expected verses Actual Defects Expected verses Actual Defects

Uncovered TimelineUncovered Timeline Defects Uncovered verses Corrected Gap Defects Uncovered verses Corrected Gap

TimelineTimeline Average Age of Uncorrected Defects by Average Age of Uncorrected Defects by

TypeType Defect DistributionDefect Distribution Relative Defect DistributionRelative Defect Distribution Testing ActionTesting Action

Functional Testing Functional Testing Status ReportStatus Report

This report will show percentages of This report will show percentages of the functions which have been:the functions which have been:

Fully TestedFully Tested

Tested With Open DefectsTested With Open Defects

Not TestedNot Tested

Functional Testing Status Report

40

30

20

10

Fully Tested Not Tested with open defects Tested

Functions Working Functions Working TimelineTimeline

This report will show the actual plan This report will show the actual plan to have all functions working verses to have all functions working verses the current status of functions the current status of functions working.working.

An ideal format could be a line An ideal format could be a line graph.graph.

40

30

20

10

Functions Working Timeline

Jan Feb Mar

Timeline

Actual Status

Current Status

Expected verses Actual Expected verses Actual Defects DetectedDefects Detected

This report will provide an analysis This report will provide an analysis between the number of defects between the number of defects being generated against the being generated against the expected number of defects expected number of defects expected from the planning stage.expected from the planning stage.

Expected Versus Actual Defects Detected

40

30

20

10

M1 M2 M3

Expected defects

Actual defects

Defects Detected verses Defects Detected verses Corrected GapCorrected Gap

This report, ideally in a line graph This report, ideally in a line graph format, will show the number of format, will show the number of defects uncovered verses the defects uncovered verses the number of defects being corrected number of defects being corrected and accepted by the testing group.and accepted by the testing group.

If the gap grows too large, the If the gap grows too large, the project may not be ready when project may not be ready when originally planned.originally planned.

40

30

20

10

Defect Detected Versus Corrected Gap

Module

Detected

Corrected

No

Of

D

E

F

E

C

T

S

Gap

Average Age Average Age Uncorrected Defects by Uncorrected Defects by

TypeType This report will show the average This report will show the average

days of outstanding defects by type days of outstanding defects by type (Sev 1, Sev 2, etc.). (Sev 1, Sev 2, etc.).

In the planning stage, it is beneficial In the planning stage, it is beneficial to determine the acceptable open to determine the acceptable open days by defect type.days by defect type.

Average Age UnCorrected Vs Defects By Type

20

10

5

0

Critical Major Minor

No

Of

D

A

Y

S

Defect DistributionDefect Distribution

This report will show the defect This report will show the defect distribution by function or module. distribution by function or module. It can also show items such as It can also show items such as numbers of tests completed.numbers of tests completed.

Defect Distribution

40

30

20

10

M1 M2 M3

No

Of

D

E

F

E

C

T

S

15

30

20

Relative Defect Relative Defect DistributionDistribution

This report will take the previous report This report will take the previous report (Defect Distribution) and normalize the (Defect Distribution) and normalize the level of defects.level of defects.

   An example would be one application might An example would be one application might

be more in depth than another, and would be more in depth than another, and would probably have a higher level of defects.probably have a higher level of defects.

However, when normalized over the However, when normalized over the number of functions or lines of code, would number of functions or lines of code, would show a more accurate level of defects.show a more accurate level of defects.

Testing ActionTesting Action

This report can show many different This report can show many different things, including possible shortfalls things, including possible shortfalls in testing.in testing.

   Examples of data to show might be Examples of data to show might be

number of Sev 1 defects, tests that number of Sev 1 defects, tests that are behind schedule, and other are behind schedule, and other information that would present an information that would present an accurate testing picture.accurate testing picture.

Final Test ReportFinal Test Reportss

Test reports should be prepared at the conclusion of each level of testing. This might include:

Unit Test Report Integration Test Report System Test Report Acceptance Test Report

  

Unit Test ReportUnit Test Report

Integration Test ReportIntegration Test Report

Contents of System Test Contents of System Test ReportReport

General InformationGeneral Information Test Results and FindingsTest Results and Findings Software Function and FindingsSoftware Function and Findings