COMPARING THE DEFECT REDUCTION BENEFITS OF CODE INSPECTION AND TEST-DRIVEN

DEVELOPMENT

By Veronika Movagharianpourand Adam Brakel

IntroductionSoftware Developers face challenges:

Producing high-quality software with low-defect levels while doing it on-time and on-budget

4 Cases to ExploreCase 1: No Code Inspections or Test Driven Development

Case 2: Code Inspection Only

Case 3: Test Driven Development

Case 4: Code Inspection and Test Driven Development

No Code Inspections and NoTest Driven Development

We know from our reading and other student’s reports that programs will have bugs.

That the everyday debugging that occurs does not catch all of them.

So NOT testing in addition to the minimal amount carried out is just a poor design choice!

Code Inspection

We all know what the process for Code Inspections is…

Error Detection Efficiency of Software Inspection

Defect yield rates for SI ranges between 66 and 82 percent where the total number of defects prior to inspection is:

The Defect Detection Yield was found to be:

Test Driven Development

Writing automated unit tests before code Subsequent coding completed when all new

and previous tests succeed Process completed iteratively until software

is complete Successful unit tests execution verified after

write code and refactor steps

The Process of TDD Identifies and removes defects at the

introduced point in the development process

Early defect elimination is the TDD most significant benefit

Leads to huge cost saving due to early defect removal

An Example of Test Driven Development

Example Explanation –

Run Demo -

Purpose & Research Questions: 1. Which software defect reduction method

is most effective at reducing software defects?

H1:Code inspection is more effective than TDD at reducing software defects. 2. Are there interaction effects associated

with the combined use of these methods?H2: The combined use of code inspection and TDD is more effective than either method alone. 3. What are the relative costs of these

software defect reduction methods?H3:Code Inspection and TDD differ in implementation cost

Research Method

Programming assignment: creating part of a spam filter using Java

554 non-commentary source statements on average

261 NCSS provided in a starter project

Participants Excluded by Group:

Participants: undergraduate computer science students With good Java programming and object-

oriented design knowledge Highly motivated to perform well

Experimental Procedures: All Participants were trained in the

Inspection Process and TDD using JUnit. They were then given a detailed

description of the spam filter they were to code and were given two weeks to do so.

Then the resulting code from each of the four groups was analyzed.

Measurements:

Measurement explanation: - Using “yield” to measure selected method effectiveness- Yield measure is not reliably for TDD

number of defects found by using the method Yield = --------------------------------------------------------------------Total Number of defects prior to inspection

Cost of Testing Methods: The total cost is the sum of three areas of

activity; 1. the original development hours by the code

author. 2. the hours spent by software inspectors and

moderator (preparation time + meeting time). 3. hours spent by the author correcting

defects by inspection. TDD seems to save the implementation

cost Inspection had the highest

implementation cost

Implementation Cost:

Threats to Validity: Internal:

Section bias: possibility of unequal participants division

Mortality bias: unequal number of members in each group

Maturation bias: unequal participants learning rates during experiment

Order bias: vulnerability to order bias caused by performed inspections order and whether they were performed on the first or second inspection day

Implementation bias: variability in the treatment conditions implementing or applying means

Threats to Validity (continue)

External: Participants were undergraduate junior or senior

students, not professional programmers Code inspectors were also college students

without code inspection experience Code inspection process did not follow the

normal regulations. Authors did not participate the inspections The iterative cycle of rework, follow-up, and reinspection

did not performed Inspectors performed multiple inspections of

code performing the same function in a short time resulting in finding more defects in the later inspections

Results:

Implementation Cost Summaries:

Summary of Results

Summary and Conclusions: Compare of defect rates and implementation

costs associated with code inspection and test-driven development software defect reduction methods: Inspection:

More effective at reducing defects More expensive to implement

TDD May result in implementation cost savings

More research is require to validate this findings

¿¿¿ Questions ???