The Survey Of Unit Testing Practices

Per Runeson,Lund University,

Sweden Th7

A Survey of Unit Testing Practices

Prof. Per Runeson – Lund UniversityPer Beremark – SPIN-syd

What is Unit Testing?

“testing of individual softw are or hardw are units or groups of related units” [IE E E 610.12]

“U nit testing tests individual softw are com ponents or a collection of components. Testers define the input domain for the units in question and ignore the rest of the system. Unit testing sometimes requires the construction of throwaway driver code and stubs and is often performed in a debugger” [W hittaker 00]

“a test, executed by the developer in a laboratory environment, that should demonstrate that the program meets the requirements set in the design specification” [K oom en and P ol 00]

What Does It Mean In Practice?

1. What do people refer to as unit testing?

2. What are the strengths regarding unit testing in industry?

3. Which are the problems regarding unit testing?

Key Findings on Unit Testing

• What?– Technical!– Specified or not?

• How?– Structure-based– Not formally

• Who?– Developer (team)– Not test or QA

• When?– Each build/day/week– Takes seconds to

hours to run• Why?

– Assuring functionality

Participants:SPIN-syd of Southern Sweden

• 19 companies– Telecom– Automation– Case tools– Information

systems– Banking– Health care– Consulting– Transportation

• Company size– 1-1000 developers

• 17 participants in focus group

• 15 in questionnaire

Survey Procedures

Unit testing is…

Unit testing-What-Strengths-Problems

Analysis-What?-How?-Where?-Who?

Q ’s

Note

1. Individual reflection

2. Focus groupdiscussion

Note

3. Documentand structure

4. Analysis

5. Surveypreparation

6. Survey conduct

7. Survey analysis

Q & A ’sQ & A ’s

Q & A ’s

Note

3rd party verification

Unit test –smallest specified unit

Test first

Training

Focus Group Meeting

Analysis Framework

• What?• How?• Where?• Who?• When?• Why?

Unit test

Unit test

Unit test

Unit test

Unit test

Unit test

Unit test

Unit test

Questionnaire

Analysis

0

2

4

6

8

10

12

14 Fully agree Agree

Neutral Disagree

Totally disagree N/A

What TimeWhenWho decides?Who conducts?How Why

What?

• Definition– Test of smallest unit or units

• Strength– Unit identification– Test of surrounding modules

• Weaknesses– GUI test– Unit identification– Test scripts and harness maintenance– Data structures

How?

• Definition– Structure-based– Preferably automated

• Strength– Test framework

• Weaknesses– Documentation– Framework tailoring– Test selection– Test metrics

Who?

• Definition– By developer

• Strength– Independent test– Competence network

• Weaknesses– Competency– Independence– Introduction strategy

When?

• Definition– Quick feedback

• Strength– Continuous regression test

• Weaknesses– Stopping criteria

Why?

• Definition– Ensure functionality

• Strength– External requirement (safety)– Agile methods

• Weaknesses– Cost vs. value

Further Use of the Survey

• Clarify unit test definition in your company

• Lightweight assessment of unit testing practices– Benchmark– Improve

R ead m ore… … in IE E E S oftw are July/A ugust 2006

http://serg.telecom.lth.se

Definitions

0

2

4

6

8

10

12

14 Fully agreeAgreeNeutralDisagreeTotally disagree

N/A

What TimeWhenWho Who How Why

Strengths and weaknesses

0

2

4

6

8

10

12Very goodGoodNeutralBadVery badN/A

What WhenHow WhyWho