Testing, The Ugly Stepchild of Software Development

Better Living through Software Testing

Chris Berry – Feb 21, 2000

Chris Berry cberry@cyberplus.com

Why Test?

It’s easy to fool yourself.

Increase confidence.

Increase productivity.

Increase courage.

It simply makes programming more fun!

Test Early, Test Often

Defects are order of magnitude cheaper to fix when programmers find their own errors.

No communication cost.

The bug tracking system does not become involved.

The error does not block or corrupt the work of other developers.

Preemptive Testing

The Art of catching bugs before they happen.

Defensive Programming.

Programming by Contract.

The Assertion

The hallmark of Defensive Programming.

Allows you to assert that a particular condition is met, and commonly, if it is not, terminate execution.

Pre and post conditions.

Not to be confused with Exceptions.

Sanity Checking

A simple coding convention which requires all Classes to provide a boolean isSane method.

Used to determine whether a particular Instance is in a sane state.

Logging/Debug Prints

Low tech debugging technique. Often the only way to debug threaded code.

Can speed development.

Permanent artifact.

Can result in “Scroll-blindness”. Best to divert logging to a file rather than standard out.

The log4j package for Java.

Unit Testing

Test each unit of code independently.

In OO, this equates to testing a Class and its public interface.

Involves debugging, simultaneously, the code, the test, and the Spec.

Unit Testing is Glass Box Testing

You Must be aware of

All branches of the method under test.

What inputs should produce what results or Exceptions.

The interplay of affected member variables and arguments.

Building a Good Unit Test

Test for

Data integrity:

Valid input: Insure that correct outputs are generated for correct input conditions.

Invalid input: Insures proper error handling.

Deciding on Unit Test Input

Equivalence Partitioning.

Good partitioning gives rigor without redundancy.

Boundary Analysis

Things go wrong at the edges.

Expected and unexpected Exceptions.

Equivalence Partitioning

An equivalence class is a definition of a group of inputs, any one of which should be treated exactly the same by the method under test.

Eliminates redundant tests. The behavior of the method should be “equivalent” so only one test need be run for any equivalence class.

Boundary Analysis

The term for choosing the appropriate values to test a particular equivalence subset.

Choose values at the edge of the set.

Also a good idea to include the transition values.

Dealing with Exceptions

Catch all unplanned Exceptions.

Match all thrown Exceptions with the conditions expected to produce them.

Classic Unit Test construct

Generate argument providers

Loops over all argument providers

Construct the Object under test (OUT)

Optionally save the state of the OUT

Call the method under test (MUT)

If an exception was generated

Test for unhandled exceptions

Else

Assertions on the resultant, OUT, andarguments

End loops

Integration Testing

Tests which prove that the program properly executes some required functionality.

Typically involves the integration of many Classes.

Often a test from the perspective of the customer.

eXtreme Programming

“A lightweight, low-risk, flexible, predictable, and fun way to develop software”.

Takes commonsense best practices and applies them to an extreme degree.

XP in a Nutshell

Code is king.

Embrace change.

Constant design.

Emphasis on refactoring.

The Four Variables

Cost

Time

Quality

Scope.

The Four Values of XP

Communication

Simplicity

Feedback

Courage.

Central Principles of XP

Rapid Feedback.

Assume Simplicity.

Incremental change.

Embrace change.

Quality work.

Secondary Principles of XP

Teach Learning.

Small Initial Investment.

Play to Win.

Concrete Experiments.

Honest communication.

Secondary Principles of XP, continued

Work with instincts.

Accepted responsibility.

Local adaptation.

Travel Light.

Honest measurement.

Four Basic Activities of XP

Coding.

Testing.

Listening.

Designing.

XP Development Practices

On-site Customer

System Metaphor

Pair Programming

Tests Before Code

Simple Design

XP Development Practices, continued

Merciless Refactoring

Continuous Integration

Collective Ownership

Coding Standards

Testing the XP way

A simplified, pragmatic, and less rigorous approach to testing.

Tests things that might break. Testing simple methods, like accessors, is valueless.

Write the tests first.

Keep the tests running.

Tests are automatic. Run the full suite several times a day.

JUnit, a Java testing framework. A C++ version (CPPUnit) is now available.

Summary

The earlier you catch a bug the less it is going to cost you.

Testing can actually speed up your development time, and improve your quality of life.

XP is a pragmatic approach to writing better software.

XP embraces change.

Acknowledgements

Some of the Slide Notes for this presentation were lifted verbatim from several sources; “Test Infected. Programmers Love Writing Tests”

by Eric Gamma and Kent Beck.

“eXtreme Summary - An Overview of eXtreme Programming eXplained: Embrace Change” by Brad Appleton.

“Divide and Conquer - Preemptive and Unit Testing” by Chris Berry

“The Pragmatic Programmer” by Andrew Hunt and David Thomas.

Resources

www.xprogramming.com/software.htm

www.extremeprogramming.org

www.pragmaticprogrammer.com