Chapter 8

Testing and Debugging

Goals • To learn how to carry out unit tests• To understand the principles of test case

selection and evaluation • To learn how to use logging and assertions • To become familiar with the debugger • To learn strategies for effective debugging

Unit Test

Test classes in isolation, outside the program in which they are used

Test one class at a timeSupply test inputs through test harness Test harness = program that feeds test

inputs to a class

Root Approximator• Example program to illustrate testing: square

root approximator Algorithm known to the ancient Greeks (Heron) Task: to compute the square root of a Given: a guess x (ok to start with 1) Actual square root lies between x and a/x

• Take midpoint (x + a/x) / 2 as a better guess Method converges rapidly. Square root of 100:

• Guess #1: 50.5 Guess #2: 26.24009900990099 Guess #3: 15.025530119986813 Guess #4: 10.840434673026925 Guess #5: 10.032578510960604 Guess #6: 10.000052895642693 Guess #7: 10.000000000139897 Guess #8: 10.0

File RootApproximator.java1 /** 2 Computes approximations to the square

root of 3 a number, using Heron's algorithm 4 */ 5 public class RootApproximator 6 { 7 /** 8 Constructs a root approximator for a

given number 9 @param aNumber the number from

which to extract the square root 10 (Precondition: aNumber >= 0) 11 */ 12 public RootApproximator(double

aNumber) 13 { 14 a = aNumber; 15 xold = 1;

16 xnew = a; 17 } 19 /** 20 Compute a better guess from

the current guess.21 @return the next guess 22 */23 public double nextGuess() 24 { 25 xold = xnew; 26 if (xold != 0) 27 xnew = (xold + a / xold) /

2; 28 return xnew; 29 }

31 /** 32 Compute the root by repeatedly improving the current guess until two

successive guesses are approximately equal.

34 @return the computed value for the square root 35 */ 36 public double getRoot() 37 { 38 while (!Numeric.approxEqual(xnew, xold)) 39 nextGuess(); 40 return xnew; 41 } 42 private double a; // the number whose square root is computed 44 private double xnew; // the current guess 45 private double xold; // the old guess 46}

File RootApproximatorTest.java1 import javax.swing.JOptionPane;

2

3 /**

4 This program prints ten approximations for a square root.

5 */

6 public class RootApproximatorTest

7 {

8 public static void main(String[] args)

9 {

10 String input =

11 JOptionPane.showInputDialog("Enter a number");

12 double x =

Double.parseDouble(input);

13 RootApproximator r = new RootApproximator(x);

14 final int MAX_TRIES = 10;

15 for (int tries = 1; tries <= MAX_TRIES; tries++)

16 {

17 double y = r.nextGuess();

18 System.out.println("Guess #" + tries + ": " + y);

19 }

20 System.exit(0);

21 }

22}

Unit Test with BlueJ

File RootApproximatorTest2.java1 import javax.swing.JOptionPane; 2 3 /** 4 This program computes square roots of

user-supplied inputs. 5 */ 6 public class RootApproximatorTest2 7 { 8 public static void main(String[] args) 9 { 10 boolean done = false; 11 while (!done) 12 {

13 String input =

JOptionPane.showInputDialog( 14 "Enter a number, Cancel to

quit"); 15

16 if (input == null) 17 done = true; 18 else 19 { 20 double x =

Double.parseDouble(input); 21 RootApproximator r = new

RootApproximator(x); 22 double y = r.getRoot(); 23 24 System.out.println("square

root of " + x + " = " + y); 26 } 27 } 28 System.exit(0); 29 } 30}

Read Inputs from File Prepare a file with test inputs, such as– 100

20410.250.01

Use input redirection– java RootApproximatorTest3 < test.in Output– square root of 100.0 = 10.0

square root of 20.0 = 4.47213595499958. . .

File RootApproximatorTest3.java1 import java.io.BufferedReader; 2 import java.io.InputStreamReader; 3 import java.io.IOException; 4 5 /** 6 This program computes square roots

of inputs supplied through System.in

8 */ 9 public class RootApproximatorTest3 10{ 11 public static void main(String[]

args) 12 throws IOException 13 { 14 BufferedReader console 15 = new BufferedReader(new InputStreamReader(System.in)); 16 boolean done = false;

17 while (!done) 18 { 19 String input =

console.readLine(); 20 if (input == null) done = true; 21 else 22 { 23 double x = Double.parseDouble(input);

RootApproximator r = new RootApproximator(x);

25 double y = r.getRoot(); 26

System.out.println("square root27 of " + x+ " = " + y); 29 } 30 } 31 } 32}

Sources of Test Data

• Provided by humansRootApproximatorTest3Computer-generated sequenceRootApproximatorTest4 Random sequenceRootApproximatorTest5

RootApproximatorTest41 /** Computes square roots of input3 values supplied by a loop. 4 */ 5 public class RootApproximatorTest4 6 { 7 public static void main(String[] args) 8 { 9 final double MIN = 1; 10 final double MAX = 10; 11 final double INCREMENT = 0.5; 12 for (double x = MIN; x <= MAX;

x = x + INCREMENT) 13 { 14 RootApproximator r = new

RootApproximator(x); 15 double y = r.getRoot(); 16 System.out.println("square root of "17 + x + " = " + y); 18 } }}

1 import java.util.Random; 3 /** 4 Computes square roots of random inputs. 5 */ 6 public class RootApproximatorTest5 7 { 8 public static void main(String[] args) 9 { 10 final double SAMPLES = 100; 11 Random generator = new Random(); 12 for (int i = 1; i <= SAMPLES; i++) 13 { // generate random test value 14 15 double x = 1.0E6 *

generator.nextDouble();16 RootApproximator r = new

RootApproximator(x); 17 double y = r.getRoot(); 18 System.out.println("square root of "19 + x + " = " + y); 20 } } }

RootApproximatorTest5

Test Cases Positive test case: expect

positive outcomeE.g square root of 100

Negative test case: expect negative outcomeE.g square root of 100

Boundary test case: at boundary of domainFrequent cause for errorsE.g square root of 0

ManualRootApproximatorTest3

Check property of resultE.g. square root squared = original valueRootApproximatorTest6

Oracle = slower way of computing answerE.g. square root of

x = x1/2

RootApproximatorTest7

Evaluation

File RootApproximatorTest6.java1 import java.util.Random; 3 /** 4 This program verifies the computation5 of square root values by checking a mathematical property of square roots. 6 */ 7 public class RootApproximatorTest6 8 { 9 public static void main(String[] args) 10 { 11 final double SAMPLES = 100; 12 int passcount = 0; 13 int failcount = 0; 14 Random generator = new Random(); 15 for (int i = 1; i <= SAMPLES; i++) 16 {

double x = 1.0E6 * generator.nextDouble();

20 RootApproximator r = new RootApproximator(x);

21 double y = r.getRoot(); 22 System.out.println("square root of "23 + x + " = " + y); 25 if (Numeric.approxEqual(y * y, x)) 28 { 29 System.out.println("Test passed."); 30 passcount++; 31 } 32 else { 34 System.out.println("Test failed."); 35 failcount++; 36 } 37 } 38 System.out.println("Pass: " + passcount); 39 System.out.println("Fail: " + failcount); 40 } 41}

File RootApproximatorTest7.java1 import java.util.Random; 3 /** 4 Verifies the computation of square5 root values by using an oracle. 6 */ 7 public class RootApproximatorTest7 8 { 9 public static void main(String[] args) 10 { 11 final double SAMPLES = 100; 12 int passcount = 0; 13 int failcount = 0; 14 Random generator = new Random(); 15 for (int i = 1; i <= SAMPLES; i++) 16 { 19 double x = 1.0E6 *

generator.nextDouble(); 20 RootApproximator r = new

RootApproximator(x); 21 double y = r.getRoot();

22 System.out.println("square root of " 23 + x + " = " + y); 25 double oracleValue=Math.pow(x,0.5);

26 if (Numeric.approxEqual(y, oracleValue))

30 { 31 System.out.println("Test passed."); 32 passcount++; 33 } 34 else { 36 System.out.println("Test failed."); 37 failcount++; 38 } 39 } 40 System.out.println("Pass: " +

passcount); 41 System.out.println("Fail: " +

failcount); 42 } 43}

Regression Testing• Save test cases Automate testing• java Program < test1.in > test1.out

java Program < test2.in > test2.outjava Program < test3.in > test3.out

• Repeat test whenever progam changes Test suite = collection of test cases Cycling = bug that is fixed but reappears in later versionsRegression testing = testing against past failures

Test Coverage• Black-box testing: test functionality without

understanding internal structure

• White-box testing: take internal structure into account when designing tests

• Test coverage: the code that is actually executed during test cases

• Easy to overlook error branches during testingMake sure to execute each branch in at least one test case

Program Trace Output statements in your program for diagnostic purposes

if (status == SINGLE) { System.out.println("status is SINGLE"); ... } ...

Stack trace tells you the contents of the call stackThrowable t = new Throwable(); t.printStackTrace(System.out);

• Looks like exception report:– java.lang.Throwable

at TaxReturn.getTax(TaxReturn.java:26) at TaxReturnTest.main(TaxReturnTest.java:30)

Drawback of trace messages: Need to remove them when testing is complete, stick them back in when another error is found

Logging Get global logger object:

Logger logger = Logger.getLogger("global"); Log a message

logger.info("status is SINGLE"); By default, logged messages are printed. Turn them off with

logger.setLevel(Level.OFF);

Assertions Assertion = assumption that you believe to be true

– assert y >= 0;root = Math.sqrt(y);

If assertion fails, program is terminated Can be used to assert pre/postconditions Must compile/run with special flags

– javac -source 1.4 MyProg.javajava -enableassertions MyProg

The Debugger Debugger = program to run your program,

interrupt it, and inspect variables Three key commands:

Set Breakpoint

Single Step

Inspect Variable

Two variations of Single Step Step Over = don't enter method call

Step Inside = enter method call

The Debugger Stopping at a Breakpoint

Inspecting Variables

Sample Debugging Session Word class counts syllables in a word Each group of adjacent vowels (aeiouy) is a

syllable However, an e at the end of a word doesn't count If algorithm gives count of 0, increment to 1 Constructor removes non-letters at beginning

and end Buggy output:• Syllables in hello: 1

Syllables in regal: 1Syllables in real: 1

File Word.java1 public class Word 2 { 3 /** 4 Constructs a word by removing

leading and trailing non- letter characters, such as punctuation marks.

6 @param s the input string 7 */ 8 public Word(String s) 9 { 10 int i = 0; 11 while (i < s.length() && 12 !Character.isLetter(s.charAt(i))) 13 i++; 14 int j = s.length() - 1; 15 while (j > i && !

Character.isLetter(s.charAt(j))) 16 j--;

17 text = s.substring(i, j + 1); 18 } 19 /** 20 Returns the text of the

word, after removal of the 21 leading and trailing non-

letter characters. 22 @return the text of the

word 23 */ 24 public String getText() 25 { 26 return text; 27 } 29 /** 30 Counts the syllables in the

word. 31 @return the syllable count 32 */

33 public int countSyllables() 34 { 35 int count = 0; 36 int end = text.length() - 1; 37 if (end < 0) return 0; 38 char ch =

Character.toLowerCase(text.charAt(end));

41 if (ch == 'e') end--; 43 boolean insideVowelGroup44 = false; 44 for (int i = 0; i <= end; i++) 45 { 46 ch= Character.toLowerCase

(text.charAt(i)); 47 if ("aeiouy".indexOf(ch)

>= 0)

48 {

49 // ch is a vowel 50 if (!insideVowelGroup) 51 { 52 // start of new vowel group 53 count++; 54 insideVowelGroup = true; 55 } 56 } 57 else 58 insideVowelGroup = false; 59 } 61 // every word has at least one

syllable 62 if (count == 0) 63 count = 1; 65 return count; 66 } 68 private String text; 69}

File WordTest.java1 import java.util.StringTokenizer; 2 import javax.swing.JOptionPane; 4 public class WordTest 5 { 6 public static void main(String[] args) 7 { 8 String input =9 JOptionPane.showInputDialog ("Enter a sentence"); 10 StringTokenizer tokenizer =

new StringTokenizer(input); 11 while

(tokenizer.hasMoreTokens()) 12 {

13 String token = tokenizer.nextToken();

14 Word w = new Word(token); 15 int syllables =

w.countSyllables(); 16 System.out.println(

"Syllables in " + w.getText() + ": " + syllables);

18 } 19 System.exit(0); 20 } 21}

Final Letter Test is Not Correct

Set breakpoint in line 35 (first line of countSyllables)

Start program, supply input hello Method checks if final letter is 'e'Run to line 41 Inspect variable ch Should contain final letter but contains 'l'

Debugging the countSyllables Method

The Current Values of the Local and Instance Variables

More Problems Found end is set to 3, not 4 text contains "hell", not "hello" No wonder countSyllables returns 1 Culprit is elsewhere Can't go back in time Restart and set breakpoint in Word

constructor

Debugging the Word Constructor Supply "hello" input again

Break past the end of second loop in constructor

Inspect i and j

They are 0 and 4--makes sense since the input consists of letters

Why is text set to "hell"?

Off-by-one error: Second parameter of substring is the first position not to include

text = substring(i, j);should betext = substring(i, j + 1);

Debugging the Word Constructor

Another ErrorFix the error Recompile Test again:–Syllables in hello: 2

Syllables in regal: 1Syllables in real: 1

Oh no, it's still not right Start debuggerErase all old breakpoints Supply input "regal"

Debugging countSyllables (again) Break in the beginning of countSyllables Single-step through loop First iteration ('r'): skips test for vowel Second iteration ('e'): passes test, increments

count Third iteration ('g'): skips test Fourth iteration ('a'): passes test, but doesn't

increment count insideVowelGroup was never reset to false Fix and retest: All test cases pass Is the program now bug-free? The debugger

can't answer that.

The First Bug

Therac-25 Facility