Upload
morgan-austin
View
231
Download
1
Tags:
Embed Size (px)
Citation preview
Learning Objectives
• Define specifications with exceptions• Choose between checked/unchecked
exceptions and implement them• Decide when to handle exceptions or
propagate them• Define the goals and characteristics of testing• Write black-box test cases for programs• Write glass-box test cases for programs
What’s wrong with this ?
• Caller must remember to not call factorial() with n < 0 behavior is undefined otherwise.
• These are called partial procedures as they have a REQUIRES clause, as opposed to full procedures.
public static int factorial (int n) // REQUIRES: n >= 0 // EFFECTS: returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0.
Why not do this ?
public static int factorial (int n) // EFFECTS: returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0. // and -1 for n < 0
Why is this not an elegant solution ?1.Not really the purpose of a return value2.Caller may forget to check for this case3.What if every return value is a possible result ?
Exceptions
• Mechanism to signal abnormal/unexpected inputs or conditions in the program
• Typically used for signaling errors at runtime
• May be used during development to indicate situations that haven’t yet been handled
Exceptions in Java
• Exceptions need to be specified in procedure signature (declaration) with throws clause– Strictly true only for checked exceptions (later)– Separate multiple exceptions with commas
public static int factorial( int n ) throws NonPositiveException
Exceptions: Specifications
• EFFECTS clause must reflect both the exception and the condition under which it is thrown
• REQUIRES clause should not list the condition• No changes to the MODIFIES clause (usually)
public static int factorial(int n) throws NonPositiveException {
// EFFECTS: If (n < 0) throws NonPositiveException,// otherwise returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0
Learning Objectives
• Define specifications with exceptions• Choose between checked/unchecked
exceptions and implement them• Decide when to handle exceptions or
propagate them• Define the goals and characteristics of testing• Write black-box test cases for programs• Write glass-box test cases for programs
Exceptions: Implementation
public static int factorial(int n) throws NonPositiveException {// EFFECTS: If (n < 0) throws NonPositiveException,// otherwise returns n!, which is 1 for n=0, // and 1*2*3*...*(n-1)*n for any n > 0 if (n < 0) throw new NonPositiveException(“Factorial”); ….
Exception class must be defined by the user
Checked vs. Unchecked
checked unchecked
listed in the
procedure’s signature
mandatory optional
user codemust handle
can handle
Exception types in Java
Throwable
Error Exception
RuntimeException
unchecked exceptions
checked exceptions
NullPointerException IOException
What happens when an exception is thrown in Java ?
• If it is a checked exception, then the calling procedure must handle the exception or explicitly propagate it
• If it is an unchecked exception, it propagates in stack till it finds an appropriate catch block
foofoo
fooCallerfooCaller
mainmainstackgrowth
propagationdirection
Stack
Some Rules of Thumb …
• Always use Checked Exceptions unless– When you know the caller is unlikely to exercise
the exceptional condition e.g., internal routine– When the situation being handled is truly out of
the ordinary (e.g., Out of memory error)– There is a convenient way to check for the
condition apriori e.g., NullPointerException– The exception is meaningless outside its context
Exceptions: Group Activity• Consider the search procedure below. Can you
rewrite it to use exceptions ? Which of the exceptions will you make checked and why ?
public static int search( int[] a, int x) {// REQUIRES: a is not NULL
// MODIFIES: None // EFFECTS: Examines each of a[0], a[1]… in any // order and returns the index of an array // element that equals to x. If no such // element is found, return -1.
Learning Objectives
• Define specifications with exceptions• Choose between checked/unchecked
exceptions and implement them• Decide when to handle exceptions or
propagate them• Define the goals and characteristics of testing• Write black-box test cases for programs• Write glass-box test cases for programs
How to handle exceptions ?• Using the try-catch-finally construct in Java
try {doSearch(); // Can throw three types of exceptions
} catch (NullPointerException np) {// Take appropriate action 1
} catch (ElementNotFoundException ep) { // Take appropriate action 2
} finally { // Common cleanup code for all catch blocks // Called even if the exception handling code throws
exceptions}
What can we do in the catch block ?
Handling the exception• Correct the circumstances that
led to the exception or print a descriptive error message.
catch(ElementNotFoundException e)
{ // Can use the variable e system.out.println(e);}
Throw a new exception• Sometimes the exception may
indicate a larger problem in the program, which makes sense only to caller. The caller then throws a new exception.
catch (NullPointerException np) { // Possible out of memory
throw new OutOfMemException(“foo”);
}
Exception Propagation
• When is it better to not handle the exception– Because you have insufficient info at the caller– Because you know the exception is handled later, say
by your caller (s), so avoid duplication of effort
How to propagate an exception ?static void insertIfNotFound(int[] a, int x) throws
NullPointerException { // EFFECTS: If the element x is not present in a, // insert it into a. Otherwise, do nothing.
When should you propagate an exception ?
• Propagate an Exception if it is handled elsewhere OR if you have insufficient information to handle the error at this level
• Make sure you write the exception in the proc. declaration if it is a checked exception
• Handle an exception if you have sufficient information to bypass the error, print a error message or if you need to re-interpret it (i.e., throw a new exception)
• Make sure that you don’t throw the same exception again
Learning Objectives
• Define specifications with exceptions• Choose between checked/unchecked
exceptions and implement them• Decide when to handle exceptions or
propagate them• Define the goals and characteristics of testing• Write black-box test cases for programs• Write glass-box test cases for programs
Validation
• Does the software do what was wanted?– Increase confidence that a program will function as desired.– This is difficult to determine and involves subjective judgements
• Does the software meet its specification?– This can be objective if the specifications are sufficiently precise
Experiment withthe program(Testing)
Experiment withthe program(Testing)
Reason aboutthe program(Formal verification)
Reason aboutthe program(Formal verification)
Inspect theprogram(Reviews)
Inspect theprogram(Reviews)
Validation
Code Reviews
• Inspect other’s code for bugs (manually)
• Why don’t code reviews alone work ?– Ad-hoc techniques, no standard way of doing it– Not reproducible – too much reliance on humans– Errors can occur even when all standard coding
practices are followed (corner cases)– Often, things that are difficult for one person are
also difficult for another (Knight & Leveson study)
Formal Verification
• Tests the correctness of a piece of code without executing it– Requires detailed formal specifications of behavior– Can reveal the presence of hard-to-find bugs
Theorem-proving-Can prove correctness in all cases (inductive)-Requires significant expert intervention
Model-Checking-Proves correctness up to a finite depth bound-Can be completely automated, but not always scalable
Formal Verification
EECE 310 - 24
Testing
Testing shows that a program meets its specification
Testing can never be exhaustive for non-trivial programs
So pick a partition of the input space for testing
Systematic testing depends on partitioning1. Partition the set of possible behaviors of the system2. Choose representative samples from each partition3. Make sure we covered all partition
How do you identify suitable partitions?That’s what testing is all about!!!Methods: black box, glass box, ...
Learning Objectives
• Define specifications with exceptions• Choose between checked/unchecked
exceptions and implement them• Decide when to handle exceptions or
propagate them• Define the goals and characteristics of testing• Write black-box test cases for programs• Write glass-box test cases for programs
Black-box Testing
• Generate test cases from the specification only (don’t look at code)
• Advantages:–Avoids making the same assumptions as the
programmer–Test data is independent of the procedure’s
implementation–Results can be interpreted without knowing
implementation details–Forces you to write a good specification
Test cases for black box testingPaths through the spec
– E.g., choose test cases that cover each part of the ‘requires’, ‘modifies’ and ‘effects’ clauses
Boundary conditions– Choose test cases that are at or close to boundaries for ranges of
inputs– Test for aliasing errors (e.g., two parameters referring to the same
object)
Invalid input– The program should degrade gracefully, without experiencing
data loss
Black Box Testing: Example
• Paths through the spec:– “x 0” means “x>0 or x = 0”, so test both “paths”– x < 0 is another path
• Boundary conditions:– x >=0 choose:
• 1, 0, as values for x
– x < 0 choose:• -1 as value for x
– Also, try very large & very small values for x
• Other cases• Invalid output: Not allowed by the type-system• Aliasing: Int is a primitive type, so not possible
• Values for epsilon > 0.001, values for epsilon < 0.00001
int abs(int x) { // returns: x < 0 => returns -x // otherwise => returns x
int abs(int x) { // returns: x < 0 => returns -x // otherwise => returns x
Aliasing: ExampleStatic void appendVector(Vector v1, Vector v2) throws NullPointerException {// REQUIRES: v1 and v2 contain non-NULL elements// EFFECTS: if v1 or v2 is null, throws NullPointerException // else removes all elements of v2 and appends them// in reverse order to the end of v1
if (v1 == null) throw new NullPointerException(“Vector.appendVector”); while (v2.size ( ) > 0 ) {
v1.addElement( v2.lastElement( ) ); v2.removeElementAt(v2.size() – 1 ); }}
What happens if v1==v2 ?
Black Box Testing: Abs Program
• Can you spot the error in the code below ?– Will this error be detected by the test suite ?
int abs(int x) { // returns: x < 0 => returns -x // otherwise => returns x if (x < -5) return –x; else return x;}
int abs(int x) { // returns: x < 0 => returns -x // otherwise => returns x if (x < -5) return –x; else return x;}
Group Activity• Write black box test cases for the search
procedure with exceptions. Also specify for each test case, what aspect it tests (e.g., invalid input).
public static int search( int[] a, int x) throw NullPointerException, ElementNotFoundException{
// MODIFIES: None // EFFECTS: Examines each of a[0], a[1]… in any // order and returns the index of an array // element that equals to x. Otherwise, If // no such element is found, throw // an ElementNotFoundException. // if a is NULL, throw NullPointerException
Learning Objectives
• Define specifications with exceptions• Choose between checked/unchecked
exceptions and implement them• Decide when to handle exceptions or
propagate them• Define the goals and characteristics of testing• Write black-box test cases for programs• Write glass-box test cases for programs
Glass Box testing
• Examine the code and test all paths– Because black box testing can never guarantee we
exercised all the code• Path completeness:
– A path is a sequence of statements in the code– A test set is path complete if each path through
the code is exercised by at least one case in the test set
• Not the same as saying each statement in the code is reached!! (Why ?)
Glass Box Testing: Example
• There are four paths in the following code
• So we need at least 4 test-cases. Example:– x=3, y=2, z=1– x=3, y=2, z=4– x=2, y=3, z=2– x=2, y=3, z=4
• Are the above tests sufficient ?
int maxval(int x,int y,int z) {// EFFECTS: returns maximum// value of the three inputs if (x > y) {
if (x > z) return xelse return z }
else {if (y > z) return yelse return z } }
int maxval(int x,int y,int z) {// EFFECTS: returns maximum// value of the three inputs if (x > y) {
if (x > z) return xelse return z }
else {if (y > z) return yelse return z } }
x > yx > y
x > zx > z y > zy > z
Weaknesses of path completeness• Path completeness is insufficient
Consider the test case x=4, y=1, z=2. This is path complete. The program performs correctly on this test case but
the program is still wrong !!• Path completeness is usually infeasible
– How many paths are there through this program segment (for any a[i])?
– Loops are problematic. Try:• test 0, 1, 2, n-1, and n iterations, (n is the max number of
iterations possible)
for (j=0, i=0; i<100; i++)if a[i]=true then j=j+1
for (j=0, i=0; i<100; i++)if a[i]=true then j=j+1
int maxval (int x, y, z) {/* EFFECTS: returns maximum value
of the three inputs */ return z;}
int maxval (int x, y, z) {/* EFFECTS: returns maximum value
of the three inputs */ return z;}
How to approximate Path Completeness ?
1. Loops:– With variable amounts of iteration
• 0, 1, 2 iterations• All possible ways to exit loop for each case
2. For each statement where an exception could be raised– Test case that attempts to cause the exception
3. Straight-line codeAll paths through the code
THESE ARE ALL HEURISTICS. THERE IS NO SILVER BULLET.
Path Completeness Approximation: Example
• Test NPException:– factorial(-1)
• Paths through the loop:– 0 iterations (n = 1)
• returns 1– 1 iteration (n = 2)
• returns 1 (Bug found !)• 2 iterations (n = 3)
• returns 2 (Bug found !)
The program has a bug the loop condition should be (i <= n) not (i < n)
int factorial( int n ) {int result = 1; int i;
if (n < 0) throw new NPException(“fact”);
for (i = 1; i < n; i++) { result = result * i;
} return result;}
Group Activity
• Write glass-box test cases for an implementation of search that was written by some other group in the class. Do not attempt to correct any bugs unless they’re egregious. The goal is to use glass-box tests to find them.
Learning Objectives
• Define specifications with exceptions• Choose between checked/unchecked
exceptions and implement them• Decide when to handle exceptions or
propagate them• Define the goals and characteristics of testing• Write black-box test cases for programs• Write glass-box test cases for programs