Upload
marianna-lloyd
View
222
Download
0
Embed Size (px)
Citation preview
CS4723 Lecture 3Unit Testing
2
Unit testing
Testing of an basic module of the software A function, a class, a component
Typical problems revealed Local data structures
Algorithms
Boundary conditions
Error handling
3
Unit test framework
xUnit Created by Kent Beck in 1989
This is the same guy we mentioned in XP, design patterns
The first one was sUnit (for smalltalk)
Junit The most popular xUnit framework
There are about 70 xUnit frameworks for corresponding languages
Never in the annals of software engineering was so much owed by so many to so few lines of code
-------Martin Fowler
4
An example: writing test cases without a test framework
Class Definition Consider a class that handles and manages a Fibonacci
array
Initialized the class Fibonacci f = new Fibonacci ();
Extend to a certain length f. extend (length)
Provide the number at certain index f. get(index)
Get Length f.getLength()
Provide a range of numbers f. getRange(start, end)
5
An example: writing automatic test cases without a test framework
public static void main(String args[]){ Fibonacci f = new Fibonacci(); if(f.getLenght != 0){ System.err.print(“wrong results …”); } f.extend(50); if(f.getLenght != 0){ System.err.print(“wrong results …”); } int num = f.get(50); if(num != 12586269025){ System.err.print(“wrong results …”); } int[] range = f.getRange(40, 50) if(!ValueEquals(range, …)){ System.err.print(“wrong results …”); } …}
Not Enough!What happened if
an exception is thrown
6
An example: writing automatic test cases without a test framework
public static void main(String args[]){ Fibonacci f = new Fibonacci(); if(f.getLenght != 0){ System.err.print(“wrong results …”); } f.extend(50); if(f.getLenght != 0){ System.err.print(“wrong results …”); } int num = f.get(50); if(num != 12586269025){ System.err.print(“wrong results …”); } int[] range = f.getRange(40, 50) if(!ValueEquals(range, …)){ System.err.print(“wrong results …”); } …}
Potential test interference!
Consider if the get method does an extension when length is smaller than index:
while in getRange, such extension is forgotten to be added
7
Common Operations
Comparison between actual results and expected results
Report error messages
Catch Exceptions
Re-initialize the class to be tested
8
Unit Test Framework
System under test: the system/module/component we are testing
Test Fixture: SUT + DOC
Test Method: The actual code of the test
Test Case: A collection of tests with common purpose/setup
9
Writing a Test Casepublic class FibonacciTest extends TestCase { protected Fibonacci fTest; protected static int groudtruth[] = {0, 1,...,12586269025}; public void setUp(){ fTest = new Fibonacci(); } @Test public testInitialize(){ assertEquals(“Length of fTest after initialization”, 0, fTest.getLength()); } @Test public testExtend(){ fTest.extend(50); assertEquals(“…”, 50, fTest.getLength()); } @Test public testGet(){ fTest.extend(50); assertEquals(“…”, groudtruth[49], fTest.get(50)); } …}
10
Benefits of using test framework
Clear structure of testing Each test method for one feature
Common initialization to setup
Assertion methods You can always define new assertion methods
Reduced Interference Try catch to run all test methods
Always do re-initialization in setup
11
Tips of writing test cases
Bottom Up If you know that obj.m1 calls obj.m2, test obj.m2 first
Why? You may not understand the error in m1 if you don’t
know whether m2 is working or not
Externalize data Externalize expected results to files and load them at
the test case constructor
Why? Easier to maintain if you want to reuse them in several
places
12
Tips of writing test cases
Use development Database Do not use real database for testing
Why? Obvious
Do enough logging in the code during testing Why?
Help debugging
Testing for exceptions How?
public testException(){
try{
do(invalid); fail();
}catch(xxxException e){}
}
13
Writing Assertions
public void testCapacity() { // a test method
….
assertTrue(fFull.size() == 100+size); //assertion
}
If assertion fails:Assertion failed: myTest.java:150 (expected true but was
false)Not so good!
Try: assertEquals(100+size, fFull.size()); //expected value firstAssertion failed: myTest.java:150 (expected 102 but was
103)Better!
Try: assertEquals(“list length”, 100+size, fFull.size()); Assertion failed: myTest.java:150 (list length expected 102
but was 103)
14
Assertions
Extend TestCase and write your own assertions AssertStringContains, AssertArrayEquals, …
Use fail(String) to fail a test with certain message
Feel free to change the fixture Each test will re-setup
public void testRemoveAll() {
fFull.removeAllElements();
fEmpty.removeAllElements();
assertTrue(fFull.isEmpty());
assertTrue(fEmpty.isEmpty());
}
15
State machine of JUnit
Initialization Of Test Class
setUptry{ testMethod} catch {…}
tearDown
Tear down Consider the following test code
void setUp() { File f = open(“foo”); File b = open(“bar”);}void testAAA() { use f and b}void testBBB(){ use f and b}
Problems?
void setUp() { File f = open(“foo”); File b = open(“bar”);}void testAAA() { try { use f and b } finally { clean&close f, b }}void testBBB() { try { use f and b } finally { clean&close f, b }}
Better?
17
Tear down Consider the following test code
void setUp() { File f = open(“foo”); File b = open(“bar”);}void testAAA() { use f and b}void testBBB(){ use f and b}void tearDown{ clean&close f, b}
Problems?
void setUp() { File f = open(“foo”); File b = open(“bar”);}…void tearDown{ try{ clean&close f, b }catch{ … }}
void setUp() { File f = open(“foo”); File b = open(“bar”);}…void tearDown{ try{ clean&close f }catch{ … } the same for b}
Better?
18
Tear down
Do some cleanup job after the test is executed Close a file
Clear a global data structure
Revert changes to the database
Close network connections
…
19
Tear down
Be careful about tear down If tear down is not complete, a test failure may affect
the following test cases
Recover the changes done to global data that are not well handled by the setup Database, files, network, global variables
Clean resources
Caution of exceptions in tear down itself
20
The Practical Difficulty in Unit Testing
Decoupling It is rare that you are writing test cases for a class
with no dependencies (e.g., fibonacci array)
Dependencies are everywhere Complex structures System calls Databases File systems Network …
21
Problem of dependencies
Maintenance during setUp and tearDown
Expensive to setUp Network, loading from file, start a database…
Hard to find bugs Bugs may be related to the dependencies
They are still bugs, but should not be considered until system testing
22
Handling dependencies
Explicitly show dependencies Hidden dependencies are dangerous
Dependency injection
Remove unnecessary dependencies Reasonable design
Double dependencies When dependency is large or not ready yet
Use mock object
23
A story from google
A new developer go to a group and want to try on some methods
testCreditCardCharge() { CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);}
This code:Only works when you run as part of the suite.When run in isolation, throws NullPointerException.When you get your credit card bill, you are out $100 for every time the test runs.
24
A story from google
After a lot of digging, you learn that you need to initialize the CreditCardProcessor.testCreditCardCharge() { CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);}
Still not working
25
A story from google
After a lot of digging, you learn that you need to initialize the OfflineQueue.testCreditCardCharge() { OfflineQueue.init(); CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);}
Still not working
26
A story from google
After a lot of digging, you learn that you need to initialize the Database.testCreditCardCharge() { Database.init(); OfflineQueue.init(); CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);}
But sometimes you have to find out the correct sequence!
27
Solution with dependency injection
testCreditCardCharge() { Database db = Database(); OfflineQueue q = OfflineQueue(db); CreditCardProcessor ccp = new CreditCardProcessor(q); CreditCard c = new CreditCard("1234 5678 9012 3456", 5, 2008); c.charge(ccp, 100);}
28
Lessons Learned
Always try to make dependencies explicitly shown
Never do testing with real credit card number,
Or at least try $1 first
29
Remove Dependencies Thin method interfaces
Example: Get the total rent of a car
getRent(car, customer) -> getRent(car.getDays(), car.getPrice(), customer.getDiscount())
Duplicate common dependencies Example:
The class for Graduate students and under-graduate students both depend on a calendar module
It says, “if(student.isGraduate){…}else{…}” Split it to two parts and put them into the class for
both types of students
30
Double Dependencies Removing Dependencies is not always doable
Lots of necessary dependencies
It is mostly something in design, so maybe unchangeable at the testing time
It is not always good to remove all unnecessary dependencies increase maintenance effort reduce the benefit of software reuse
make code more complex and hard to read
31
Double Dependencies
Solution: Double it!
Double: highly trained replacement, vaguely resembling the actor No need to be a good actor
May require different skills for different cases
32
Types of test doubles
Dummies
Test stubs
Fake objects
Mock objects
33
Dummies
Simplest test double No behavior, just serve as an argument
Null can be viewed as a simple double
Examplepublic class Order{ private Shop sp; public Order (Shop sp){ this.sp = sp; } public Shop getShop(){ return sp; } …}
public class ShopDummy extends Shop{ ...}
public class OrderTest{ @Test public void testInit(){ Order o = new order(new ShopDummy()); ... }}
When to use dummies
SUT does not invoke any method on the dependency
The dependency class is an interface or abstract class with no implementation
or
The dependency class is hard to initialize so that you can put the initialization in the dummy once E.g., Shop sp = new Shop(new A(…), new B(…), new
C(…), new D(…), new E(…));
Can put the whole thing in ShopDummy
35
Test Stubs
More powerful than dummies
Provide a fix value or fixed behavior for a certain method invocation
Example: Always return 0 for a integer method
Do nothing for a void method
36
Hard-coded Test Stubs
The value or behavior is hard coded in the Stub Class
public class OrderTest{ @Test public void test(){ Order o = new order(new ShopStub()); o.add(1122, 3); ... AssertEquals(expect, o.getTotal()); o.save(); }}
public class ShopStub extends Shop{ public void save(Order o){ } public double getShopDiscount(){ return 0.9; }}
37
Configurable Test Stubs You may set different values for different test
casespublic class ShopStub extends Shop{ private Exception saveExc; private discount; public setException(Exception e){ this.saveExc = e; } public setDicount(Float f){ this.discount = f; } public void save(Order o){ if(this.saveExc!=null){throw saveExc;} } public double getShopDiscount(){ return 0.9; }}
public class OrderTest{ @Test public void testAbnormalDiscount(){ ShopStub stub = new ShopStub(); stub.setDiscount(1.1); Order o = new order(stub); o.add(1122, 3); ... AssertEquals(expect, o.getTotal()); o.save(); }}
Fake Objects
More powerful than stubs
A simplified implementation of the DOC Example: a data table to fake a database
Example: use a greed algorithm to fake a complex optimized algorithm
Guidelines Slow -> Fast
Complex -> Simple
Fake Objects
Tips for fake objects As simple as possible (as long as not too time-
consuming)
Go to a higher level if some object is hard to fake Example:
URLStatus sts = HttpConnection.open("http://api.dropbox.com/files/myfile");if(sts.status == 200){ return sts.data;}else{ return “Error”;}
Need to double1. Difficult to reproduce2. Maybe slow3. Affected by lots of
factors
Fake Objects Example (1) Fake HttpConnection
We need to fake URLStatus also
public class FakeUrlStatus{ public FakeUrlStatus(int status, string data){...}; public int getStatus(){...}; public String getData(){...};}
public class FakeDropBoxApi{ private files = { }; public FakeUrlStatus read(fname){ if(files.contain(fname)){ return FakeUrlStatus(200, files[fname]); }else{ return FakeUrlStatus(-1, "Error"); } }}
FakeURLStatus sts = FakeDropBoxApi.read("myfile");if(sts.status == 200){ …
Fake Objects Example (2) Fake the HttpConnection + URLStatus
public class FakeDropBoxHigherApi{ private files = { }; public String read(fname){ if(files.contain(fname)){ return files[fname]; }else{ return "Error"; } }}
return FakeDropBoxHigherApi.read("myfile");
Fake Objects
Choose a reasonable abstraction level to do the faking Do not need to fake lots of dependencies (put them in
one box)
Write simpler code (less potential bugs)
43
Mock objects
Mock objects is not similar to any of the above For the ability to imitate DOC, it is similar to
configurable stubs
But it does some different things
Mock objects do behavior-based testing Usually we only check return values or status
AssertEquals (expected, actual); AssertEquals (expected, array.length);
Can we do something like this? Why? Assert ( testObject.f1 calls DOC.f)
44
Mock objects
Go back to the order example
44
public class OrderTest{ @Test public void test(){ Order o = new order(new ShopStub()); o.add(1122, 3); ... AssertEquals(expect, o.getTotal()); o.save(); }}
public class ShopStub extends Shop{ public void save(Order o){ } public double getShopDiscount(){ return 0.9; }}
Problems???
Mock objects
Do behavioral assertions for SUT Only cares about whether the unit to be tested is
correct
Do not care about the correctness of DOC
Test Code
SUT DOC
DON’TCARE!!
Mock objects
Example EasyMock
@Testpublic void testOrder() { Order o = new Order(sp); o.add(1234, 1); o.add(4321, 3);
//initialize Shop sp = EasyMock.CreateMock(Shop.class);
//record EasyMock.expect(sp.getDiscount()).andReturn(0.9); EasyMock.expect(sp.save(o));
//replay EasyMock.replay(sp); AssertEquals(expect, o.getTotal()); o.Save(); EasyMock.verify(sp)}
47
Mock objects
Procedure
I am a mock object, just generated
Record:What I am expected to be invoked, and what should I returnExpect Methods
Replay:See how SUT invoked meSUT
Verification Results
48
Verification
Verifies whether the expected methods are actually invoked Exception: missing, expected save(0xaaaa)
Verifies whether the expected methods are invoked in an expected way Exception: save(null) expected save(0xaaaa)
49
Richer ways to do expectation
Different frameworks support it in different ways
Easymock: isA: ignore the value of argument
EasyMock.expect(sp.save(isA(Order.class)))
Find: expect the argument to contain a certain substring EasyMock.expect(mock.call(find(“pattern”)))
Geq: expect a number larger than the given value EasyMock.expect(mock.call(Geq(1000)))
50
Orders of expected method calls
Different frameworks support it in different ways
Easymock Order is not checked by default createStrictMock to check order
Mockito Order is not checked by default Create a InOrder object at the verification phase and add
the expected invocations that should happen in order
jMock More complex order control
51
Fluent Mocking
Record phase: Read expectation as specifications
Instead of directly generating code, the mock object generates an internal presentation, e.g. Automaton
Replay phase: Check the real invocations with the internal
presentation
52
Fluent Mocking: jMock
53
Types of test doubles
Dummies Simplest, only for arguments
Test stubs Return fixed value
Can be configurable at runtime
Fake objects A simplified implementation
Mock objects Similar to configurable stubs
Behavior checking
Automatic framework support
54
Review of Unit Testing
Using unit test framework to do unit testing It does all the common things, reduce test
interference
Inside test framework setUp -> test -> assert -> tearDown
Writing informative assertions
Writing complete tear downs
Dependencies are evil! Dependency injection
Remove dependencies
Test doubles
55
Announcement of Assignment I
Task Writing unit test cases for a real-world Java class
One of two classes from the Apache project org.apache.commons.email.Email org.apache.commons.imaging.ImageParser
Write test method for only the listed 12 methods
Work individually
56
Announcement of Assignment I
Deliverable Send the test code, and required dependencies to my
email account
Dependencies include JUnit Mock Framework if used The tested code and required dependencies
Comment your test code properly
Due Feb 28th 2014
57
Announcement of Assignment I
Evaluation (20 points total) Test code compiles and has no runtime error (failing
test cases are not counted as runtime errors) – 8 points
Proper setUp, tearDown, and assertions – 4 points
Proper test doubles – 4 points
Good test coverage – 4 points
58
Demo
Writing a JUnit testcase for a class in your project
Run it as a JUnit test case