Test-First Java Concurrency for the Classroom
SIGCSE 2010
Mathias Ricken and Robert Cartwright
Rice University
March 12, 2009
2
Two Trends
Test-driven development Concurrent programming
Brian Goetz, Java Concurrency in Practice, Addison-Wesley, 2006
3
Unit Testing Benefits
• Occurs early
• Automates testing
• Keeps the shared repository clean
• Prevents bugs from reoccurring
• Allows safe refactoring
• Serves as documentation
4
Unit Testing in Assignments
• Hand out test cases to students– Improves confidence and understanding
• Instill good practices– Require students to extend test suites
• Automated grading– Part graded automatically, part by hand
5
Moore’s Law Requires Concurrency
Adopted fromSutter 2009
6
Concurrency Is Difficult
Unit testing not effective in multi-threaded programs
7
Existing Unit Testing Frameworks
• JUnit, TestNG
• Don’t detect test failures in threads other than main thread– Failures in event thread not detected either
• Don’t ensure that other threads terminate
• Tests that should fail may succeed
8
Sample JUnit Tests
public class SimpleTest extends TestCase { public void testException() { throw new RuntimeException("booh!"); } public void testAssertion() { assertEquals(0, 1); }}
if (0!=1) throw new AssertionFailedError();
}} Both tests fail.
Both tests fail.
9
JUnit Test with Child Thread
public class SimpleTest extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); }}
new Thread() { public void run() { throw new RuntimeException("booh!"); }}.start();
throw new RuntimeException("booh!");
Main thread
Child thread
Main thread
Child thread
spawns
uncaught!
end of test
success!
Uncaught exception, test should fail but
does not!
10
ConcJUnit
• Backward compatible replacement for JUnit
• Detects exceptions in all threads– Exception handler for all child threads and the
event thread
• Ensures that child threads have terminated and event thread is done– Enumerate live threads after test– Inspect event queue
• Requires all child threads to be joined– Analyze join graph
11
Thread Creation Coordinates
• In Thread.start() record stack trace of Thread.currentThread()– Easy to find where a thread that caused a
failure was started
– Also shows where threads that outlived the test were started
12
Creation Coordinates Example
class Main { void foo() { // which one? new Helper(true).start(); new Helper(false).start(); // ... }}
AssertionError:at Helper.m(Helper.java:2)at Helper.run(Helper.java:3)
Started at:at Main.foo(Main.java:4)at Main.bar(Main.java:15)at Main.main(Main.java:25)
class Helper extends Thread { void m() { if (b) Assert.fail(); } public void run() { m(); } private boolean b; // …}
13
ConcJUnit Demo
14
Concurrency Examples
• In-class discussion– Multi-threaded counter: data races– Multi-threaded bank: deadlock
• Homework– Bounded buffer– Readers-writer lock– Test suite handed out to help students
• Multi-threaded Breakout
15
Example: Counter
• Class that can increment an integer variable N times
• Write test first
public class CounterTest extends TestCase { final long PER_THREAD = 1000000; public void testSingle() { Counter c = new Counter(); c.incrementNTimes(PER_THREAD); assertEquals(PER_THREAD, c.getCount()); }}
16
Counter: Implementation
• Write implementation
public class Counter { private long count = 0;
public long getCount() { return count; } public void incrementNTimes(long n) { for(long i=0; i<n; ++i) { ++count; } }}
Test passes!
17
Counter: Multi-threaded Test
• Write multi-threaded test
public void testMulti() { final Counter c = new Counter(); for(int i=0; i<NUM_THREADS; ++i) { new Thread() { public void run() { c.incrementNTimes(PER_THREAD); } }.start(); } TestUtils.waitForOtherThreads(); assertEquals(NUM_THREADS*PER_THREAD,c.getCount()); }
Test fails (most likely)!
18
Shared Data• Why does the multi-threaded counter test fail?
– The count field is shared among threads– The ++count operation is not atomic– Thread may be interrupted after reading count, but
before writing back to count
count=0 regA=? regB=?A1 regA = count; 0 0 ? B1 regB = count; 0 0 0A2 regA = regA + 1; 0 1 0A3 count = regA; 1 1 0 B2 regB = regB + 1; 1 1 1 B3 count = regB; 1 1 1
19
Data Races
• Definition– Two threads access the same data– At least one access is a write– Nothing prevents the order from changing
• Would like code to execute atomically (without interruption)– Java does not support atomicity
(for general code)
20
Java Locks & Synchronized
• Java provides “lock objects” and synchronized blocks
synchronized(lock) { ++count; }
– Thread must compete for ownership of lock object before entering synchronized block
– Synchronized block is not atomic– But once a thread has a lock object, no other
thread can execute code protected by the same lock object
21
Counter: Re-Write
• Rewrite implementation
// ... private Object lock = new Object(); public void incrementNTimes(long n) { for(long i=0; i<n; ++i) { synchronized(lock) { ++count; } } }
Test passes!
22
Concurrency Still Difficult
• Even race-free, deadlock-free programs are not deterministic– Thread scheduling is essentially non-
deterministic
• Different schedules may compute different results– May or may not be acceptable, depending
on the task
23
Multi-threaded Breakout• Uses ACM Java Task Force material
– Based on “Breakout - Nifty Assignment” by Eric Roberts, SIGCSE 2006
• Multiple balls, each in its own thread– Atomicity assumption when removing bricks– Ends game before all bricks are removed
• Other problems– X,Y coordinate changes not atomic– X,Y coordinates not volatile or synchronized, event
thread may never see the updates
• Correctly synchronized version still not deterministic
24
Future Work• Testing all schedules is intractable
• Insert random delays/yields before synchronization operations– Must consider volatile variable accesses to
comply with Java Memory Model– Re-run program several times– Can detect a number of sample problems
• Record schedule, replay if test fails– Makes failures reproducible if found
3
25
Conclusion
• Unit testing has important benefits in industry and in the classroom
• Concurrent programming is becoming more important, and it’s difficult
• ConcJUnit helps…
www.concutest.orgwww.drjava.org
Notes
27
Notes1. Also cannot detect uncaught exceptions in a
program’s uncaught exception handler (JLS limitation) ←
2. Only add edge if joined thread is really dead; do not add if join ended spuriously. ←
3. Have not studied probabilities or durations for sleeps/yields:One inserted delay may negatively impact a second inserted delayExample: If both notify() and wait() are delayed. ←
28
public class Test extends TestCase { public void testException() { Thread t = new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); } }); t.start(); while(t.isAlive()) { try { t.join(); } catch(InterruptedException ie) { } } }}
Thread t = new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); }});t.start();while(t.isAlive()) { try { t.join(); } catch(InterruptedException ie) { }}
throw new RuntimeException("booh!");
Loop since join() may end
spuriously
4. ←
Spurious Wakeup
Image Attribution
30
Image Attribution1. Left image on Two Trends:
Test Driven Development, Damian Cugley.
2. Right image on Two Trends: adapted from Brian Goetz et al. 2006, Addison Wesley.
3. Graph on Moore’s Law:Adapted from Herb Sutter 2009
4. Image on Concurrency Is Difficult:Caption Fridays
Extra Slides
32
Changes to JUnit (1 of 3)
• Thread group with exception handler– JUnit test runs in a separate thread, not main thread– Child threads are created in same thread group– When test ends, check if handler was invoked
Reasoning:• Uncaught exceptions in all threads must cause
failure
33
JUnit Test with Child Thread
public class Test extends TestCase { public void testException() { new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); } }).start(); }}
new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); }}).start();
throw new RuntimeException("booh!");
invokeschecks
TestGroup’s Uncaught Exception Handler
34
JUnit Test with Child Thread
public class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); }}
new Thread() { public void run() { throw new RuntimeException("booh!"); }}.start();
throw new RuntimeException("booh!");
Test thread
Child thread
uncaught!
end of test
Main thread
spawns and joins resumes
check exception handler
invokes exception handler
failure!
35
Child Thread Outlives Parent
public class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); }}
new Thread() { public void run() { throw new RuntimeException("booh!"); }}.start();
throw new RuntimeException("booh!");
Test thread
Child thread
uncaught!end of test
success!
invokes exception handler
Main thread
check exception handler
Too late!
36
Changes to JUnit (2 of 3)
• Check for living child threads after test ends
Reasoning:• Uncaught exceptions in all threads must cause
failure• If the test is declared a success before all child
threads have ended, failures may go unnoticed• Therefore, all child threads must terminate
before test ends
37
Check for Living Child Threads
public class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); }}
new Thread() { public void run() { throw new RuntimeException("booh!"); }}.start();
throw new RuntimeException("booh!");
Test thread
Child thread
uncaught!end of test
failure!
invokes group’s handler
Main thread
check for livingchild threads
check group’s handler
38
Correctly Written Test
public class Test extends TestCase { public void testException() { Thread t = new Thread() { public void run() { /* child thread */ } }; t.start(); t.join(); }}
Thread t = new Thread() { public void run() { /* child thread */ }};t.start();t.join(); // wait until child thread has ended
/* child thread */
Test thread
Child thread
end of test
success! Main thread
check for livingchild threads
check group’s handler
4
39
Changes to JUnit (3 of 3)
• Check if any child threads were not joined
Reasoning:• All child threads must terminate before test ends• Without join() operation, a test may get “lucky”• Require all child threads to be joined
40
Fork/Join Model
• Parent thread joins with each of its child threads
• May be too limited for a general-purpose programming language
Child thread 1
Child thread 2
Main thread
41
Example of Other Join Models
• Chain of child threads guaranteed to outlive parent
• Main thread joins with last thread of chain
Child thread 1
Child thread 2
Main thread
Child thread 3
42
Modifying the Java Runtime
• Changing Thread.start()and join()– Need to modify Java Runtime Library– Utility to process user’s rt.jar file– Put new jar file on boot classpath:-Xbootclasspath/p:newrt.jar
• Still works without modified Thread class– Just does not emit “lucky” warnings
43
Join with All Offspring Threads
• Main thread joins with all offspring threads, regardless of what thread spawned them
Child thread 1
Child thread 2
Main thread
44
Generalize to Join Graph
• Threads as nodes; edges to joined thread
• Test is well-formed as long as all threads are reachable from main thread
Child thread 1
Child thread 2
Main thread
Child thread 3
MT
CT1
CT2
CT3
45
Child thread 1
Child thread 2
Main thread MT
CT1
CT2
Child thread 1
Child thread 2
Main thread
MT
CT1
CT2
Join Graph Examples
46
Child thread 1
Child thread 2
Main thread
MT
CT1
CT2
Unreachable Nodes
• An unreachable node has not been joined– Child thread may outlive the test
47
childThread
main Thread
MT
CT
Constructing the Graph// in mainThreadchildThread.start();
• Add node for childThread
48
// in mainThreadchildThread.join();
• When leaving join(), add edge from mainThread to childThread
childThread
main Thread
MT
CT
Constructing the Graph
2
49
Example: Multi-threaded Bank
• Program simulating checking accounts• Account balances are shared data
– To avoid data races, use synchronized– Need access to two accounts for transfers
synchronized(locks[from]) { synchronized(locks[to]) { accounts[from] -= amount; accounts[to] += amount; }} Test hangs!
50
Deadlock• Thread A transfers from account 0 to 1• Thread B transfers from account 1 to 0• Thread A gets interrupted after acquiring locks[0]
// thread A // thread Bsynchronized(locks[0]) {
synchronized(locks[1]) { synchronized(locks[0]) // can’t continue, locks[0] // is owned by thread A */
synchronized(locks[1]) // can’t continue, locks[1] // is owned by thread B */
51
Lock Acquisition Order• No deadlock if both threads had attempted to
acquire lock 0 first
• When acquiring more than one lock object, always acquire them in the same order– e.g. acquire lower account’s lock object first
synchronized(locks[Math.min(from,to)]) { synchronized(locks[Math.max(from,to)]) { accounts[from] -= amount; accounts[to] += amount; }}
52
Homework Assignment
• Common structures students will see time and again– Bounded buffer– Readers-writer lock
• Grade correctness and efficiency, e.g.– Maximize concurrency– Only wake up as few threads as possible
• Provide students with test suites
53
Many Thanks To…• My advisor
– Corky Cartwright
• My committee members– Walid Taha– David Scott– Bill Scherer
• NSF and Texas ATP– For providing partial funding