CAPP: Change-Aware Preemption Prioritization

CAPP: Change-Aware Preemption Prioritization

Vilas Jagannath, Qingzhou Luo, Darko Marinov

Sep 6th 2011

Motivation• Parallel code required for performance

– Dominant paradigm is shared-memory, multithreaded code

• Difficult to develop (correct) multithreaded code– Different behavior under different schedules– Afflicted by bugs like data races, deadlocks, atomicity violations…

• Difficult to test multithreaded code– Exploration required, time consuming– Current exploration techniques focus on one code version– Code evolves, need efficient regression testing

• CAPP: Change-Aware Preemption Prioritization2

@Testpublic void testFilterWriteThreadSafety() throws InterruptedException { … FilterWriteThread fwThread1 = new FilterWriteThread(); FilterWriteThread fwThread2 = new FilterWriteThread(); fwThread1.start(); fwThread2.start(); fwThread1.join(); fwThread2.join(); assertEquals(0, fwThread1.result); assertEquals(0, fwThread2.result);}class FilterWriteThread extends Thread { int result = 0; ProtocolCodecFilter pc; … public void run() { try { pc.filterWrite(nextFilter, session, writeRequest); … } catch (Exception e) { e.printStackTrace(); result = 1;} } }

Apache Mina Test for filterWrite()

3

Exploration

4

S0

S1

T1

S2

T1

S3

T1

S..

T2

S..

T1

S..

T2

Schedule

Preemption

S..

T2

Transition

(S0, T1) (S0, T2)

Multithreaded Testing

5

Code

Test

Test execution explores many schedules

Work Pool

(S1, T2)

Exploration Order

6

(S0, T1) (S0, T2)

S1

T1

(S1, T1)

S2

T1

S0

Eg.DFS: StackBFS: Queue…

Current Work Pool

Exploration Prioritization

7

S1

T1

S2

T1

S0

Next Work Pool

(S0, T1) (S0, T2)

(S1, T2)(S1, T1) S..

T2

S..

T2

(S.., T2)

(S.., T1)….. …..

Eg.CHESSGambitActive TestingConTestAVIOCTrigger…

Regression Multithreaded Testing

8

Code

Test

Code`

Change-Unaware Prioritization

9

Code

Test

Code`

Change-Unaware Prioritization

10

Code

Test

Code`

CAPP:Change-Aware Preemption Prioritization

11

Code

Test

Code`

Use evolution information to prioritize exploration of schedules that perform preemptions at change impacted code.

CAPP is a general framework. We instantiate with 14 heuristics.

Apache Mina r912148 to r912149 public void filterWrite(NextFilter nextFilter, IoSession session, WriteRequest writeRequest) throws Exception { … flushWithoutFuture(); …}public void flushWithoutFuture() { Queue<Object> bufferQueue = getMessageQueue(); for (;;) { Object encodedMessage = bufferQueue.poll(); encodedMessage == null) { break; }if (// Flush only when the buffer has remaining. if (!(encodedMessage instanceof IoBuffer) || ((IoBuffer) encodedMessage).hasRemaining()) { …} } }

12

public void filterWrite(NextFilter nextFilter, IoSession session, WriteRequest writeRequest) throws Exception { … Queue<Object> bufferQueue = getMessageQueue(); while (!bufferQueue.isEmpty()) { Object encodedMessage = bufferQueue.poll(); // Flush only when the buffer has remaining. if (!(encodedMessage instanceof IoBuffer) || ((IoBuffer) encodedMessage).hasRemaining()) { … } } …

• DIRMINA-803• 58 schedules to detect failure using CHESS• 5 schedules using one of the CAPP heuristics

CAPP Parts

Impacted Code Points (ICP) Collection(Static)

Change-Aware Exploration Prioritization(Dynamic)

13

Impacted Code Points (ICP) Collection

14

Code Code`

• Inspect structure diff to collect impacted:– Classes, Methods, Lines, Fields

• Focus on changes to thread communication:– Expand synchronized blocks, methods– Fields accessed by changed lines– Fields accessed in methods called by changed lines

Current Work Pool

Change-Aware Exploration Prioritization

15

S1

T2

S2

T1

S0

Next Work Pool

(S0, T1) (S0, T2)

(S1, T2)(S1, T1)

Use ICPs to partition

Use ICPs to partition

Family of Heuristics• Defined by ICP Match Mode and Prioritization Mode

• ICP Match Mode: Is transition impacted by changes?– Class (on Stack), Method (on Stack), Line (on Stack), Field

• Prioritization Mode: Which work pool to add transitions to?– All, Some

• 14 heuristics– CA (Class All)– …– MOS (Method on Stack Some)…

– …

16

ICP Match Mode• ICP Match Mode: Is transition impacted by changes?

– Class (on Stack), Method (on Stack), Line (on Stack), Field

17

S..

ICPs: {………, class A,………….....}

(S.., T1) (S.., T2)

T1: Executing class A, method m, line 55 T2: Executing class B, method k, line 32 Class

(S.., T1) (S.., T2)(S.., T1) (S.., T2)

Prioritization Mode

18

Current Work Pool

S..

Next Work Pool

(S.., T1)

• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted

(S.., T2) (S.., T3)

Prioritization Mode

19

Current Work Pool

S..

Next Work Pool

• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted

All (S.., T1) (S.., T2) (S.., T3)

Prioritization Mode

20

Current Work Pool

S..

Next Work Pool

• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted

All (S.., T1) (S.., T2) (S.., T3)

Prioritization Mode

21

Current Work Pool

S..

Next Work Pool

• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted

Some (S.., T1) (S.., T2) (S.., T3)

Prioritization Mode

22

Current Work Pool

S..

Next Work Pool

• Prioritization Mode: Which work pool to add transitions to?– All: Only prioritize preemptions among impacted transitions– Some: Prioritize preemptions among impacted and not-impacted

Some (S.., T1) (S.., T2) (S.., T3)

T1

Implementations• Two implementations• Java PathFinder (JPF)

– Stateful exploration using checkpoints– Depth first, unbounded– CAPP implemented using a custom SimplePrioritySearch

• ReEx– Stateless exploration using re-execution– CHESS (iterative context bound 2)– CAPP implemented using a custom SchedulingStrategy

23

Evaluation Subjects

• 15 multithreaded programs:– Ranging from 52-54,000 loc, with 2-607 classes, running 2-9 threads– Evolution from non-buggy version to buggy version

• 7 deadlock faults, 8 data race and atomicity violation faults24

Subject SLOC Threads ICPsAirline 136 6 6Allocation 209 3 5BoundedBuffer 110 9 2BubbleSort 89 4 4Deadlock 52 3 3ReadersWriters 154 5 2ReplWorkers 432 3 2RAXextended 166 6 2Groovy 54,872 3 60Lang 48,369 3 3Mina 34,804 3 36Pool1 10,042 3 148Pool2 4,473 3 201Pool3 10,802 2 29Pool4 10,783 3 47

Experiments• Independent variables:

– 14 heuristics

– Stateful vs. Stateless exploration

– Default exploration order vs. 50 Random exploration orders

• 19,890 explorations, around a month of computing time

• Measured savings in exploration cost compared to change-unaware exploration:– Transitions for JPF implementation

– Schedules for ReEx implementation

25

Default Non-Randomized Exploration

26

CA CS COA COS MA MS MOAMOS LA LS LOA LOS FA FS0

1

2

3

4

5

6StatefullStateless

Heuristics

Ave

rage

Exp

lora

tion

Red

uctio

n

5.3x

2.7x0.8x

CAPP• Current multithreaded testing techniques focus on one code

version

• Code evolves, need techniques that leverage evolution info

• CAPP uses evolution info to prioritize exploration:– Up to 5.3x average reduction in exploration cost

• Ongoing work:– Regression Schedule Selection [ICST 10, STVR in

revision]– Schedule Expression [ESEC/FSE 11]– Multithreaded Tests Generation

27

Possible Project Ideas

28

• Better impact analysis

• Combine selection and prioritization (RMC)

• Prioritize across tests

• Multiple faults

• Improve ReEx, revisit stateless randomized search (PRSS)• Combine IMUnit with CAPP

– How to find/generate/simplify buggy IMUnit schedules?– How to decide whether some IMUnit schedules is valid or not

after code evolves (a.k.a., identify bad concurrency bug fixes)?

UPCRC

29

• Universal Parallel Computing Research Center– Funded by Intel and Microsoft– Goal: improve parallelism for client computation– Two centers: Berkeley and Illinois

• UPCRC Illinois– About 20 faculty + 40 grad students– Research on entire computation stack: architecture, runtime

systems, compilers, programming languages, design patterns, applications… and testing

Results Summary• Does CAPP help?

– Up to 5.3x reduction on average– Only one heuristic (out of 28) increased cost (0.8x)

• How do heuristics compare?– Field ICP Match Mode heuristics perform best– ALL Prioritization Mode heuristics perform best

• Stateful vs. Stateless?– Greater benefit for stateless exploration

• Default vs. Random?– Conclusions from random results similar to default results– Except ICP Match Mode

• Detailed results in the paper30

Multicore World

31

Performance!

Shared Memory

Multithreaded

Difficult to Develop Multithreaded Code

• Scheduling non-determinism• Data races• Deadlocks• Atomicity violations• …

32

Correct

Shared Memory

Multithreaded

Difficult to Test Multithreaded Code• Exploration required• Time consuming• Current techniques focus

on one code version• Code evolves• Need efficient regression

testing

33

Code

Test