Upload
leon-underwood
View
236
Download
4
Tags:
Embed Size (px)
Citation preview
CHESS: A Systematic Testing Tool for Concurrent Software
CSCI6900
George
Outline Introduction
Using CHESS
Capturing nondeterminism
Exploring nondeterminism
Evaluation
Related work
Introduction Concurrent programming is difficult
Testing concurrent program is difficult Famous concurrency bug: “Heisenbugs”
Most strategy used in finding concurrency bugs Stress testing
Introduction What is “Concurrency testing”
Concurrency scenario testing
Model checking techniques
Three key challengers by applying model checking
Perturbation problem Rich and complex concurrency APIs in system State-space explosion.
Introduction What is “CHESS”
Concurrency scenario testing
Model checking techniques
Scale to large concurrent program
Reproduce an erroneous execution
Been integrated into the test framework inside Microsoft
Introduction How does CHESS solve the previous key
problems?
Testing methodology of CHESS reduce the perturbation
Only one perturbation in CHESS
Wrappers to provides enough hooks
Variety of techniques to address the state-explosion
Contributions of this paper The first system for integrating model
checking into concurrent systems and test frameworks with minimal perturbation;
Techniques for systematic exploration of systems code for fine-grained concurrency with shared memory and multithreading;
Contributions of this paper Validation of the CHESS tool and its
accompany testing and wrapper methodology on three different platforms;
A substantial number of previously unknown bugs, even in well-tested systems;
The ability to consistently reproduce crashing
bugs with unknown cause.
Using CHESS Traditional testing methodology
Disadvantages of this testing methodology
Using CHESS CHESS architecture
Using CHESS Testing methodology in CHESS
Using CHESS To guarantee the previous advantages, CHESS
has to control the scheduling of the tasks
Two ways to control it
Modify the scheduler
Severely limited the deployment
Using CHESS Testing methodology in CHESS
Using CHESS Two assumption of about the testing program
Quiescence Definition of “Quiescence State”
How does CHESS handle “Quiescence State”
Capturing nondeterminism Tasks in CHESS
Threads Timers Asynchronous I/Os
Wrappers in CHESS Goal of wrappers
Capturing nondeterminism Design of Wrappers in CHESS
Capturing nondeterminism Synchronization wrappers
Capturing nondeterminism Synchronization wrappers
Capturing nondeterminism Hooking the wrappers
use various mechanisms to dynamically intercept calls to the real API functions and forward them to the wrappers.
Programs in this paper Win32 .NET Singularity
Exploring nondeterminism How CHESS systematically drives the test
Basic scheduler operation Allows only one thread to execute at a time
Repeatedly executes the same test driving each iteration of the test through a different schedule
Three phases in each iteration
Exploring nondeterminism Replay phase
Replays a sequence of scheduling choices from a trace file
Record phase Schedules a thread till the thread yields the
processor
Search phase Uses the enabled information at each schedule
point to determine the scheduler for the next iteration
Exploring nondeterminism Imperfect replay
Not rely on perfect replay capability
Common sources of nondeterminism Lazy-initialization
Interference from environment
Nondeterministic calls
Exploring nondeterminism Ensuring fair schedules
Can not enumerate all fair schedules
State-explosion Definition of State-explosion
Inserting preemptions
Capturing states
Evaluation Systems on which CHESS has been run on
Evaluation Test scenarios and findings
Evaluation Validation CHESS against stress-testing
Common objection of the CHESS
Failure and bug in CHESS
Succeeded in reproducing every stress-test failure
Evaluation Description of two bugs
PLINQ bug
Evaluation Description of two bugs
Singularity bug
Related work Repeatable deterministic testing
Systematic generation of thread schedules
Applying state exploration directly to executing concurrent programs
Replay a concurrent execution