Embed Size (px)
DESCRIPTION
cloud technology
Citation preview
CLOUD9: A SOFTWARE TESTING SERVICE
Liviu Ciortea, Cristian Zam!r,Stefan Bucur, Vitaly Chipounov, George Candea
SOFTWARE TESTING
• Software testing is laborious and expensive
• Bugs are still very common
• Human testing is prone to errors
• Current automatic test case generation is limited
2
GOALS
• Autonomy: Minimize intervention in test generation
• Usability: Minimize con!guration effort
• Performance: Maximize results relevance
3
OVERVIEW
• System Interface
• Parallel Symbolic Execution
• Cloud9 Design
• Preliminary Results
4
WHAT IS CLOUD9?
• Web service for automated testing
• Easy to use interface
• Relies on thorough testing technique
• Can operate autonomously
• Massive parallelization in the cloud
• Brings scalable performance5
6
SERVICE INTERFACE
7
8
OVERVIEW
• System Interface
• Parallel Symbolic Execution
• Cloud9 Design
• Preliminary Results
9
SYMBOLIC EXECUTIONvoid read(int x) {
}
if (x < 0) {
} else {if (x < 5)bar(x);
else {...
}}
if (x > -3)foo(x);
else {...
} x = -2
Concrete value:
10
SYMBOLIC EXECUTIONvoid read(int x) {
}
if (x < 0) {
} else {
}
if (x > -3)foo(x);
else {...
} x = 3
Concrete value:
if (x < 5)bar(x);
else {...
}
11
SYMBOLIC EXECUTIONvoid read(int x) {
}
if (x < 0) {
} else {
}
if (x > -3)foo(x);
else {...
}
Symbolic execution tree:
if (x < 5)bar(x);
else {...
}
λ
λ<0 λ>=0
λ<0&& λ>-3
λ>=0&& λ<5
Symbolic value
Constraint
λ<0 && λ>-3 Constraintsolver λ=-2 (a solution)
12
PATH EXPLOSION
Large memory consumptionCPU Intensive
Large memory consumptionCPU Intensive
We massively parallelize in the cloud
CHALLENGES
• Tree structure is not known a priori
15
CHALLENGES
• Tree structure is not known a priori
• Naive approach: pre-allocate workers equally on the tree
15
CHALLENGES
• Tree structure is not known a priori
• Naive approach: pre-allocate workers equally on the tree
• Slightly better: examine the !rst h levels, then decide work allocation
h levels
15
CHALLENGES
• State transfer
• Avoid work and memory redundancy
• Coordination
16
OVERVIEW
• System Interface
• Parallel Symbolic Execution
• Cloud9 Design
• Preliminary Results
17
TREE EXPLORATION
Explorationfrontier
Exploration strategy = which node to expand next?
w1
w2
18
TREE EXPLORATION
Explorationfrontier
Exploration strategy = which node to expand next?
w1
w2
18
LOAD BALANCING
w1 w2
LB
19
LOAD BALANCING
w1 w2
LB
19
negotiate
LOAD BALANCING
w1 w2
LB
19
STATE TRANSFER DECISIONS• State copying vs. state reconstruction
• Reconstruction optimizations
Copying(Network intensive)
Reconstruction(CPU intensive)
c1&&(c2||c3)&&...
w1 w2 c1&&(c2||c3)&&...
w1 w2
10010...
1
00
1
0
20
STRATEGY PORTFOLIO
• No “one size !ts all” exploration strategy
• Different workers with different strategies
• Invest in few workers, then select successful methods
21
OVERVIEW
• System Interface
• Parallel Symbolic Execution
• Cloud9 Design
• Preliminary Results
22
CLOUD9 PROTOTYPE
• We built Cloud9 on KLEE
• State-of-the art sequential symbolic execution engine
• Tested real programs and found bugs
• Use Amazon EC2 as cloud computing platform
23
TESTING METHODOLOGY
• We compare with KLEE for testing Coreutils
• ls, cat, chmod, cp, mv, etc.
• Cloud9 and KLEE run for 1 hour
• 16 workers for Cloud9
24
CLOUD9 SPEEDUPFix code coverageand measure time
(Linear) 16 50
100
150
200
250
echojoinheadln cp ptx
du chgrp
cutdatecksum
ls pr commcsplit
fmtod factor
chmod
printf
basename
chown
expand
mv catnl dircolors
readlink
paste
chcon
rm mktemp
fold
Speedup(tKLEE/tCloud9)
Tool25
CODE COVERAGE
26
Fix resources (CPU time)and measure code coverage
0
20
40
60
80
100
ptxls chcon
cp headdate
joinpr fmt
du catfold
dircolors
csplit
echood chmod
expand
mv ln nl paste
cutcksum
chgrp
commrm chown
factor
basename
printf
mktemp
readlink
Cove
rage
(%)
Tool
Cloud9 (1h x 16w)KLEE (16h)
CONCLUSIONSAutonomy:
• Symbolic execution
Usability:
• Web service interface
• No local setup overhead
Performance: up to 250x speedup
• Parallel symbolic execution
• Dynamic load balancing
• Adaptive state transfer
• Strategy portfolio
✓
✓
✓
27
