22
Symbolic Execution with Mixed Concrete-Symbolic Solving Corina Pasareanu 1 , Neha Rungta 2 and Willem Visser 3 1 Carnegie Mellon, 2 SGT Inc./NASA Ames 3 University of Stellenbosch

Symbolic Execution with Mixed Concrete-Symbolic Solving Corina Pasareanu 1 , Neha Rungta 2 and Willem Visser 3 1 Carnegie Mellon, 2 SGT Inc./NASA Ames

  • Upload
    eze

  • View
    46

  • Download
    0

Embed Size (px)

DESCRIPTION

Symbolic Execution with Mixed Concrete-Symbolic Solving Corina Pasareanu 1 , Neha Rungta 2 and Willem Visser 3 1 Carnegie Mellon, 2 SGT Inc./NASA Ames 3 University of Stellenbosch. Symbolic Execution. Program analysis technique King [Comm. ACM 1976] , Clarke [IEEE TSE 1976 ] - PowerPoint PPT Presentation

Citation preview

Page 1: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Symbolic Execution with Mixed Concrete-Symbolic Solving

Corina Pasareanu1, Neha Rungta2 and Willem Visser31Carnegie Mellon, 2SGT Inc./NASA Ames

3University of Stellenbosch

Page 2: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Symbolic Execution Program analysis technique

King [Comm. ACM 1976] , Clarke [IEEE TSE 1976] Executes a program on symbolic inputs Maintains path condition (PC) – checked for satisfiablity with decision procedures

Received renewed interest in recent years due to Algorithmic advances Increased availability of computational power and decision procedures

Applications: Test-case generation, error detection, …

Tools, many open-source UIUC: CUTE, jCUTE, Stanford: EXE, KLEE, UC Berkeley: CREST, BitBlaze Microsoft’s Pex, SAGE, YOGI, PREfix NASA’s Symbolic (Java) Pathfinder IBM’s Apollo, Parasoft’s testing tools etc.

Page 3: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; }}

S0, S1, S3, S4 = statements we wish to cover

Symbolic Execution

Page 4: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; }}

Assume hash is native or can not be handled by decision procedure

S0, S1, S3, S4 = statements we wish to cover

Symbolic ExecutionCan not handle it!

Solution:Mixed concrete-symbolic solving

Page 5: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Mixed Concrete-Symbolic Solving

EXE results: stmt “S3” not covered DART results: path “S0;S4” not covered

Mixed concrete-symbolic solving: all paths coveredExample

Predicted path “S0;S4” != path taken “S1;S4”

//hash(x)=10*x

Page 6: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Mixed Concrete-Symbolic Solving

Use un-interpreted functions for external library calls

Split path condition PC into:simplePC – solvable constraintscomplexPC – non-linear constraints with un-

interpreted functionsSolve simplePC

Use obtained solutions to simplify complexPCCheck the result again for satisfiability

Page 7: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Mixed Concrete-Symbolic Solving Assume hash(x) = 10 *x:

PC: X>3 ∧ Y>10 ∧ Y=hash(X)

simplePC complexPC

Solve simplePCUse solution X=4 to compute h(4)=40Simplify complexPC: Y=40Solve again: simplified PC: X>3 ∧ Y>10 ∧ Y=40 Satisfiable!

Page 8: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; }}

int hash(x) { if (0<=x<=10) return x*10; else return 0;}

Symbolic Execution

PC: true

PC: X>0 PC: X<=0

PC: X>0 & Y=hash(X) S0

PC: X>3 & Y>10 & Y=hash(X) S3

PC: X>0 & X<=3 & Y=hash(X) S4

…Solve X>0hash(1)=10Check X>0 & Y=10

Solve X>3 & Y>10hash(4)=40Check X>3 & Y>10 & Y=40

Page 9: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Potential for Unsoundness

test (int x, int y) { if (x>=0 && x>y && y == x*x) S0; else S1;}

Not Reachable

PC: X>=0 & X > Y & Y = X*X S0

X=0, Y=-1 Y=0*0=0

X>=0 & X>Y Y = X*X simplePC complexPC Must add constraints on the solutions back into simplified PC

DART/Concolic will diverge instead

X>=0 & X>Y & Y=0 & X=0Not SAT!

Is SAT which impliesS0 is Reachable!

X>=0 & X>Y & Y=0simplified PC

Page 10: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Directed Automated Random Testing (DART) Godefroid, Klarlund and Sen 2005

or Concolic Execution

• Collects path conditions along concrete executions

• Negates constraints on the PC after a run and• Executes again with the newly found solutions• Can overcome the weaknesses of classic

symbolic execution

Page 11: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; }}

native int hash(x) { if (0<=x<=10)

return x*10; else return 0;}

test(1,0)

X>0 & Y!=10 & X>3

DART/Concolic Execution

X > 0

X > 0 & Y != 10 S1

X>0 & Y!=10 & X<=3 S4

test(4,0)X > 0

X > 0 & Y != 40 S1

X>0 & Y!=40 & X>3 & Y<= 10 S4

Page 12: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; }}

native int hash(x) { if (0<=x<=10)

return x*10; else return 0;}

X>0 & Y!=40 & X>3 & Y>10 X>0 & Y=40 & X>3 & Y>10

DART/Concolic Execution

test(4,11)X > 0

X > 0 & Y != 40 S1

X>0 & Y!=40 & X>3 & Y>10 S3

test(4,40)

X > 0

X > 0 & Y = 40 S0

X>0 & Y=40 & X>3 & Y>10 S3

Page 13: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; else S4; }}

native int hash(x) { if (0<=x<=10)

return x*10; else return 0;}

X>0 & Y=40 & X<=3 & Y>10

Divergence!

Aimed to get S0;S4But reached S1;S4

DART/Concolic Execution

test(1,40)X > 0

X > 0 & Y != 10 S1

X>0 & Y!=10 & X<=3 S4

Page 14: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Mixed Concrete-Symbolic Solvingvs DART

Both incompleteIncomparable in power (see paper)Mixed concrete-symbolic solving can handle only

“pure”, side-effect free functionsDART does not have the limitation; will likely diverge

Page 15: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Addressing Incompleteness: 3 Heuristics

Incremental Solving

User Annotations

Random Solving

Page 16: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (y > 10) S3; else S4; }}

int hash(x) { if (0<=x<=10) return x*10; else return 0;}

Incremental Solving

PC: true

PC: X>0 & Y>10 & Y=hash(X) S3

PC: X>0 PC: X<=0

PC: X>0 & Y=hash(X) S0

PC: X>0 & X<=3 & Y=hash(X) S4

…Solve X>0hash(1)=10Check X>0 & Y=10

Solve X>0 & Y>10Solution: X=1hash(1)=10Check X>0 & Y>10 & Y=10

Not SAT!

Solution: X=2hash(2)=20Check X>0 & Y>10 & Y=20

Get another solution:

SAT!

Page 17: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

@Partition({“x>3”,”x<=3”})void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (y > 10) S3; else S4; }}

int hash(x) { if (0<=x<=10) return x*10; else return 0;}

User Annotations

PC: true

PC: X>0 & Y>10 & Y=hash(X) S3

PC: X>0 PC: X<=0

PC: X>0 & Y=hash(X) S0

PC: X>0 & X<=3 & Y=hash(X) S4

…Solve X>0hash(1)=10Check X>0 & Y=10

Solve X>0 & Y>10 & X>3Hash(4)=40Check X>0 & Y>10 & Y=40 SAT!Add user partitions one at a time

Page 18: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Random Solving

• Pick solutions randomly from the solution space

• Current implementation only picks randomly if the solution space is completely unconstrained

Page 19: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Imple

mentat

ion

Java PathFinder

Symbolic PathFinder

SPF

Mixed Concrete-Symbolic Solving

Model Checker for JavaOpen Sourcehttp://babelfish.arc.nasa.gov/trac/jpf

Symbolic Execution Extension for JPF (jpf-symbc)

Custom Listeners on SPF

Experience TSAFE (Tactical Separation Assisted Flight Environment) Apollo Lunar Pilot Example PC: 37 constraints in simplePC and 6 in complexPC

Page 20: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Related WorkTools that perform mixture of concrete and

symbolic executionEXE, DART, CUTE, PEX, SAGE, …

“Higher order test generation” – P. Godefroid [PLDI’11]Uses combination of validity checking and un-

interpreted functionsGenerates tests from validity proofs Implementation challenge

Page 21: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Conclusions and Future WorkMixed concrete-symbolic solving to address problems

with classic symbolic executionHandling native libraries Incomplete decision procedures

Open source implementation for JavaFuture Work

More experimentsMore heuristicsHandle data structures executed outside symbolic

execution Use JPF’s serialization

Page 22: Symbolic Execution with Mixed Concrete-Symbolic Solving Corina  Pasareanu 1 ,  Neha  Rungta 2  and Willem Visser 3 1 Carnegie Mellon,  2 SGT Inc./NASA Ames

Thank you!