Probabilistic CEGAR*Björn Wachter

Joint work with Holger Hermanns, Lijun Zhang

AVACS

Supported by

Uni Saar

*To appear in CAV

2

Introducing

Probabilistic Model Checking

CEGAR (counterexample-guided abstraction refinement) PASS does CEGAR for probabilistic models

Reach· 0:03(f ail)?

1

3

PRISM & PASS

PRISM Very popular probabilistic model checker Finite-state

PASS Supports PRISM models handles infinite-state as well Under the Hood:

Predicate abstraction SMT Interpolation

4

Comparison to PRISM

Network protocols Wireless LAN, CSMA Bounded Retransmission Sliding Window

Model (#)

State reduction

Speed-up

WLAN (3)WLAN (1)

16x-152x?

1,3x-7xTO->311s

CSMA (4)

41x-248x

1x-2x

BRP (3) 1x 1/2x - 1/3x

PRISM vs PASS

5

Basics Paths, Markov Chains, MDPs Counterexamples Probabilistic Programs Predicate Abstraction

Abstraction Refinement Abstract Counterexamples Path Analysis Strongest Evidence CEGAR algorithm

Experimental Results Conclusion

ProgramReach· p(e)?

e

Probabilistic Reachability Problem

Overview

6

Paths, MCs, MDPs

Weighted Path

Markov Chain

non-determinism …

2/3

1/31/3

1/3

1/3

1/3 2/3 1/3

7

Paths, MCs, MDPs

2/3

1/31/3

1/3

1/3

2/31/3

1/3

1/3

1/3

1

1/21/2

1/3 2/3 1/3Weighted Path

Markov Chain

MarkovDecisionProcess

8

Adversary

Adversary resolves transition non-determinism

2/31/3

1/3

1/3

1/3

1

1/21/2

9

Probabilistic Reachability

Probability to get from green to red Weighted Path

Markov Chain

Markov Decision Process

2/3

1/31/3

1/3

1/3

2/31/3

1/3

1/3

1/3

1

1/21/2

X

¼P (¼) =

13

P (¼) = 227

maxM C

P (M C) =13

1/3 2/3 1/3

10

Guarded command language à la PRISM Variables: integer, real, bool Non-determinism: interleaving

Example:

Program = (variables, commands, initial condition)

Probabilistic Programs

x > 0 ! 0:2 : (x0:= x + 1) + 0:8 : (x0:= x + 2)

x=1

0.2: (x‘:=x+1)

x=2

Update #1

0.8: (x‘:=x+2)x=3

Update #2

Guard: x>0

guard

Labels for CEX Analysis

11

Predicates: partition the state space are boolean expressions

x>0, x<y, x + y = 3 (variables x,y)

Abstract MDP Probabilistic may-transitions

Similar to Blast, SLAM, Magic … See our [Qest’07] paper

Abstraction guarantees upper bound

Predicate Abstraction

actual

1

0

Probability:

Abstract MDP

12

May Transitions

Hier ist‘s noch nicht verständlich genug! Besseres Beispiel wo #abs. trans < #conc.

trans

0.2

0.8

1.0

0.2

0.8

1.0

abstractconcrete

13

CEGAR Loop

pactual

upperabstract check

refine

Probability

CEX?

Real CEX

Low enough

14

Counterexamples (CEX)

Resolution of non-determinism initial state adversaryinduces a Markov chain

Counterexample: Resolution of non-detsuch that probability threshold exceeded

Example:CEX for

Witness of Reachability probability

in MDP

Reach1=62/3

1/31/3

1/3

1/3

1

1/21/2

15

Path 1 Path 2 Path 3 Path 4 …

Counterexample Analysis: Idea

Idea: Enumerate paths of Markov chain Sort paths by probability [Han\Katoen2007]: visit paths with highest measure first Realizable Spurious

Path 1 Path 2 Path 3 Path 4 …

Probability of Abstract CEX / Markov Chain

How much MEASURE is REALIZABLE? More than p?

16

Path Analysis

Abstract path: Two cases

Realizable if there‘s a corresponding concrete path

Spurious: no corresponding path

Splitter predicate exists iff path spurious Interpolation: predicate from unsatisfiable path formula

u u´ u´´

u u´ u´´

u u´

u´´

Reachable with prefix

Can do postfix

Pathformula

SAT

UNSAT

Logic (SMT)

17

Path Analysis

Abstract path: Two cases

Realizable if there‘s a corresponding concrete path

Spurious: no corresponding path

Splitter predicate (interpolant):

u u´ u´´

u u´ u´´

0 1x´:=x+1

2x´:=x+1

109x´:=x+1

Reachable with prefix

Can do postfix

Pathformula

SAT

UNSAT

Logic (SMT)

x=0 x=1

X 10x>1 ¸

x · 2

18

Example

1.0

concrete abstract

0.2

0.8

0.50.5

0

Probability:

Upper: 1.0

0.8 0.2 ?

19

Example(cont): after refinement

0.4

Concrete abstract

0.4

0

Probability:

Upper: 0.4

0.8

0.5

lower

20

Example 2

1.00.8

1.0

0.80.8

0.2

0.8

0.2

0.2

concrete abstract

0.8

0.2

0.2

0

lower0.8

Upper1.0

MultipleInitial states

21

Example 2

1.00.8

1.0

0.80.8

0.2

0.8

0.2

concrete abstract

0.8

0.2

0.2

Maximum

Find Maximal Combination by MAX-SMT ( paper)

0.80.8

0

Probability:

lower0.8

Upper1.0

22

CEX Analysis:Semi decision procedure Problem in general: undecidable Too many spurious paths abort counterexample

analysis

Output: collection of predicates

Enough realizable probability

Path 1 Path 2 Path 3 Path 4 …Path 1 Path 2 Path 3 Path 4 …

> CLimit # of

spurious paths to enforce

termination

Path 1 Path 2 Path 3 Path 4 …Path 1 Path 2 Path 3 Path 4 …

Can take many pathsTo obtain enough realizableprobability

0

lower= real

23

Related Work

Probabilistic Counterexamples: … however not in the context of abstraction

Hermanns/Aljazzar (FORMATS’05) , Han/Katoen (TACAS’07) Abstraction Refinement for Prob. Finite-state Models

CEGAR for stochastic games, Chatterjee et al (UAI’05) Not based on counterexamples

D‘Argenio (Papm-Probmiv02), Fecher & al (SPIN’06): simulation

Magnifying-lens, de Alfaro et al (CAV’07): probability values

24

Conclusion & Future Work

Abstraction refinement … Counterexamples ~ Markov Chains

Markov Chains have cycles Model Checking Infinite-state Probabilistic Models Speed-up for huge finite-state models Future Work

Better Lower bounds

25

References

Tool website http://depend.cs.uni-sb.de/pass Literature

Our work Hermanns, Wachter, Zhang: Probabilistic CEGAR (CAV’08) Wachter, Zhang, Hermanns: MC Modulo Theories (Qest’07)

Counterexamples Hermanns, Aljazar: CEX for timed prob reachability, FORMATS‘05 Han, Katoen: CEX in probabilistic model checking, TACAS‘07

Probabilistic Abstraction Refinement De Alfaro, Magnifying-lens abstraction for MDPs, CAV‘07 Chatterjee, Henzinger, Majumdar: CEX-guided planning, UAI’05

26

Questions?

27

Is Counterexample analysis problem undecidable? Semi-decision algorithm heuristics If we only need finiteley many paths decidable if logic is If we need infinitely many undecidable