SampleSearch: A scheme that searches for Consistent

Samples

Vibhav Gogate and Rina Dechter

University of California, IrvineUSA

Outline Background

Bayesian Networks with Zero probabilities Importance Sampling Rejection Problem

The SampleSearch Scheme Algorithm Sampling Distribution and its Approximation

Experimental Results

Bayesian Networks: Representation(Pearl, 1988)

))(|()(

))(|(

1

i

n

i

i

ii

XpaXPXP

XpaXP

:CPTs

lung Cancer

Smoking

X-ray

Bronchitis

DyspnoeaP(D|C,B)

P(B|S)

P(S)

P(X|C,S)

P(C|S)

P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B)

(A) Probability of EvidenceP(smoking=no, dyspnoea=yes)=?

(B) Belief Updating:P (lung cancer=yes | smoking=no, dyspnoea=yes ) = ?

Complexity Belief Updating

NP-hard when zeros are present General case when all CPTs are positive, not known. Relative Approximation

Randomized Polynomial time algorithm when all CPTs are positive (Dagum and Luby 1997)

Probability of Evidence NP-hard when zeros are present Relative Approximation

Randomized Polynomial time algorithm when all CPTs are positive and (1/P(e)) is polynomial (Karp, Dagum and Luby 1993)

Importance Sampling (Rubinstein ’81)

N

i i

i

N

Q

Xx

Xx

)Q(x

)f(x

NM:

). from Q(x,...,x,xsamples xGenerate

Q(x) f(x)

Q(x)

f(x)EQ(x)

Q(x)

f(x)M

rite n Q(x) rewistributioroposal) dGiven a (p

f(x)M

1

21

1~sampling Importance

00

Importance Sampling for Belief Updating

EX

eE

XiEX

eExiXi

iiii

XP

XP

eP

exXPexXP

\

|

)(\

,|

)(

)(

)(

),()|(

Zz

eExiXi

zfM

XPzf

XiEXZ

)(1

)()(

)(\

,|

Yy

eE

yfM

XPyf

EXY

)(2

,)()(

\

|

Generating i.i.d. samples from Q

Q(A,B,C)=Q(A)*Q(B|A)*Q(C|A,B)

Q(A)=(0.8,0.2) Q(B|A)=(0.4,0.6,0.2,0.8) Q(C|A,B)=Q(C)=(0.2,0.8)

A=0

B=0 B=1 B=0 B=1

A=1

C=1C=1C=1 C=0 C=0 C=0 C=1

Root

0.8 0.2

0.4 0.6

0.2 0.8 0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

C=0

),...,|(.....*)|(*)( 11121 nn XXXQXXQXQ Q(X)

Rejection Problem Importance Sampling requirement

f(xi)>0 => Q(xi)>0

Conversely, Q(xi) can be >0 even if f(xi)=0.

So if the probability of sampling ∑Q(xi|f(xi)>0) is very small A large number of assignments will have zero

weight Extreme case: Our approximation = zero.

N

i i

i

xQ

xf

NM

1 )(

)(1~

Rejection Problem

All Blue leaves correspond to solutions i.e. f(x) >0All Red leaves correspond to non-solutions i.e. f(x)=0

solution. anot is xif 0)(

)(

)(1~ :sampling Importance

i

1

i

N

i i

i

xf

xQ

xf

NM

A=0

B=0

C=0

B=1 B=0 B=1

A=1

C=1C=1C=1 C=0 C=0 C=0 C=1

Root

0.8 0.2

0.4 0.6

0.2 0.8 0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Example: map coloring Variables - countries (A,B,C,etc.)

Values - colors (red, green, blue)

Constraints:

A Solution is an assignment that satisfies all constraints

etc. ,ED D, AB,A

C

A

B

DE

F

G

Constraint Networks (Dechter 2003)

Constraint networks to model “zeros”

A

F

C

D

B

G

A C P(C|A)

0 0 0

0 1 1

1 0 1

1 1 0

Constraints

A=0, C=0 not allowed

A=1, C=1 not allowed

Or A≠C

Why constraints?

• For a partial sample if a constraint is violated f(X=x)=0 for any full extension X=x of the sample.

•For every full assignment X=x

•solution implies f(X=x) >0 and

•non-solution f(X=x)=0

Using Constraints

A=0

B=0

C=0

B=1 B=0 B=1

A=1

C=1C=1C=1 C=0 C=0 C=0 C=1

Root

0.8 0.2

0.4 0.6

0.2 0.8 0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Constraints

A≠B, A≠C

solution. anot is xif 0)(

)(

)(1~ :sampling Importance

i

1

i

N

i i

i

xf

xQ

xf

NM

Using Constraints

A=0

B=0 B=1

C=1C=1 C=0 C=0

Root

0.8

0.4 0.6

0.2 0.8 0.2 0.8

C=0

Constraints

A≠B, A≠C

Constraint A≠B violated

solution. anot is xif 0)(

)(

)(1~ :sampling Importance

i

1

i

N

i i

i

xf

xQ

xf

NM

Outline BackgroundBackground

Bayesian NetworksBayesian Networks Importance SamplingImportance Sampling Rejection PrblemRejection Prblem

The SampleSearch Scheme Algorithm Sampling Distribution and Approximation

Experimental Results

Algorithm SampleSearch

A=0

B=0

C=0

B=1 B=0 B=1

A=1

C=1C=1C=1 C=0 C=0 C=0 C=1

Root

0.8 0.2

0.4 0.6

0.2 0.8 0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Constraints

A≠B, A≠C

Algorithm SampleSearch

A=0

B=0

C=0

B=1 B=0 B=1

A=1

C=1C=1C=1 C=0 C=0 C=0 C=1

Root

0.8 0.2

0.4 0.6

0.2 0.8 0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Constraints

A≠B, A≠C

1

Algorithm SampleSearch

A=0

B=1 B=0 B=1

A=1

C=1C=1C=0 C=0 C=0 C=1

Root

0.8 0.2

0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Constraints

A≠B, A≠C

Resume Sampling

1

Algorithm SampleSearch

A=0

B=1 B=0 B=1

A=1

C=1C=1C=0 C=0 C=0 C=1

Root

0.8 0.2

1

0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Constraints

A≠B, A≠C

Constraint Violated Until Solution i.e. f(x)>0 found

1

Generate more Samples

A=0

B=0

C=0

B=1 B=0 B=1

A=1

C=1C=1C=1 C=0 C=0 C=0 C=1

Root

0.8 0.2

0.4 0.6

0.2 0.8 0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Constraints

A≠B, A≠C

Generate more Samples

A=0

B=0

C=0

B=1 B=0 B=1

A=1

C=1C=1C=1 C=0 C=0 C=0 C=1

Root

0.8 0.2

0.4 0.6

0.2 0.8 0.2 0.8 0.2 0.8 0.2 0.8

0.2 0.8

Constraints

A≠B, A≠C

1

Traces of SampleSearch

A=0

B=1

C=1

Root

A=0

B=0B=1

C=1

Root

A=0

B=1

C=1

Root

C=0

A=0

B=0 B=1

C=1

Root

C=0

Constraints

A≠B, A≠C

The Sampling distribution QR of SampleSearch

A=0

B=0 B=1

C=1C=1 C=0

Root

0.8

0 1

0 0 0 1

C=0

What is probability of generating A=0?

QR(A=0)=0.8

Why? SampleSearch is systematic

What is probability of generating B=1?

QR(B=1|A=0)=1

Why? SampleSearch is systematic

What is probability of generating B=0?

Simple: QR(B=0|A=0)=0

All samples generated by SampleSearch are solutions

Did you generate samples from Q? -NO!

Backtrack-free distribution

N

i i

i

xQ

xf

NM

1 )(

)(1~

Computing QR

Invoke an oracle or a complete search procedure O(n) times per sample

A=0

B=1

C=1

Root

?? Solution

?? Solution

?? Solution

Approximation AR of QR

A=0

B=0 B=1

C=1C=1 C=0

Root

0.8

0 1

0 0 0 1

C=0

Hole

Don’t know

No solutions here

No solutions here

•IF Hole THEN AR=Q

•IF No solutions on the other branch THEN AR=1

Approximation AR of QR

A=0

B=1

C=1

Root

A=0

B=0 B=1

C=1

Root

C=0

Problem: Can’t guarantee convergence

?

?

0.8 0.8

0.6 1

0.8

A=0

B=0B=1

C=1

Root

0.8

?

1

0.8

A=0

B=1

C=1

Root

C=0

0.8

?

0.6

1 1

Guarantee convergence in the limit Store all possible traces

A=0

B=1

C=1

Root

C=0

0.8

?

0.6

1

A=0

B=0B=1

C=1

Root

0.8

?

1

0.8

Approximation ARN

IF Hole THEN ARN=Q

IF No solutions on other branch THEN AR

N=1

unbiasedallyAsymptotic

MxA

xfE

xQ

xfExfM

RN

N

RXx

])(

)([lim

])(

)([)(

A=0

B=1

C=1

Root

0.8

1

1

?

Improving Naive SampleSeach

Handle Non-binary domains See the paper, Proof is complicated.

Better Search Strategy Can use any state-of-the-art CSP/SAT solver e.g.

minisat (Sorrenson et al 2006) All theorems and result hold

Better Importance Function Use output of generalized belief propagation to

compute the initial importance function Q (Gogate and Dechter 2005)

Experimental Results Previous Algorithms

Likelihood weighting (LW) Proposal=Prior

IJGP-sampling (IJGP-S) (Gogate and Dechter 2005) Proposal=Output of generalized belief propagation

Adding SampleSearch SampleSearch with LW (S+LW) SampleSearch with IJGP-sampling (S+IJGP-S)

Linkage BN_69

Linkage BN_73

Linkage BN_76

Conclusions

• Belief networks with zero probabilities lead to the Rejection problem in importance Sampling.

• We presented a SampleSearch scheme that works with any importance sampling scheme to circumvent the Rejection Problem.

• Sampling Distribution of SampleSearch is the backtrack-free distribution QR

– Expensive to compute– Approximation of QR based on storing all traces that yields an

asymptotically unbiased estimator• Empirically, when a substantial number of zero

probabilities are present, SampleSearch based schemes dominate their pure sampling counter-parts.