400

Exact and Approximate Sampling bySystematic Stochastic Search

Vikash MansinghkaMIT BCS & CSAIL

Daniel RoyMIT CSAIL

Eric JonasMIT BCS

Joshua TenenbaumMIT BCS & CSAIL

Abstract

We introduce adaptive sequential rejectionsampling, an algorithm for generating exactsamples from high-dimensional, discrete dis-tributions, building on ideas from classical AIsearch. Just as systematic search algorithmslike A* recursively build complete solutionsfrom partial solutions, sequential rejectionsampling recursively builds exact samplesover high-dimensional spaces from exact sam-ples over lower-dimensional subspaces. Ouralgorithm recovers widely-used particle fil-ters as an approximate variant without adap-tation, and a randomized version of depthfirst search with backtracking when appliedto deterministic problems. In this paper,we present the mathematical and algorith-mic underpinnings of our approach and mea-sure its behavior on undirected and directedgraphical models, obtaining exact and ap-proximate samples in a range of situations.

1 Introduction

Efficient inference in high-dimensional, discrete prob-ability models is a central problem in computationalstatistics and probabilistic AI. In this paper, we intro-duce a recursive algorithm for exact sampling aimedat solving this problem in the presence of multimodal-ity. We do this by generalizing ideas from classic AIsearch to the stochastic setting. Just as systematicsearch algorithms like A* recursively build completesolutions from partial solutions, sequential rejectionsampling recursively builds exact samples over high-

Appearing in Proceedings of the 12th International Confe-rence on Artificial Intelligence and Statistics (AISTATS)2009, Clearwater Beach, Florida, USA. Volume 5 of JMLR:W&CP 5. Copyright 2009 by the authors.

dimensional spaces from exact samples over lower-dimensional subspaces. Our method exploits and gen-eralizes ideas from classical AI search for managing de-terministic dependencies, including depth first traver-sal with early constraint checking and backtracking, totractably generate exact and approximate samples.

Many popular approaches to inference, such as mean-field variational methods (Jordan et al., 1999), con-vex relaxations (Wainwright et al., 2002; Sontag andJaakkola, 2008), and generalized belief propagation(Yedidia et al., 2001), focus on approximating MAPassignments or (low-dimensional, e.g. 1 or 2 variable)marginal distributions. While effective in many set-tings, low-dimensional marginals (and MAP assign-ments) often do not capture the essential features of adistribution, especially in the presence of multimodal-ity. Homogeneous Ising models, where the probabilityof a state x = (x1, . . . , xn) is1

P (x) ∝ exp{−J

∑(i,j)∈E

xixj

}, x ∈ {−1, 1}n, (1)

provide one source of extreme examples. As the cou-pling parameter J increases, the joint distribution onspins approaches a 2-component mixture on the “allup” and “all down” states, which has only 1 bit of en-tropy. A MAP approximation misses the fundamentalbimodality of the distribution, while the minimum-KLproduct-of-marginals approximation confuses this dis-tribution with the uniform distribution on spins.

When a distribution contains many widely separatedmodes, and is therefore difficult to parametricallyapproximate, simulation-based inference seems ideal.Exact and approximate sampling are also of intrin-sic interest in computational physics (Propp and Wil-son, 1996; Edwards and Sokal, 1988). Unfortunately,popular methods like Gibbs sampling often run into

1We use f(x) ∝ g(x, y) to mean that f(x) = c(y) g(x, y)for some (in the case of distributions, normalizing) constantof proportionality c(y).

401

Exact and Approximate Sampling by Systematic Stochastic Search

severe convergence problems in precisely these set-tings. This difficulty motivates a number of specializedsamplers that exploit sophisticated data augmentationtechniques (Swendsen and Wang, 1987), as well as avariety of model-specific proposal designs.

Our algorithm mitigates the problems of multimodal-ity by generalizing ideas for managing deterministicdependencies from the constraint satisfaction litera-ture. In particular, it operates by a stochastic gen-eralization of systematic search. In deterministic sys-tematic search, solutions to a problem are built uppiece-by-piece2. The first complete candidate solutionis either exact (as in backtracking search or A*) orapproximate (as in beamed searches), and strategiesfor search tree expansion are often used to managedeterministic dependencies among chains of choices.Our algorithm automatically recovers a randomizedvariant of one such method, depth first search withbacktracking and early constraint checking, when ap-plied to constraint satisfaction problems, generalizingthese ideas to the setting of sampling. Furthermore,if the rejection step in our algorithm is replaced withimportance sampling with resampling (and particularrestricted choices of variable orderings are used) we re-cover widely-used particle filtering algorithms for ap-proximate sampling.

In this paper, we present the mathematical and algo-rithmic underpinnings of our approach and measure itsbehavior on Ising models, causal networks and a stereovision Markov random field, obtaining exact and ap-proximate samples in a range of situations.

2 The Adaptive Sequential RejectionAlgorithm

Consider the problem of generating samples from anhigh-dimensional discrete distribution with densityP (x), x ∈ X. In many cases, we can only efficientlycompute the density to within a multiplicative con-stant; that is, we can compute a function P (x) suchthat P (x) = P (x)/Z. This setting arises in fac-tor graph inference, where the normalization constantZ =

∑x∈X P (x) is due to either the partition function

2Contrast with local search, e.g. fixed-point iteration,where a complete but possibly poor quality approximatesolution is repeatedly iterated upon until it stabilizes.Markov chain Monte Carlo methods generalize this fixed-point iteration idea to the space of distributions, asymptot-ically converging to the correct distribution and recoveringdeterministic fixed-point algorithms for particular proposaland target choices. Techniques like coupling from the past(Propp and Wilson, 1996; Huber, 2002; Childs et al., 2000)provide the distributional analogues of termination analy-sis, sometimes allowing automatic determination of whenexact samples have been obtained.

of the underlying undirected model or the marginallikelihood of the evidence in the underlying directedmodel. Let P (x) = P (y, z) = ψ1(y)ψ2(y, z), whereX = Y × Z is a decomposition of the state that re-sults in a factored representation for P . For example,we might take Z to be one variable, and Y to be allthe rest. Our algorithm generates exact samples fromP (y, z) by recursively generating an exact sample yfrom P ′(y) = ψ1(y) (which we assume has an analo-gous decomposition, i.e. Y and ψ1 factor), and thenextending y to an exact sample (y, z) from P (y, z) byrejection.

In order to apply our algorithm to an arbitrary factorgraph, we will need a way of recursively decompose amodel into a nested sequence of distributions. We willreturn to this issue later in the paper, using ideas fromdeterministic search, where variables will be broughtin one at a time. First, we present the basic recursionin our algorithm, assuming a decomposition has beenchosen.

Assume (by induction) that we have an exact sampley from P ′(y). Let P (y) =

∑z′ P (y, z′) be the (un-

normalized) marginal distribution of y under P . Wedefine3 a (Gibbs) transition kernel from Y into Z by

QP (z | y) ,P (y, z)P (y)

(2)

=ψ1(y)ψ2(y, z)∑z′ ψ1(y)ψ2(y, z′)

(3)

=ψ2(y, z)∑z′ ψ2(y, z′)

, (4)

and sample z from QP ( · | y). We then treat (y, z),whose density is P ′(y)QP (z | y), as a proposal to arejection sampler for P (y, z). Let x = (y, z) and definethe weight of the sample x as

WP ′→P (x) ,P (y, z)

P ′(y)QP (z | y)(5)

=P (y)P ′(y)

(6)

=∑

z′ ψ1(y)ψ2(y, z′)ψ1(y)

(7)

=∑z′

ψ2(y, z′). (8)

Note that the weight does not depend on z and so weconsider the weight WP ′→P (y) a function of y.

We then accept x as an exact sample from P withprobability

WP ′→P (y)CP ′→P

, (9)

3We will use , to denote definitions.

402

Mansinghka, Roy, Jonas, Tenenbaum

where CP ′→P is any constant such that CP ′→P ≥WP ′→P (y) for all y ∈ Y. In general, loose up-per bounds on WP ′→P (y) are easy to find but in-troduce unnecessary rejections. On the other hand,overconfident values of C are also easy to find, butwill result in approximate samples (or, more precisely,exact samples from the distribution proportional tomin{P (y, z), CP ′→P P

′(y)QP (z | y)}.) Both variantsmay have practical value. Here, we focus on the settingwhere we actually use the optimal rejection constant:

C∗P ′→P , maxy

WP ′→P (y) = maxy

∑z′

ψ2(y, z′). (10)

If y = (y1, . . . , yn) is high-dimensional, then the worstcase complexity of calculating C∗P ′→P is exponentialin n. However, when the sequence of distributions weare using has sparse dependencies (i.e., when ψ2(y, z)is a function of only O(log n) dimensions yi), then wecan calculate C∗P ′→P in polynomial time. For exam-ple, in 2-d grid Ising models, ψ2 depends on at mostthree neighbors and therefore C∗ can be calculated inconstant time.

This inductive argument describes the non-adaptivesequential rejection sampler. We apply it to samplingfrom the joint distribution of factor graph models byautomatically constructing a nested sequence of distri-butions from orderings on the variables, using machin-ery we introduce later. Sequential importance sam-pling with resampling (SIR) can be viewed as an ap-proximate variant, where the rejection step - produc-ing one exact sample, or failing - is replaced with animportance sampling and resampling step propagat-ing k particles approximately drawn from the target.The weight per particle is the same as in sequentialrejection.

The choice of the Gibbs transition kernel is important.Incorporating the ψ2(y, z) factor into the proposal pre-vents the algorithm from proposing samples z that arealready known to be incompatible with the setting y.Thus we recover early constraint checking, and gen-eralize it to favor paths that seem probable given thecurrent partial assignment.

2.1 Adaptation Stochastically GeneralizesBacktracking

Systematic searches typically avoid reconsidering par-tial solutions that have been discovered inconsistent;this behavior is known as backtracking, and requiresdynamically recording the information about inconsis-tent states obtained over the course of search. Weaccomplish this in the broader setting of sampling byintroducing an adaptation rule into our sampler, whichrecovers this deterministic avoidance in the limit of de-terministic inconsistency.

Following Eq. 9, the non-adaptive sampler with theoptimal rejection constants C∗ accepts samples withprobability

αP ′→P =EP ′(WP ′→P (y))

C∗P ′→P

. (11)

From Eq. 6, we have that

WP ′→P (y) ∝ P (y)P ′(y)

, (12)

and therefore, using the definition of C∗P ′→P and can-celing the constant of proportionality shared betweenW and C∗, we have

αP ′→P =

∑y P

′(y) P (y)P ′(y)

maxyP (y)P ′(y)

(13)

= miny

P ′(y)P (y)

. (14)

Note that the acceptance probability αP ′→P dependsonly on the choice of P ′ and P and is precisely thesmallest ratio in probability assigned to some y ∈ Y.4

An interesting special case is when the simpler distri-bution P ′(y) matches the marginal P (y). In this case,Wp′→p = 1 and we always accept.5 Assuming eachattempt to generate samples from P ′ by rejection suc-ceeds with probability αP ′ , the entire rejection sam-pler will succeed with probability αP ′αP ′→P . If thisprobability is O(2−w), where w is the tree width ofthe factor graph, then, in expectation, we will be nobetter off than using variable clustering and dynamicprogramming to calculate marginals and sample ex-actly.

Our goal then is to drive αP ′→P → 1 (and inductively,αP ′ → 1). Consider the extreme case where a sam-pled value y is revealed to be inconsistent. That is,ψ2(y, z) = 0 for all z and therefore WP ′→P (y) = 0.We should then adjust P ′ (and its predecessors, recur-sively) so as to never propose the value y = y again.Certainly if P ′ is the marginal distribution of P (re-cursively along the chain of rejections), this will takeplace.

Consider the (unnormalized) proposal density

P ′S(y) = P ′(y)∏y′∈S

(WP ′→P (y′)C∗P ′→P

)δyy′

(15)

4In particular, the acceptance is positive only if P (y) >0 =⇒ P ′(y) > 0 (i.e., P ′ is absolutely continuous withrespect to P ).

5While it may be tempting to think the problem issolved by choosing P = P ′, if each stage of the algo-rithm performed this marginalization, the overall complex-ity would be exponential. The key to adaptation will beselective feedback.

403

Exact and Approximate Sampling by Systematic Stochastic Search

where S ⊂ Y and δyy′ is the Kronecker delta satisfyingδyy′ = 1 if y = y′ and 0 otherwise. Then

WP ′S→P (x) ,

P (y, z)P ′S(y)QP (z | y)

(16)

=WP ′→P (y)∏

y′∈S

(WP ′→P (y′)

C∗P ′→P

)δyy′(17)

=

{C∗P ′→P y ∈ SWP ′→P (y) y 6∈ S,

(18)

where step (17) follows from Eq. (6). ThereforeC∗P ′

S→P = C∗P ′→P . In particular, if S = Y, thenWP ′

S→P (y) = C∗P ′S→P = C∗P ′→P and every sample is

accepted. In fact,

P ′S=Y(y) = P ′(y)∏

y′∈Y

(WP ′→P (y′)C∗P ′→P

)δyy′

(19)

∝ P ′(y)WP ′→P (y) (20)

= P ′(y)P (y)P ′(y)

(21)

= P (y) (22)

and therefore an exact sample from P ′Y is a samplefrom the marginal distribution of P . The Gibbs kernelexactly extends this to a sample from the joint.

Adaptation then involves the following modificationto our algorithm: after proposing a sample (y, z), weaugment S with y. As S → Y, P ′S(y) → 1

C∗P ′→P

P (y)pointwise.

This change can be implemented efficiently by stor-ing a hashmap of visited states for every distribu-tion in the sequence and modifying density evaluation(and, therefore, the Gibbs kernels) to reflect hashmapcontents. Each stage of the sampler pushes statesto the previous stage’s hashmap as adaptation pro-ceeds, moving each proposal distribution towards theideal marginal. Because such adaptation leaves C∗

unchanged (see Appendix), adaptation increases thealgorithmic complexity by only a linear factor in thenumber of sampling attempts, with overall complex-ity per attempt still linear in the number of variables.Taken together, the hashmaps play the role of thestack in a traditional backtracking search, recordingvisited states and forbidding known bad states frombeing proposed.

2.2 Sequences of Distributions for GraphicalModels

To apply this idea to graphical models, we need a wayto generically turn a graphical model into a sequence

a)

x1f1 x2 f2f12

x3f3

f13

x4 f4f34

f24

b)

x1f1

x3f3

f13

Figure 1: A four node Ising model, and its restriction tothe variables x1 and x3.

of distributions amenable to adaptive sequential rejec-tion. We accomplish this - and introduce further ideasfrom systematic search - by introducing the idea of asequence of restrictions of a given factor graph, basedon a variable ordering (i.e. permutation of the vari-ables in the model). Each sequence of restrictions canbe deterministically mapped to a nested sequence offactor graphs which, for many generic orderings, cap-ture a good sequence of distributions for sequentialrejection under certain analytically computable condi-tions.

We denote by Xi a random variable taking values xi ∈Xi. If V = (X1, . . . , Xk) is a vector random variables,then we will denote by XV the cartesian product spaceX1 × · · · ×Xk in which the elements of V take valuesv = (x1, . . . , xk).

Definition 2.1 A factor graph G = (X,Ψ, V )is an undirected X,Ψ-bipartite graph where X =(X1, . . . , Xn) is a set of random variable and Ψ ={ψ1, . . . , ψm} is a set of factors. The factor ψi rep-resents a function XVi

7→ [0,∞] over the variablesVi ⊂ X adjacent to ψi in the graph. The graph repre-sents a factorization

P (v) = P (x1, . . . , xn) =1Z

∏i

ψi(vi) (23)

of the probability density function P , where Z is thenormalization constant.

Definition 2.2 The restrictionGS of the factor graphG = (X,Ψ, V ) to a subset S ⊂ X is the subgraph(S,ΨS , VS) of G consisting of the variables S, thecollection of factors ΨS = {ψi ∈ Ψ | Vi ⊂ S}that depend only on the variables S, and the edgesVS = {Vi | ψi ∈ ΨS} connecting the variables S andfactors ΨS. We denote by ZS the normalization con-stant for the restriction.

See Figure 1 for an example restriction of an Isingmodel. Consider a factor graph G = (X,Ψ, V ) and letX1:k = {x1, . . . , xk} ⊂ X, (k = 1, . . . , n) be the first kvariables in the model under some order. The sequence

404

Mansinghka, Roy, Jonas, Tenenbaum

of distributions we consider are the distributions givenby the restrictions GX1:k , k = 1, . . . , n.

We recover likelihood weighting (generalizing it to in-clude resampling) on Bayesian networks when we usethe importance variant of our algorithm and a topo-logical ordering on the variables. Similarly, we recoverparticle filtering when we apply our method to time se-ries models, with resampling instead of rejection andan ordering increasing in time.

In this paper, we focus on generically applicable strate-gies for choosing an ordering. All our exact sam-pling results use a straightforward ordering which firstincludes any deterministically constrained variables,then grows the sequence along connected edges inthe factor graph (with arbitrary tie breaking). Thisway, as in constraint propagation, we ensure we sat-isfy known constraints before attempting to addressour uncertainty about remaining variables. If we donot do this, and instead sample topologically, we findthat unlikely evidence will lead to many rejections (andapproximate rejection, i.e. likelihood weighting, willexhibit high variance). In general, we expect findingan optimal ordering to be difficult, although heuris-tic ordering information (possibly involving consider-able computation) could be exploited for more efficientsamplers. An adaptive inference planner, which dy-namically improves its variable ordering based on theresults of previous runs, remains an intriguing possi-bility.

3 Experiments

First, we measure the behavior on ferromagnetic Isingmodels for a range of coupling strengths, including thecritical temperature and highly-coupled regimes whereGibbs samplers (and inference methods like mean-fieldvariational and loopy belief propagation) have well-known difficulties with convergence; see Figure 3 showssome of our results.

We have also used our algorithm to obtain exact sam-ples from 100x100-dimensional antiferromagnetic (re-pulsive) grid Ising models at high coupling, with no re-jection, as is expected by analytic computation of theαs, describing probability of acceptance. At this scale,exact methods such as junction tree are intractabledue to treewidth, but the target distribution is verylow entropy and generic variable orderings that respectconnectedness lead to smooth sequences and there-fore effective samplers. We have also generated fromexact samples from 20x20-dimensional ferromagneticgrid Isings at more intermediate coupling levels, whereadaptation was critical for effective performance.

We also measured our algorithm’s behavior on ran-

Figure 2: (left 4) Exact samples from a 10x10-dimensional grid ferromagnetic Ising just below the crit-ical temperature. (right 4) Exact samples from a 10x10-dimensional grid ferromagnetic Ising just above the criticaltemperature.

0 1 2 3 4 5Coupling Parameter (Attractive)

0.2

0.4

0.6

0.8

1.0

Pro

babi

lity

of S

ucce

ss

5x5-Dimensional Ising

0 1 2 3 4 5Coupling Parameter (Repulsive)

0.0

0.2

0.4

0.6

0.8

1.0

Pro

babi

lity

of S

ucce

ss

10x10-Dimensional Ising

Figure 3: (left) Ferromagnetic (right) Antiferromag-netic. (both) Frequency of acceptance in nonadaptive(blue, lower) and adaptive (red, higher) sequential rejec-tion as a function of coupling strength J . Naive rejectionapproaches suffer from exponential decay in acceptanceprobability with dimension across all coupling strengths,while generic MCMC approaches like Gibbs sampling failto converge when the coupling reaches or exceeds the criti-cal value. Note that adaptive rejection improves the boundon the region of criticality.

domly generated (and in general, frustrated) Isingmodels with coupling parameters sampled fromU [−2, 2]. We report results for a typical run of theadaptive and non-adaptive variants of sequential re-jection sampling on a typical problem size; see Figure4 for details. We also note that we have successfullyobtained exact samples from 8x8-dimensional Isingswith randomly generated parameters, using adapta-tion. On the models we tested, we obtained our firstsample in roughly 5000 attempts, reducing to roughlyone sample per 1000 attempts after a total of 100,000had been made.

Given the symmetries in the pairwise potentials in(even) a frustrated Ising model without external field -the score is invariant to a full flip of all states - our al-gorithm will always accept with probability 1 on tree-structured (sub)problems. This is because the com-bination of Gibbs proposals (i.e. the arc-consistencyinsight) with the generic sequence choice (connectedordering) can always satisfy the constraints inducedby the agreement or disagreement on spins in these

405

Exact and Approximate Sampling by Systematic Stochastic Search

0 20000 40000 60000 80000 1000000

100020003000400050006000700080009000

Figure 4: Comparison of adaptive (top-red, center) andnonadaptive (top-blue/dashed, bottom) rejection samplingon a frustrated 6x6-dimensional Ising model with uniform[−2, 2] distributed coupling parameters. (top) Cumulativecomplete samples over 100,000 iterations. (lower plots) Ablack dot at row i of column j indicates that on the jthiteration, the algorithm succeed in sampling values for thefirst i variables. Only a mark in the top row indicates a suc-cessful complete sample. While the nonadaptive rejectionsampler (bottom) often fails after a few steps, the adaptivesampler (center), quickly adapts past this point and startsrapidly generating samples.

settings. Accordingly, our algorithm is more efficientthan other exact methods for trees (such as forwardfiltering with backward sampling) in these cases. If,on the other hand, the target distribution does notcontain this symmetry (so some of the initial choicesmatter), there will be some probability of rejection,unlike with forward filtering and backward sampling.This helps to explain the bands of rejection sometimesseen in the nonadaptive algorithm and the opportunityfor adaptation on Ising models, as it is impossible forthe algorithm to reject until a variable is added whenits already added neighbors disagree.

We also applied our method to the problem of diagnos-tic reasoning in bipartite noisy-OR networks. Theseproblems motivated several variational inference algo-rithms and in which topological simulation and beliefpropagation are known to be inaccurate (Saul et al.,1996). Furthermore, as in the ferromagnetic Ising set-ting, it seems important to capture the multimodal-ity of the posterior. A doctor who always reportedthe most probable disease or who always asserted youwere slightly more likely to be sick having visited himwould not have a long or successful professional life.The difficulty of these problems is due to the rarity ofdiseases and symptoms and the phenomenon of “ex-plaining away”, yielding a highly multimodal posteriorplacing mass on states with very low prior probability.We explored several such networks, generating sets of

Figure 5: Comparison of adaptive (top-red, center) andnonadaptive (top-blue/dashed, bottom) rejection samplingfor posterior inference on a randomly generated medical di-agnosis network with 20 diseases and 30 symptoms. Theparameters are described in the main text. (top) Cumu-lative complete samples over 100,000 iterations. (lowerplots) show the trajectories of a typical adaptive and non-adaptive run in the same format as Figure 4. Here, adap-tation is critical, as otherwise the monolithic noisy-OR fac-tors result in very low acceptance probabilities in the pres-ence of explaining away.

symptoms from the network and measuring both thedifficulty of obtaining exact samples from the full pos-terior distribution on diseases and the diagnostic ac-curacy. Figure 5 shows exact sampling results, withand without adaptation, for a typical run on a typicalnetwork, generated in this regime. This network had20 diseases and 30 symptoms. Each possible edge waspresent with probability 0.1, with a disease base rateof 0.2, a symptom base rate of 0.3, and transmissionprobabilities of 0.4.

The noisy-OR CPTs result in large factors (with alldiseases connected through any symptoms they share).Accordingly, the sequential rejection method gets nopartial credit by default for correctly diagnosing asymptom until all values for all possible diseases havebeen guessed. This results in a large number of rejec-tions. Adaptation, however, causes the algorithm tolearn how to make informed partial diagnoses betterand better over exposure to a given set of symptoms.

Finally, we applied our method to a larger-scale ap-plication: approximate joint sampling from a MarkovRandom Field model for stereo vision, using param-eters from (Tappen and Freeman, 2003). This MRFhad 61,344 nodes, each with 30 states; Figure 6 showsour results. We suspect exact sampling is intractableat this scale, so we used the importance relaxationof our algorithm with 1 and 5 particles. Because ofthe strong - but not deterministic - influence of the

406

Mansinghka, Roy, Jonas, Tenenbaum

0 50 100 150 200 250−5

−4

−3

−2

−1

0x 10

6

Relative Work (Gibbs Iterations)

Log

Pro

babi

lity

1 particle5 particlesAnnealed Gibbs

a. b. c.

Figure 6: Comparison of an aggressively annealed Gibbssampler (linear temperature schedule from 20 to 1 over200 steps) to the non-adaptive, importance relaxation ofour algorithm. The red circle denotes the mean of three1-particle runs. The horizontal bars highlight the qualityof our result. (a) Gibbs stereo image after sampling workcomparable to an entire 1-particle pass of (b) our algo-rithm. (c) Gibbs stereo image after 140 iterations.

external field, we needed a more informed ordering.In particular, we ordered variables by their restrictedentropy (i.e. the entropy of their distribution underonly their external potential), then started with themost constrained variable and expanded via connect-edness using entropy to break ties. This is one reason-able extension of the “most constrained first” approachto variable choice in deterministic constraint satisfac-tion. The quality of approximate samples with veryfew particles is encouraging, suggesting that appropri-ate sequentialization with arc consistency can lever-age strong correlations to effectively move through thesample space.

4 Discussion

Our experiments suggest that tools from systematicsearch, appropriately generalized, can mitigate prob-lems of multimodality and strong correlation in sam-pler design. When variables (and their attendant softconstraints) are incorporated one at a time, a samplermay be able to effectively find high probability regionsby managing correlations one variable at a time. Addi-tionally, any sample produced by sequential rejectionis, by definition, exact.

Prior work generating samples from very large (andsometimes critical) ferromagnetic Isings has insteadrelied on specialized cluster sampling methods to man-age the correlation problem. Given a convergentMarkov chain, coupling from the past techniques pro-vide termination analysis; i.e., they give a proof of

exact convergence. Comparing and combining theseapproaches seems fruitful, and would likely build onthe theory of sequential Monte Carlo samplers (and inparticular, backward kernels) from (Del Moral et al.,2006) and (Hamze and de Freitas, 2005). In the case ofapproximate sampling, other work has used determin-istic search as the subroutine of a sampler (Gogate andDechter, 2007), rather than recovering search behaviorin a limit. (Southey et al., 2002) uses local (not sys-tematic) search to improve the quality of the proposaldistribution for an importance sampler.

The rejection rate plots for Ising models show thatour algorithm runs into difficulty near the phase tran-sition, where the distribution is the most complex. Itseffectiveness may track semantic features of the dis-tribution, and it would be interesting to study thisrelationship analytically. It would also be interestingto explore SAT problems, which are known to empiri-cally exhibit a phase transition in hardness. It wouldalso be interesting to see how the rejection rate andmemory consumption of adaptation in our algorithmrelate to the cost of dynamic programming (ala junc-tion tree), and to explore the behavior of straightfor-ward blocked variants of our method where multiplevariables are added simultaneously.

Exact sampling may be truly intractable for largeproblems, with exact samplers useful primarily as asource of proposals for approximate algorithms. How-ever, it seems that recursive control structures fromcombinatorial optimization may be generally usefulin sampler design, and encourage the development ofsamplers whose efficiency actually improves as softconstraints harden and probabilistic reasoning prob-lems turn into satisfiability. By constraining our sam-pling algorithms to sample uniformly from satisfyingsolutions in the deterministic limit, we may arrive atmore useful methods for the uncertain reasoning prob-lems central to probabilistic AI.

AcknowledgementsThis work was partially supported by gifts from NTTCommunication Science Laboratories, the Eli LillyCorporation and Google, and National Science Foun-dation fellowships. The authors would like to thankNando de Freitas and Kevin Murphy for helpful dis-cussions, Keith Bonawitz for clarifying the notion ofrestrictions, and Jonathan Gonda for accidentally pro-viding critical inspiration.

References

A. M. Childs, R. B. Patterson, and D. J. C. MacKay. Exactsampling from non-attractive distributions using sum-mary states. arXiv:cond-mat/0005132v1, 2000.

P. Del Moral, A. Doucet, and A. Jasra. Sequential Monte

407

Exact and Approximate Sampling by Systematic Stochastic Search

Carlo Samplers. Journal of the Royal Statistical Society,68(3):411–436, 2006.

R. G. Edwards and A. D. Sokal. Generalizations of theFortuin-Kasteleyn-Swendsen-Wang representation andMonte Carlo algorithm. Physical Review, 38:2009–2012,1988.

V. Gogate and R. Dechter. Samplesearch: A scheme thatsearches for consistent samples. In Artificial Intelligenceand Statistics, 2007.

F. Hamze and N. de Freitas. Hot coupling: A particle ap-proach to inference and normalization on pairwise undi-rected graphs. In Advances in Neural Information Pro-cessing Systems 17, 2005.

M. Huber. A bounding chain for Swendsen–Wang. RandomStructures & Algorithms, 22(1):53–59, 2002.

M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K.Saul. An introduction to variational methods for graph-ical models. Machine Learning, 37(2):183–233, 1999.

J. G. Propp and D. B. Wilson. Exact sampling with cou-pled Markov chains and applications to statistical me-chanics. Random Structures and Algorithms, 9(1&2):223–252, 1996.

L. K. Saul, T. Jaakkola, and M. I. Jordan. Mean fieldtheory for sigmoid belief networks. Journal of ArtificialIntelligence Research, 4(4):61–76, 1996.

D. Sontag and T. Jaakkola. New outer bounds on themarginal polytope. In J. Platt, D. Koller, Y. Singer, andS. Roweis, editors, Advances in Neural Information Pro-cessing Systems 20, pages 1393–1400. MIT Press, 2008.

F. Southey, D. Schuurmans, and A. Ghodsi. Regularizedgreedy importance sampling. In Advances in Neural In-formation Processing Systems 14, pages 753–760, 2002.

R. H. Swendsen and J.-S. Wang. Nonuniversal criticaldynamics in monte carlo simulations. Physics ReviewLetters, 58(2):86–88, Jan 1987. doi: 10.1103/Phys-RevLett.58.86.

M. F. Tappen and W. T. Freeman. Comparison of graphcuts with belief propagation for stereo, using identicalmrf parameters. In Int. Conf. on Computer Vision, 2003.

M. J. Wainwright, T. S. Jaakkola, and A. S. Willsky. Anew class of upper bounds on the log partition function.In Uncertainty in Artificial Intelligence, pages 536–543,2002.

J. S. Yedidia, W. T. Freeman, and Y. Weiss. Generalizedbelief propagation. In Advances in Neural InformationProcessing Systems 13, pages 689–695. MIT Press, 2001.

5 Appendix

If the distribution P (x) is not the target distribu-tion but instead the distribution at some intermedi-ate stage in a sequential rejection sampler, the down-stream stage will adapt P to match its marginal. LetR ⊂ X, and consider the adapted distribution PR(x)with additional factors φx ∈ [0, 1] for x ∈ R. Forx 6∈ R, let φx = 1. We show that these additional fac-tors satisfy the existing pre-computed bound C∗ andthat sequential rejection on the adapted distribution

PR eventually accepts every sample. In this case, theweight of a sample is

WP ′→PR(y, z) ,

PR(y, z)P ′(y)QPR

(z | y)(24)

=∑z′

(ψ2(y, z′)

∏x∈R

φδx(y,z′)x

)(25)

and therefore

WP ′→PR(y, z) ≤

∑z′

ψ2(y, z′) = WP ′→P (y). (26)

We claim that WP ′S→PR

(y, z) ≤ C∗P ′→P . Let R′ andφ′ be the set of x = (y, z) and weights that have beenfed back to P in previous iterations of the algorithm.Consider

WP ′S→PR

(y, z) ,PR(y, z)

P ′S(y)QPR(z | y)

(27)

=WP ′→PR

(y)∏y′∈S

(WP ′→P

R′(y′)

C∗P ′→P

)δyy′(28)

=

WP ′→PR(y) y 6∈ S

WP ′→PR(y)

WP ′→PR′

(y)C∗P ′→P y ∈ S. (29)

Eq. (26) implies that when y 6∈ S, we haveWP ′

S→PR(y, z) ≤WP ′→P (y) ≤ C∗P ′→P . Therefore, the

claim is established for y ∈ S ifWP ′→PR

(y)

WP ′→PR′

(y) ≤ 1. We

have that

WP ′→PR(y)

WP ′→PR′ (y)=∑

z′ ψ2(y, z′)∏

x∈R φxδx(y,z′)∑

z′ ψ2(y, z′)∏

x∈R′ φ′xδx(y,z′)

(30)

First note that, x ∈ R′ =⇒ x ∈ R. There-fore, the inequality is satisfied if φ′x ≥ φx for all x.We prove this inductively. When a value x is firstadded to R, x 6∈ R′, hence φ′x = 1 ≥ φx. By in-duction, we assume the hypothesis for φx and showthat φ′y ≥ φy. Consider Eq. (29). If y 6∈ S, then

φy =WP ′→PR

(y)

C∗P ′→P

≤ WP ′→P (y)C∗

P ′→P

≤ 1 = φ′y by the opti-mality of C∗ and Eq. 26. If y ∈ S, we have φ′x ≥ φx

for all x by induction, proving the claim.

Evidently, the weights decrease monotonically over thecourse of the algorithm. Of particular note is the casewhen R = X and S = Y: here the acceptance ratiois again 1 and we generate exact samples from PR.Of course, |R| and |S| are bounded by the number ofiterations of the algorithm and therefore we expect sat-uration (i.e., |R| = |X|) only after exponential work.