of 66 /66
Inference in Gaussian and Hybrid Bayesian Networks ICS 275B

Inference in Gaussian and Hybrid Bayesian Networks

  • Author
    halil

  • View
    46

  • Download
    0

Embed Size (px)

DESCRIPTION

Inference in Gaussian and Hybrid Bayesian Networks. ICS 275B. Gaussian Distribution. 0.4. gaussian(x,0,1). gaussian(x,1,1). 0.35. 0.3. 0.25. 0.2. 0.15. 0.1. 0.05. 0. -3. -2. -1. 0. 1. 2. 3. N( m , s ). 0.4. gaussian(x,0,1). gaussian(x,0,2). 0.35. 0.3. 0.25. 0.2. 0.15. - PowerPoint PPT Presentation

Text of Inference in Gaussian and Hybrid Bayesian Networks

  • Inference in Gaussian and Hybrid Bayesian NetworksICS 275B

  • Gaussian Distribution

  • Multivariate GaussianDefinition:Let X1,,Xn. Be a set of random variables. A multivariate Gaussian distribution over X1,,Xn is a parameterized by an n-dimensional mean vector and an n x n positive definitive covariance matrix . It defines a joint density via:

  • Multivariate Gaussian

  • Linear Gaussian DistributionDefinition:Let Y be a continuous node with continuous parents X1,,Xk. We say that Y has a linear Gaussian model if it can be described using parameters 0, ,k and 2 such that:

    P(y| x1,,xk)=N (y + 1x1 +,kxk ; ) =N([y,1,,k] , )

  • Linear Gaussian NetworkDefinitionLinear Gaussian Bayesian network is a Bayesian network all of whose variables are continuous and where all of the CPTs are linear Gaussians.

    Linear Gaussian BN Multivariate Gaussian=>Linear Gaussian BN has a compact representation

  • Inference in Continuous NetworksAB

  • Marginalization

  • Problems: When we Multiply two arbitrary Gaussians!Inverse of K and M is always well defined.However, this inverse is not!

  • Theoretical explanation: Why this is the case ?Inverse of a matrix of size n x n exists when the matrix is of rank n.If all sigmas and ws are assumed to be 1.(K-1+M-1) has rank 2 and so is not invertible.

  • Density vs conditionalHowever,Theorem: If the product of the gaussians represents a multi-variate gaussian density, then the inverse always exists.For example, For P(A|B)*P(B)=P(A,B) = N(c,C) then inverse of C always exists. P(A,B) is a multi-variate gaussian (density).But P(A|B)*P(B|X)=P(A,B|X) = N(c,C) then inverse of C may not exist. P(A,B|X) is a conditional gaussian.

  • Inference: A general algorithm Computing marginal of a given variable, say Z.Step 1:Convert all conditional gaussians to canonical form

  • Inference: A general algorithm Computing marginal of a given variable, say Z.Step 2:Extend all gs,hs and ks to the same domain by adding 0s.

  • Inference: A general algorithm Computing marginal of a given variable, say Z.Step 3: Add all gs, all hs and all ks.Step 4: Let the variables involved in the computation be: P(X1,X2,,Xk,Z)= N(,)

  • Inference: A general algorithm Computing marginal of a given variable, say Z.Step 5:Extract the marginal

  • Inference: Computing marginal of a given variable

    For a continuous Gaussian Bayesian Network, inference is polynomial O(N3).Complexity of matrix inversionSo algorithms like belief propagation are not generally used when all variables are Gaussian.Can we do better than N^3?Use Bucket elimination.

  • Bucket elimination Algorithm elim-bel (Dechter 1996)Marginalization operator

  • Multiplication OperatorConvert all functions to canonical form if necessary.Extend all functions to the same variables(g1,h1,k1)*(g2,h2,k2) =(g1+g2,h1+h2,k1+k2)

  • Again our problem!Marginalization operatorh(a,d,c,e) does not represent a density and so cannot be computed in our usual form N(,)

  • Solution: Marginalize in canonical formAlthough intermediate functions computed in bucket elimination are conditional, we can marginalize in canonical form, so we can eliminate the problem of non-existence of inverse completely.

  • AlgorithmIn each bucket, convert all functions in canonical form if necessary, multiply them and marginalize out the variable in the bucket as shown in the previous slide.Theorem: P(A) is a density and is correct.Complexity: Time and space: O((w+1)^3) where w is the width of the ordering used.

  • Continuous Node, Discrete ParentsDefinition:Let X be a continuous node, and let U={U1,U2,,Un} be its discrete parents and Y={Y1,Y2,,Yk} be its continuous parents. We say that X has a conditional linear Gaussian (CLG) CPT if, for every value uD(U), we have a a set of (k+1) coefficients au,0, au,1, , au,k+1 and a variance u2 such that:

  • CLG NetworkDefinition:A Bayesian network is called a CLG network if every discrete node has only discrete parents, and every continuous node has a CLG CPT.

  • Inference in CLGsCan we use the same algorithm?Yes, but the algorithm is unbounded if we are not careful.Reason:Marginalizing out discrete variables from any arbitrary function in CLGs is not bounded.If we marginalize out y and k from f(x,y,i,k) , the result is a mixture of 4 gaussians instead of 2.X and y are continuous variablesI and k are discrete binary variables.

  • Solution: Approximate the mixture of Gaussians by a single gaussian

  • Multiplication and MarginalizationConvert all functions to canonical form if necessary.Extend all functions to the same variables(g1,h1,k1)*(g2,h2,k2) =(g1+g2,h1+h2,k1+k2)MultiplicationStrong marginal when marginalizing continuous variablesWeak marginal when marginalizing discrete variables

  • Problem while using this marginalization in bucket eliminationRequires computing and which is not possible due to non-existence of inverse.Solution: Use an ordering such that you never have to marginalize out discrete variables from a function that has both discrete and continuous gaussian variables.Special case: Compute marginal at a discrete nodeHomework: Derive a bucket elimination algorithm for computing marginal of a continuous variable.

  • Special Case: A marginal on a discrete variable in a CLG is to be computed.Marginalization operatorB,C and D are continuous variables and A and E is discrete

  • Complexity of the special caseDiscrete-width (wd): Maximum number of discrete variables in a cliqueContinuous-width (wc): Maximum number of continuous variables in a cliqueTime: O(exp(wd)+wc^3)Space: O(exp(wd)+wc^3)

  • Algorithm for the general case:Computing Belief at a continuous node of a CLGConvert all functions to canonical form.Create a special tree-decompositionAssign functions to appropriate cliques (Same as assigning functions to buckets)Select a Strong RootPerform message passing

  • Creating a Special-tree decompositionMoralize the Bayesian Network.Select an ordering such that all continuous variables are ordered before discrete variables (Increases induced width).

  • Elimination orderwyxzStrong elimination order:First eliminate continuous variablesEliminate discrete variable when no available continuous variablesMoralized graph has this edgeW and X are discrete variables and Y and Z are continuous.

  • Elimination order (1)wyxzdim: 2dim: 2dim: 21

  • Elimination order (2)wyxzdim: 2dim: 221

  • Elimination order (3) wyxz3dim: 221

  • Elimination order (4)wyxz3421wyz321wyx342wy32Cliques 1Cliques 2separator

  • Bucket tree or Junction tree (1)wyzwyxwyCliques 1Cliques 2: rootseparator

  • Algorithm for the general case:Computing Belief at a continuous node of a CLGConvert all functions to canonical form.Create a special tree-decompositionAssign functions to appropriate cliques (Same as assigning functions to buckets)Select a Strong RootPerform message passing

  • Assigning Functions to cliquesSelect a function and place it in an arbitrary clique that mentions all variables in the function.

  • Algorithm for the general case:Computing Belief at a continuous node of a CLGConvert all functions to canonical form.Create a special tree-decompositionAssign functions to appropriate cliques (Same as assigning functions to buckets)Select a Strong RootPerform message passing

  • Strong RootWe define a strong root as any node R in the bucket-tree which satisfies the following property: for any pair (V,W) which are neighbors on the tree with W closer to R than V, we have

  • Example Strong rootStrong Root

  • Algorithm for the general case:Computing Belief at a continuous node of a CLGCreate a special tree-decompositionAssign functions to appropriate cliques (Same as assigning functions to buckets)Select a Strong RootPerform message passing

  • Message passing at a typical nodex2Node a contains functions assigned to it according to the tree-decomposition scheme denoted by pj(a)

  • Message PassingFigure from P. GreenTwo pass algorithm: Bucket-tree propagation

  • Lets look at the messagesCollect EvidenceC LMoutMinDDStrong Root

  • Distribute EvidenceEW,B EW,BWEBFStrong Root

  • Lauritzens theoremWhen you perform message passing such that collect evidence contains only strong marginals and distribute evidence may contain weak marginals, the junction-tree algorithm in exact in the sense that:The first (mean) and second moments (variance) computed are true moments

  • ComplexityPolynomial in #of continuous variables in a clique (n3)Exponential in #of discrete variables in a cliquePossible options for approximationIgnore the strong root assumption and use approximation like MBTE, IJGP, SamplingRespect the strong root assumption and use approximation like MBTE, IJGP, SamplingInaccuracies only due to discrete variables if done in one pass of MBTE.

  • Initialization (1)wyxzdim: 2dim: 2dim: 2dim: 2

    W=0W=1

    X=0X=1

    w=00.5w=10.5

    x=00.4x=10.6

  • Initialization (2)wyzwxywyCliques 1Cliques 2 (root)

    w=0g=log(0.5),h=[],K=[]w=1g=log(0.5),h=[],K=[]

    x=0g=log(0.4),h=[],K=[]x=1g=log(0.6),h=[],K=[]

    X=0X=1g = -4.1245h = [-0.02 0.12]K = [0.1 0; 0 0.1]g = -3.0310h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]

    W=0W=1g = -4.0629h = [0.0889 -0.0111 -0.0556 0.0556]K = g = -2.7854h = [0.0867 -0.0633 -0.1000 -0.1667]K =

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=1

    z=0z=1

    y=00.350.15

    y=10.10.25

    y=30.050.1

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    Sheet2

    Sheet3

  • Initialization (3)wyzwxywyCliques 1Cliques 2 (root)empty

    W=0W=1g = -4.7560h = K = g = -3.4786h = K =

    wx=00wx=10g = -5.1308h = [-0.02 0.12]K = [0.1 0; 0 0.1]g = -5.1308h = [-0.02 0.12] K = [0.1 0; 0 0.1]

    wx=01wx=11g = -3.5418h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]g = -3.5418h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    Sheet2

    Sheet3

  • Message PassingwyzwxywyCliques 1Cliques 2 (root)emptyCollect evidenceDistribute evidence

  • Collect evidence (1)wyzwxywyCliques 1Cliques 2 (root)emptyy2y3y1y2y2(y1,y2)(y2)

  • Collect evidence (2)wyzwxywyCliques 1Cliques 2 (root)emptymarginalization

    W=0W=1g = -4.7560h = K = g = -3.4786h = K =

    W=0W=1g = -0.6931h = [0.1388 0] *1.0e-16K = [0.2776 -0.0694;0.0347 0]*1.0e-16g = -0.6931h = [0 0]K = [0 0 0 0]

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    Sheet2

    Sheet3

  • Collect evidence (3)wyzwxywyCliques 1Cliques 2 (root)emptymultiplication

    W=0W=1g = -0.6931h = [0.1388 0] *1.0e-16K = [0.2776 -0.0694;0.0347 0]*1.0e-16g = -0.6931h = [0 0]K = [0 0 0 0]

    wx=00wx=10g = -5.1308h = [-0.02 0.12]K = [0.1 0; 0 0.1]g = -5.1308h = [-0.02 0.12] K = [0.1 0; 0 0.1]

    wx=01wx=11g = -3.5418h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]g = -3.5418h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]

    wx=00wx=10g = -5.8329h = [-0.02 0.12]K = [0.1 0; 0 0.1]g = -5.8329h = [-0.02 0.12] K = [0.1 0; 0 0.1]

    wx=01wx=11g = -4.2350h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]g = -4.2350h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]

  • Distribute evidence (1)wyzwxywyCliques 1Cliques 2 (root)division

    W=0W=1g = -4.7560h = K = g = -3.4786h = K =

    W=0W=1g = -0.6931h = [0.1388 0] *1.0e-16K = [0.2776 -0.0694;0.0347 0]*1.0e-16g = -0.6931h = [0 0]K = [0 0 0 0]

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    Sheet2

    Sheet3

  • Distribute evidence (2)wyzwxywyCliques 1Cliques 2 (root)

    W=0W=1g = -4.0629h = K = g = -2.7854h = K =

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    Sheet2

    Sheet3

  • Distribute evidence (3)wyzwxywyCliques 1Cliques 2 (root)Marginalize over x

    wx=00wx=10g = -5.8329h = [-0.02 0.12]K = [0.1 0; 0 0.1]g = -5.8329h = [-0.02 0.12] K = [0.1 0; 0 0.1]

    wx=01wx=11g = -4.2350h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]g = -4.2350h = [0.5 -0.5]K = [0.5 0.5;0.5 0.5]

    w=0w=1logp = -0.6931mu = [0.52 -0.12]Sigma =logp = -0.6931mu = [0.52 -0.12]Sigma =

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    Sheet2

    Sheet3

  • Distribute evidence (4)wyzwxywyCliques 1Cliques 2 (root)multiplicationCanonical form

    W=0W=1g = -4.0629h = K = g = -2.7854h = K =

    w=0w=1logp = -0.6931mu = [0.52 -0.12]Sigma =logp = -0.6931mu = [0.52 -0.12]Sigma =

    w=0w=1g = -4.3316h = [0.0927 -0.0096]K =g = -0.6931h = [0.0927 -0.0096]K =

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    0.18240.0182

    0.01820.159

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    0.18240.0182

    0.01820.159

    Sheet2

    Sheet3

  • Distribute evidence (5)wyzwxywyCliques 1Cliques 2 (root)

    W=0W=1g = -8.3935h = K = g = -7.1170h = K =

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    0.18240.0182

    0.01820.159

    0.1816-0.0207-0.05560.0556

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    0.18240.0182

    0.01820.159

    0.1816-0.0207-0.05560.0556

    0.32680.0093-0.10.0778

    0.00930.1968-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    0.18240.0182

    0.01820.159

    0.1816-0.0207-0.05560.0556

    0.32680.0093-0.10.0778

    0.00930.1968-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.1793-0.073-0.1-0.1667

    Sheet2

    Sheet3

    Sheet1

    x=0

    z=0z=1x=0x=1

    y=00.10.5y=00.350.15

    y=10.80.6y=10.10.25

    y=30.70.4y=30.050.1

    x=1w=00.5

    z=0z=1w=10.5

    y=00.90.5

    y=10.20.4

    y=30.30.6

    x=00.1444-0.0089-0.10.0778

    z=0z=1-0.00890.0378-0.0333-0.0556

    y=00.350.15-0.1-0.03330.11110

    y=10.10.250.0778-0.055600.1111

    y=30.050.1

    x=10.2083-0.14670.15-0.2333

    z=0z=1-0.14670.1033-0.10.1667

    y=00.350.150.15-0.10.50

    y=10.10.25-0.23330.166700.3333

    y=30.050.1

    0.0889-0.0111-0.05560.0556

    0.1444-0.0089-0.10.0778

    -0.00890.0378-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.0867-0.0633-0.1-0.1667

    5.5456-0.6336

    -0.63366.3616

    0.18240.0182

    0.01820.159

    0.1816-0.0207-0.05560.0556

    0.32680.0093-0.10.0778

    0.00930.1968-0.0333-0.0556

    -0.1-0.03330.11110

    0.0778-0.055600.1111

    0.1793-0.073-0.1-0.1667

    0.3907-0.12850.15-0.2333

    -0.12850.2623-0.10.1667

    0.15-0.10.50

    -0.23330.166700.3333

    Sheet2

    Sheet3

  • After Message Passingp(wyz)p(wxy)p(wy)Cliques 1Cliques 2 (root)Local marginal distributions

    A Gaussian distribution is specified by two real numbers mu and sigma and is usually written as N(mu,sigma) and the density of the Gaussian distribution is given by the following function.On Y axis is the probability values and on the X axis are the values that x can take.