MURI Progress Report, June 2001 Advances in Approximate and Hybrid Reasoning for Decision Making Under Uncertainty Rina Dechter UC- Irvine Collaborators:

MURI Progress Report, June 2001

Advances in Approximate and Hybrid Reasoning for Decision Making Under Uncertainty

Rina DechterUC- Irvine

Collaborators:Kalev Kask,Javier Larrosa,David Larkin,Robert Mateescu


Summary of Results

Mini-clustering: a universal anytime approximation scheme. Applied to probabilistic inference and to Optimization, decision making tasks

Hybrid processing of beliefs and constraints

REES: Reasoning Engine Evaluation Shell.

Online algorithms (S. Irani)


Outline

Mini-clustering approximation; approximation by partitioning, a universal anytime scheme Applied to probabilistic inference Applied to Decision Optimization tasks

Hybrid processing of beliefs and constraints

REES: Reasoning Engine Evaluation Shell. Online algorithms (S. Irani)


Mini-Clustering :Approximation by partitioning

Past work: Mini-bucket approximation for variable elimination Applied to optimization Used for static heuristic generation for search Experiments with coding tasks, medical diagnosis

Progress this year Mini-clustering approximation of tree-clustering Applied to Belief updating Applied to optimization and search


Motivation

Decision-making algorithms are all too complex (NP-Hard).

The main bottleneck is probabilistic inference: determining the posterior beliefs given evidence to help forming the right decision.

Consequently, approximate, anytime methods are essential to assist in advise-giving for decision making.


Automated reasoning Tasks

i iri Z

Y

Y S Y S Y S Y SY

i i i i i i i i

i i

i r

n

t

Z f

n inalizatio m X, , Y f

S f f f f f

n combinatio f f f f

f X f scope

f f f F

D D D

n X

} ,...,Z ,{Z , X,D,F, P task reasoning automated

i all for compute to is problem The 6.

operator arg a is and function

of scope the is where , } , , min , max { 5.

operator a is } , , { 4.

of arguments of set the is , ) ( denoted

, function of scope The functions. of set a is } ,..., { 3.

domains finite of set a is } ,..., { 2.

variables of set a is } ,..., 1{ 1.

: follows as defined

, sixtuple a is An

1

1

1

1


A Reasoning problem Graph

A

B

D

F

C

G

ABDf

ABDPf

i

i

),|(

Belief updating: y = X-y j Pj

MPE: = maxX j Pj

CSP: = X j Cj

Max-CSP: = minX j Fj


2 },|(),|({)2(},,,{)2(

dcfpbdpFDCB

4)},|({)4(

},,{)4(

fegp

GFE

3)},|({)3(

},,{)3(

fbep

FEB

)},|(),|(),({)1(},,{)1(

bacpabpapCBA

1

G

E

F

C D

B

A

)p(b|a

)p(a

),| bap(c

),dp(f|c

)P(d|b

),| fbp(e

), fp(g|e

Tree Decomposition


ABC

2

4

),|()|()(),()2,4( bacpabpapcbha

1

3BEF

EFG

),(),|()|(),( )2,3(,

)1,2( fbhdcfpbdpcbhfd

),(),|()|(),( )2,4(,

)3,2( cbhdcfpbdpfbhdc

),(),|(),( )3,4()2,3( fehfbepfbhe

),(),|(),( )3,2()4,3( fbhfbepfehb

),|(),()3,4( fegGpfeh e

EF

BF

BC

BCDF

G

E

F

C D

B

A

Cluster Tree Elimination(join-tree clustering)


Time complexity: Exponential in the induced-width

O (N dw*+1 )

Space complexity: Exponential in the separator O ( N dsep)

Tree clustering Complexity


Idea of Mini-clustering

Reduce the exponent (i.e. size of the cluster); partition into mini-clusters.

Accuracy-control parameter z = maximum number of variables in a mini-cluster

The idea was explored for variable elimination (Mini-Bucket)


Idea of Mini-clustering

Split a cluster into mini-clusters =>bound complexity

XX gh )()()O(e :decrease complexity lExponentia n rnr eOeO


ABC

2

4

3

BEF

EFG

EF

BF

BC

BCDF

),|()|()(:),(1)2,1( bacpabpapcbh

a

)2,1(H

),|(max:)(

),()|(:)(

,

2)1,2(

1)2,3(

,

1)1,2(

dcfpch

fbhbdpbh

fd

fd

)1,2(H

),|(max:)(

),()|(:)(

,

2)3,2(

1)2,1(

,

1)3,2(

dcfpfh

cbhbdpbh

dc

dc

)3,2(H

),(),|(:),( 1)3,4(

1)2,3( fehfbepfbh

e

)2,3(H

)()(),|(:),( 2)3,2(

1)3,2(

1)4,3( fhbhfbepfeh

b

)4,3(H

),|(:),(1)3,4( fegGpfeh e)3,4(H

MC(3) algorithm - example


ABC

2

4

),()2,4( cbh

1

3BEF

EFG

),()1,2( cbh

),()3,2( fbh

),()2,3( fbh

),()4,3( feh

),()3,4( feh

EF

BF

BC

BCDF

),(1)2,1( cbh

)(

)(2

)1,2(

1)1,2(

ch

bh

)(

)(2

)3,2(

1)3,2(

fh

bh

),(1)2,3( fbh

),(1)4,3( feh

),(1)3,4( feh

)2,1(H

)1,2(H

)3,2(H

)2,3(H

)4,3(H

)3,4(H

ABC

2

4

1

3BEF

EFG

EF

BF

BC

BCDF

Tree-clustering vs Mini-clustering


Properties of MC(z)

MC(z) computes a bound on the joint probability P(X,e) of each variable and each of its values.

Time & space complexity: O(n hw* exp(z))

Lower, Upper bounds and Mean approximations

Approximation improves with z but takes more time


Experiments Algorithms:

Exact IBP Gibbs sampling (GS) Mini-Clustering (MC(z))

Networks: Probabilistic Decoding networks Medical diagnosis: CPCS 54 Random noisy-OR networks Random networks


0|e|=10 max mean max mean max mean max mean

20

0.01852 0.00032 0.00064 2.450IBP 0.15727 0.03307 0.07349 2.191

0.20765 0.05934 0.14202 1.5610.49444 0.07797 0.18034 17.247

GS 0.51409 0.09002 0.21298 17.2080.48706 0.10608 0.26853 17.335

0.16667 0.07407 0.02722 0.01221 0.05648 0.02520 0.154 0.153MC(2) 0.11636 0.07636 0.02623 0.01843 0.05581 0.03943 0.096 0.095

0.10529 0.07941 0.02876 0.02196 0.06357 0.04878 0.067 0.0670.18519 0.09259 0.02488 0.01183 0.05128 0.02454 0.157 0.155

MC(5) 0.10727 0.07682 0.02464 0.01703 0.05239 0.03628 0.112 0.1120.08059 0.05941 0.02174 0.01705 0.04790 0.03778 0.090 0.0870.12963 0.07407 0.01487 0.00619 0.03047 0.01273 0.438 0.446

MC(8) 0.06591 0.05000 0.01590 0.01040 0.03394 0.02227 0.369 0.3700.03235 0.02588 0.00977 0.00770 0.02165 0.01707 0.292 0.2940.11111 0.07407 0.01133 0.00688 0.02369 0.01434 2.038 2.032

MC(11) 0.02818 0.01500 0.00600 0.00398 0.01295 0.00869 1.567 1.5710.00353 0.00353 0.00124 0.00101 0.00285 0.00236 0.867 0.869

NHD Absolute Error Relative Error Time

Performance on CPCS54 w*=15



20

0 9.0E-09 1.1E-05 0.102IBP 0 3.4E-04 4.2E-01 0.081

0 9.6E-04 1.2E+00 0.0620.51 5.0E-01 5.9E+02 12.976

GS 0.52 5.0E-01 5.9E+02 13.1600.51 5.0E-01 6.0E+02 12.976

0 0 1.6E-03 1.1E-03 1.9E+00 1.3E+00 0.056 0.057MC(2) 0 0 1.1E-03 8.4E-04 1.4E+00 1.0E+00 0.048 0.049

0 0 5.7E-04 4.8E-04 7.1E-01 5.9E-01 0.039 0.0390 0 1.1E-03 9.4E-04 1.4E+00 1.2E+00 0.070 0.072

MC(5) 0 0 7.7E-04 6.9E-04 9.3E-01 8.4E-01 0.063 0.0660 0 2.8E-04 2.7E-04 3.5E-01 3.3E-01 0.058 0.0570 0 3.6E-04 3.2E-04 4.4E-01 3.9E-01 0.214 0.221

MC(8) 0 0 1.7E-04 1.5E-04 2.0E-01 1.9E-01 0.184 0.1900 0 3.5E-05 3.5E-05 4.3E-02 4.3E-02 0.123 0.127


N=50, P=2, w*=10

Noisy-OR Networks 1



20

0.03652 0.00907 0.01894 0.298IBP 0.25200 0.08319 0.22335 0.240

0.34000 0.13995 0.91671 0.1830.17304 0.04377 0.09395 0.140

MC(2) 0.17600 0.11600 0.05930 0.04558 0.14706 0.11034 0.100 0.1030.15067 0.14000 0.07658 0.06683 0.23155 0.19538 0.075 0.0780.15652 0.04380 0.09398 0.158

MC(5) 0.15600 0.11800 0.05665 0.04320 0.13484 0.10221 0.124 0.1290.09467 0.09467 0.05545 0.05049 0.15000 0.13706 0.105 0.1070.16783 0.04166 0.08904 0.602

MC(8) 0.09800 0.08100 0.04051 0.03254 0.09923 0.07942 0.481 0.4910.05467 0.04533 0.02939 0.02691 0.07865 0.07237 0.385 0.3930.12087 0.03076 0.06550 2.986

MC(11) 0.05500 0.04700 0.02425 0.01946 0.05644 0.04533 2.307 2.3450.00800 0.00533 0.00483 0.00431 0.01307 0.01156 1.564 1.5850.06348 0.01910 0.04071 14.910

MC(14) 0.01400 0.01200 0.00542 0.00434 0.01350 0.01108 8.548 8.5780.00000 0.00000 0.00089 0.00089 0.00212 0.00211 3.656 3.676


N=50, P=3, w*=16

Random Networks 2


Outline

Mini-clustering approximation; approximation by partitioning, a universal anytime scheme Applied to probabilistic inference Applied to Optimization and decision-making

tasks Hybrid processing of beliefs and constraints REES: Reasoning Engine Evaluation Shell. Online algorithms (S. Irani)


Constraint Optimization for Decision-making (COP)

Global optimization: Find the best cost assignment subject

to constraints

Singleton optimality: Find the best cost-extension for every

singleton variable-value assignment (X,a).


5

2

1

3

4

)}6,1(),6,5({)6(}6,5,1{)6(

21 ff

6

)}5,2({)5(}5,2,1{)5(

3f

)}2,1({)2(}2,1{)2(

6f

)}4,1({)4(}4,1{)4(

4f

)}3,2({)3(}3,2{)3(

5f

Ø)1(}1{)1(

3

1

26

5

2

1

6

4

35

(a) (b) (c)

4

Example : COP

Cij = Xi Xj

Tree-width = 3sep(5,6) = {1, 5}


From Mini-bucket elimination to Mini-Bucket Tree Elimination

},{

}6,5,1{

21 ff

}{

}5,2,1{

3f

}{

}2,1{

6f

5

2

1

}{

}3,2{

5f

3

}{

}4,1{

4f

4

}1{

)}6,1(

)6,5({min:)5,1(

2

16)5,6(

f

fm

6

)}2,1(

)5,2({min:)5,1(

)5,2(

32)6,5(

m

fm

)}5,1(

)5,2({min:)2,1(

)5,6(

35)2,5(

m

fm

)1()2(

)2,1(:)2,1(

)2,1()2,3(

6)5,2(

mm

fm

)}2()2,1(

)2,1({min:)1(

)2,3()2,5(

62)1,2(

mm

fm

)1(:)1( )1,4()2,1( mm

)}3,2({min:)2( 53)2,3( fm

)}1()2,1(

)2,1({min:)2(

)2,1()2,5(

61)3,2(

mm

fm

)}4,1({min:)1( 44)1,4( fm

)1(:)1( )1,2()4,1( mm

},{

}6,5,1{

21 ff

}{

}5,2,1{

3f

}{

}2,1{

6f

5

2

1

}{

}3,2{

5f

3

}{

}4,1{

4f

4

}1{

)}6,1({min:)1(

)}6,5({min:)5(

262

)5,6(

161

)5,6(

fm

fm

6

)}5,2({min:)5,1(

)}2()2,1({min:)5,1(

)1(:)1(

323

)6,5(

2)5,2(

1)5,2(2

2)6,5(

3)5,2(

1)6,5(

fm

mmm

mm

)1(:)2,1(

)}5()5,2({min:)2,1(2

)5,6(2

)2,5(

1)5,6(35

1)2,5(

mm

mfm

)}1(:)2,1(

)2(:)2,1(

)2,1(:)2,1(

1)2,1(

3)5,2(

1)2,3(

2)5,2(

61

)5,2(

mm

mm

fm

)1(:)1(

)}2()2(

)2,1({min:)1(

2)2,5(

2)1,2(

1)2,3(

1)2,5(

621

)1,2(

mm

mm

fm

)1(:)1( 1)1,4(

1)2,1( mm

)5,6(M

)6,5(M

)5,2(M

)1,2(M

)2,1(M

)}3,2({min:)2( 531

)2,3( fm

)2(:)2(

)}1()1(

)2,1({min:)2(

1)2,5(

2)3,2(

1)2,1(

2)2,5(

611

)3,2(

mm

mm

fm

)3,2(M

)}4,1({min:)1( 441

)1,4( fm )1,4(M

)1()1(:)1( 2)1,2(

1)1,2(

1)4,1( mmm )4,1(M

)2,5(M

)2,3(M


Branch and Bound with lower bound Heuristics

BBMB(z), the earlier algorithm: Heuristic, computed by MB(z), is static,

variable ordering fixed.

BBBT(z), the new algorithm: Lower bound is computed at each node of

the search by MC(z). Used for dynamic variable and value

ordering.


BBBT(z) vs. BBMB(z)

BBBT(z) vs BBMB(z), N=50


BBBT(z) vs. BBMB(z).

BBBT(z) vs BBMB(z), N=100


Conclusion

Mini-clustering, MC(z) extends partition-based approximation from mini-buckets to tree decompositions.

For Probabilistic inference:

For Optimization and decision-making tasks

Empirical evaluation demonstrates its effectiveness and superiority (for certain types of problems).


Outline

Mini-clustering approximation; approximation by partitioning, a universal anytime scheme Applied to probabilistic inference Applied to Optimization and decision tasks

Processing beliefs and constraints REES: Reasoning Engine Evaluation

Shell. Online algorithms (S. Irani)


Task A: Representation and Integration of Uncertain Information

Challenges: Coherent and efficient extension of Bayesian networks to accommodate diverse types of information.

Subtasks: Constraint-based information Temporal information Incomplete information


Motivation

Complex queries for war scenarios:

What is the probability that either plan1 or plan2 hit the target, when plan2 or plan 3 can divert enemy fire, under bad weather or poor communication.

Observing that the enemy fire is coming either from direction 1 or direction 2, when direction 1 implies ground fire, what is the likelihood of being hit.


Hybrid Processing Beliefs and Constraints

Hybrid deterministic and probabilistic Information

Complex queries:

Complex evidence structure

All reduce to propositional queries over a Belief network.

1)0|1(,, ACPFDG

?)(

)()(

P

BDDG

?)|( XP


Hybrid (continued)

Deterministic queries and information can be handled as Conditional Probability Tables (CPTs)

Drawbacks: computational properties such as constraint propagation and unit resolution are not exploited.

Target: to exploit constraint processing whenever possible


A Hybrid Belief Network

D

G

A

B C

F

101 )|aP(c

FDG

Belief network P(g,f,d,c,b,a)=P(g|f,d)P(f|c,b)P(d|b,a)P(b|a)P(c|a)P(a)

Bucket G: P(G|

F,D)

Bucket F: P(F|B,C)

Bucket D: P(D|A,B)

Bucket C: P(C|A)

Bucket B: P(B|A)

Bucket A: P(A)

),,( CBAD

)(AC

),,( DCBF

),( BAB

),|0( DFGP

G

)|( GAP


),,( BAD

D

Bucket G: P(G|F,D)

Bucket F: P(F|B,C)

Bucket D: P(D|A,B)

Bucket C: P(C|A)

Bucket B: P(B|A)

Bucket A: P(A)

GGDFGFGD ),)()((

(a) regular Elim-CPE

Bucket G: P(G|

F,D)

Bucket F: P(F|B,C)

Bucket D: P(D|A,B)

Bucket C: P(C|A)

Bucket B: P(B|A)

Bucket A: P(A)

),,( CBAD

)(AC

),,( DCBF

),( BAB

),|0( DFGP

G

)|( GAP

(b) Elim-CPE-D with clause extraction

Variable elimination for a hybrid network:

)( ),|0( FDFGP

)( D

)(AB

)|( GAP

C)(A C)(B,F

)(DF

),( BAC


Empirical evaluation

Elim-CPE

Elim-Hidden model clauses as CPT with hidden variables

Elim-CPE-D extracts clauses from deterministic CPT’s

Benchmarks: Insurance and Hailfinder networks Random networks


test instances of the insurance network with query parameters < 15, 5 >

Insurance Network


48 test instances with network parameters < 80, 4, 75 > and query parameters < 0, 10 >

Elim-CPE vs. Elim-CPE-D


50 test instances, network parameters of < 50, 5, 0 > and query parameters < 50, 15 >

Averages over 35 test instances, network parameters of < 40, 5, 0 > and query parameters < 60, 10 >

Elim-CPE vs. Elim-Hidden


Conclusion

Elim-CPE: an extended variable elimination algorithm exploiting both constraints and probabilities

Empirical evaluation demonstrate Elim-CPE highly more effective than regular algorithms (Elim-Hidden)

Elim-CPE-D, extracting deterministic information from BN, improves performance and becomes more significant as deterministic information grows.


Outline





REES: Reasoning Engine Evaluation Shell

Generalizable and Customizable: Consistent handling of reasoning tasks Handles manually and randomly generated

problems with same user interface Add your own network types Use your own calculating engine Not limited by present AI problem types

Created by Kyle Bolen and Kalev KaskUnder direction of Dr. Rina Dechter


Interface Allows For:

Easy parameter entry

Quick access to choices

Simple selection process


Customize To:

Include only what you need

Output to a file Run multiple

instances Run multiple

algorithms


Understand The Results

Easily compare different algorithms

View only the output you want


Outline





Online Load Balancing with Multiple Resources, S. Irani

Tasks arrive in time and must be assigned to a server/agent as they arrive Each task requires a known amount of each

resource. Goal is to make assignments so that all

resources are evenly balanced among agents Results

Online algorithm whose performance within 2r of optimal. (r = number of resources)


Dynamic Vehicle Routing

Requests for service arrive at specific locations over a given area.

Each request has a deadline A single server travels between location

servicing requests Plan route of vehicle to maximize

number of requests satisfied by deadline.

Progress report for Sandy Irani


Dynamic Vehicle Routing

Results: Two different online algorithms developed

whose performance is provably close to optimal. (Which is better depends on parameters of the system)

Lower bounds showing algorithms within a constant of best online algorithms.

Progress report for Sandy Irani


Summary




Documents

MURI Progress Report, June 2001 Advances in Approximate and Hybrid Reasoning for Decision Making Under Uncertainty Rina Dechter UC- Irvine Collaborators: