Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 18: May 31st, 2006 EE512 - Graphical Models - J. Bilmes Page 1

University of WashingtonDepartment of Electrical Engineering

EE512 Spring, 2006 Graphical Models

Jeff A. Bilmes <[email protected]>Jeff A. Bilmes <[email protected]>

Lecture 18 Slides

May 31st, 2006


• READING: – Google search on “bayesian networks” “approximate

inference”

• No more homework this quarter, concentrate on final projects!!

• Final project presentations tomorrow in class!!

Announcements


• L1: Tues, 3/28: Overview, GMs, Intro BNs.• L2: Thur, 3/30: semantics of BNs + UGMs• L3: Tues, 4/4: elimination, probs, chordal I• L4: Thur, 4/6: chrdal, sep, decomp, elim• L5: Tue, 4/11: chdl/elim, mcs, triang, ci props.• L6: Thur, 4/13: MST,CI axioms, Markov prps.• L7: Tues, 4/18: Mobius, HC-thm, (F)=(G)• L8: Thur, 4/20: phylogenetic trees, HMMs• L9: Tue, 4/25: HMMs, inference on trees• L10: Thur, 4/27: Inference on trees, start poly

• L11: Tues, 5/2: polytrees, start JT inference• L12: Thur, 5/4: Inference in JTs• Tues, 5/9: away• Thur, 5/11: away• L13: Tue, 5/16: JT, GDL, Shenoy-Schafer• L14: Thur, 5/18: GDL, Search, Gaussians I• L--: Mon, 5/22: laptop crash • L15: Tues, 5/23: search, Gaussians I• L16: Thur, 5/25: Gaussians• Mon, 5/29: Holiday• L17: Tue, 5/30• L18: Wed, 5/31• L19: Thur, 6/1: final presentations

• L20: Tue, 6/6

Class Road Map


• L1: Tues, 3/28: • L2: Thur, 3/30:• L3: Tues, 4/4: • L4: Thur, 4/6:• L5: Tue, 4/11:• L6: Thur, 4/13:• L7: Tues, 4/18:• L8: Thur, 4/20: Team Lists, short abstracts I• L9: Tue, 4/25:• L10: Thur, 4/27: short abstracts II• L11: Tues, 5/2:

• L12: Thur, 5/4: abstract II + progress• L--: Tues, 5/9• L--: Thur, 5/11: 1 page progress report• L13: Tue, 5/16:

• L14: Thur, 5/18: 1 page progress report• L15: Tues, 5/23• L16: Thur, 5/25: 1 page progress report• L17: Tue, 5/30: Today• L18: Wed, 5/31:• L19: Thur, 6/1: final presentations

• L20: Tue, 6/6 4-page papers due (like a conference paper), Only .pdf versions accepted.

Final Project Milestone Due Dates

• Team lists, abstracts, and progress reports must be turned in, in class and using paper (dead tree versions only).

• Final reports must be turned in electronically in PDF (no other formats accepted).

• No need to repeat what was on previous progress reports/abstracts, I have those available to refer to.

• Progress reports must report who did what so far!!


• Other forms of inference.• Structure learning in graphical models

Summary of Last Time


• When the inference gets hard …

Outline of Today’s Lecture


Books and Sources for Today

• Various sources on approximate inference (see references in presentation below).

• Papers by Yedidia, 2000,2002,2004


When exact inference is too expensive

1. Two general approaches: either an exact solution to an approximate problem, or an approximate solution to an exact problem.

2. Exact solution to approximate problem1. Structure learning: find a low tree-width (or “cheap” in some way)

graphical model that is still “high-quality” in some way, and then perform exact inference on the approximate model.

2. This can be easy or hard depending on the tree-width and on the measure of “high-quality”, and on the learning paradigm.

3. Approximate solution to an exact problem1. Approximate inference, tries to approximate in some way what

must be computed: Loopy Belief propagation, Variational/Mean-Field-Bethe/etc., and Sampling/Pruning, and hybrids between the above


Loopy BP


Recall general MP


Towards message passing MP


Loopy BP


Loopy BP


Loopy BP (or just BP)


Loopy BP


Loopy BP and variational


Loopy BP and variational


Variational inference


Variational EM


Variational EM


Variational EM


Mean-field


Bethe Free Energy


Bethe Free Energy


Bethe Free Energy

Documents

Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,