1. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring, 2006 Graphical Models Jeff A. Bilmes Lecture 8 Slides April 20 th, 2006

2. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 2 READING: –Chapter 11,12,15 in Jordan’s book Reminder: TA discussions and office hours: –Office hours: Thursdays 3:30-4:30, Sieg Ground Floor Tutorial Center –Discussion Sections: Fridays 9:30-10:30, Sieg Ground Floor Tutorial Center Lecture Room Reminder: take-home Midterm: May 5 th -8 th, you must work alone on this. Announcements

3. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 3 L1: Tues, 3/28: Overview, GMs, Intro BNs. L2: Thur, 3/30: semantics of BNs + UGMs L3: Tues, 4/4: elimination, probs, chordal I L4: Thur, 4/6: chrdal, sep, decomp, elim L5: Tue, 4/11: chdl/elim, mcs, triang, ci props. L6: Thur, 4/13: MST,CI axioms, Markov prps. L7: Tues, 4/18: Mobius, HC-thm, (F)=(G) L8: Thur, 4/20: L9: Tue, 4/25 L10: Thur, 4/27 L11: Tues, 5/2 L12: Thur, 5/4 L13: Tues, 5/9 L14: Thur, 5/11 L15: Tue, 5/16 L16: Thur, 5/18 L17: Tues, 5/23 L18: Thur, 5/25 L19: Tue, 5/30 L20: Thur, 6/1: final presentations Class Road Map

4. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 4 L1: Tues, 3/28: L2: Thur, 3/30: L3: Tues, 4/4: L4: Thur, 4/6: L5: Tue, 4/11: L6: Thur, 4/13: L7: Tues, 4/18: L8: Thur, 4/20: Team Lists, short abstracts I L9: Tue, 4/25: L10: Thur, 4/27: short abstracts II L11: Tues, 5/2 L12: Thur, 5/4: abstract II + progress L13: Tues, 5/9 L14: Thur, 5/11: 1 page progress report L15: Tue, 5/16 L16: Thur, 5/18: 1 page progress report L17: Tues, 5/23 L18: Thur, 5/25: 1 page progress report L19: Tue, 5/30 L20: Thur, 6/1: final presentations L21: Tue, 6/6 4-page papers due (like a conference paper). Final Project Milestone Due Dates Team lists, abstracts, and progress reports must be turned in, in class and using paper (dead tree versions only). Final reports must be turned in electronically in PDF (no other formats accepted). Progress reports must report who did what so far!!

5. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 5 Factorization property on MRF, (F) When (F) = (G) = (L) = (P) inclusion-exclusion Möbius Inversion lemma Hammersley/Clifford theorem, when (G) => (F) Factorization and decomposability Factorization and junction tree Directed factorization (DF), and (G) Markov blanket Bayesian networks, moralization, and ancestral sets Summary of Last Time

6. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 6 d-Separation, (DL), (DO), and equivalence of all Markov properties on BNs. Phylogenetic Trees and Chordal Models Outline of Today’s Lecture

7. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 7 Books and Sources for Today M. Jordan: Chapters 11,12,15. Lauritzen, chapter 3. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, 1988. T. McKee “Topics in Intesection Graph Theory”

8. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 8 Preservation of (DF) in ancestral sets

9. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 9 Example (DF) – (G)

10. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 10 Example (DF) – (G)

11. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 11 d-Separation revisited

12. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 12 d-Separation revisited

13. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 13 All Together Now

14. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 14 What else can chordal graphs do?

15. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 15 Phylogenetic Tree: example species characters resulting phylogenetic tree

16. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 16 Phylogenetic Trees

17. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 17 Perfect Phylogeny

18. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 18 Phylogenetic Trees

19. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 19 Examples: G T, G I 1,1 1,2 1,3 3,1 3,2 3,3 2,1 2,3 2,2

20. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 20 Phylogenetic Trees

21. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 21 Phylogenetic Trees

22. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 22 Phylogenetic Trees

23. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 23 Intersection Graphs

24. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 24 Inference on Hidden Markov Models Hidden Markov Models (HMMs) are a ubiquitously used model in speech recognition, natural language processing, bioinformatics, financial markets, and many time-series problems. HMMs are rich enough to be interesting, but simple enough so that they are a perfect example to start with when performing exact inference. HMMs can be described either with a BN or an MRF –so this means they must be decomposable Since HMMs are already triangulated (after moralization if necessary), there is no triangulation step. Moreover, since the clique sizes are small, HMMs are easy to deal with (compexity only O(TN 2 )

25. Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 25