1. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring, 2006 Graphical Models Jeff A. Bilmes Lecture 7 Slides April 18 th, 2006

2. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 2 If you see a typo, please tell me during lecture –everyone will then benefit. –note, corrected slides will go on web. READING: –Chapter 3 & 17 in Jordan’s book –Lauritzen chapters 1-3 (on reserve in library) –Möbius Inversion Lemma handout (to be on web site) Reminder: TA discussions and office hours: –Office hours: Thursdays 3:30-4:30, Sieg Ground Floor Tutorial Center –Discussion Sections: Fridays 9:30-10:30, Sieg Ground Floor Tutorial Center Lecture Room Reminder: take-home Midterm: May 5 th -8 th, you must work alone on this. Announcements

3. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 3 L1: Tues, 3/28: Overview, GMs, Intro BNs. L2: Thur, 3/30: semantics of BNs + UGMs L3: Tues, 4/4: elimination, probs, chordal I L4: Thur, 4/6: chrdal, sep, decomp, elim L5: Tue, 4/11: chdl/elim, mcs, triang, ci props. L6: Thur, 4/13: MST,CI axioms, Markov prps. L7: Tues, 4/18: Mobius, HC-thm, (F)=(G) L8: Thur, 4/20 L9: Tue, 4/25 L10: Thur, 4/27 L11: Tues, 5/2 L12: Thur, 5/4 L13: Tues, 5/9 L14: Thur, 5/11 L15: Tue, 5/16 L16: Thur, 5/18 L17: Tues, 5/23 L18: Thur, 5/25 L19: Tue, 5/30 L20: Thur, 6/1: final presentations Class Road Map

4. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 4 L1: Tues, 3/28: L2: Thur, 3/30: L3: Tues, 4/4: L4: Thur, 4/6: L5: Tue, 4/11: L6: Thur, 4/13: L7: Tues, 4/18: Today L8: Thur, 4/20: Team Lists, short abstracts I L9: Tue, 4/25: L10: Thur, 4/27: short abstracts II L11: Tues, 5/2 L12: Thur, 5/4: abstract II + progress L13: Tues, 5/9 L14: Thur, 5/11: 1 page progress report L15: Tue, 5/16 L16: Thur, 5/18: 1 page progress report L17: Tues, 5/23 L18: Thur, 5/25: 1 page progress report L19: Tue, 5/30 L20: Thur, 6/1: final presentations L21: Tue, 6/6 4-page papers due (like a conference paper). Final Project Milestone Due Dates Team lists, abstracts, and progress reports must be turned in, in class and using paper (dead tree versions only). Final reports must be turned in electronically in PDF (no other formats accepted). Progress reports must report who did what so far!!

5. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 5 when are trees of maxcliques JTs? max/min spanning trees conditional independence relations logical axioms of conditional independence relations axioms and positivity independence and knowledge independence and separation completeness conjecture Markov properties on MRFs, (G),(L),(P) Summary of Last Time

6. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 6 Factorization property on MRF, (F) When (F) = (G) = (L) = (P) inclusion-exclusion Möbius Inversion lemma Hammersley/Clifford theorem, when (G) => (F) Factorization and decomposability Factorization and junction tree Directed factorization (DF), and (G) Markov blanket Bayesian networks and moralization Outline of Today’s Lecture

7. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 7 Books and Sources for Today M. Jordan: Chapters 17. S. Lauritzen, 1996. Chapters 1-3. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, 1988. Any good graph theory text.

8. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 8 Properties of Markov Properties

9. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 9 Markov Properties of Graphs

10. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 10 Properties of Markov Properties

11. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 11 (F) Factorization Property

12. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 12 The alphabetical theorem: (F)  (G)  (L)  (P)

13. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 13 The alphabetical theorem: (F)  (G)  (L)  (P)

14. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 14 The equivalence theorem: (F)  (G)  (L)  (P)

15. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 15 Inclusion-Exclusion

16. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 16 Möbius Inversion Lemma

17. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 17 Möbius Inversion Lemma

18. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 18 Hammersley/Clifford

19. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 19 Hammersley/Clifford

20. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 20 Hammersley/Clifford

21. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 21 Hammersley/Clifford

22. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 22 Hammersley/Clifford

23. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 23 Hammersley/Clifford by pairwise Markov property since we have unity ratios pairwise Markov property and chain rule

24. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 24 Hammersley/Clifford

25. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 25 Factorization and decomposability

26. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 26 Factorization and decomposability

27. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 27 (G), factorization, and decomposability

28. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 28 Recursive application + positivity

29. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 29 Recursive application + positivity

30. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 30 (DF)

31. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 31 (DF) and (G)

32. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 32 Markov Blanket

33. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 33 Recall from Lecture 3: Ancestral Sets

34. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 34 Preservation of (DF) in ancestral sets

35. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 35 Example (DF) – (G)

36. Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 36 Example (DF) – (G)