Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Slides:

Advertisements

Similar presentations

1 Undirected Graphical Models Graphical Models – Carlos Guestrin Carnegie Mellon University October 29 th, 2008 Readings: K&F: 4.1, 4.2, 4.3, 4.4,

Advertisements

Exact Inference. Inference Basic task for inference: – Compute a posterior distribution for some query variables given some observed evidence – Sum out.

Variational Methods for Graphical Models Micheal I. Jordan Zoubin Ghahramani Tommi S. Jaakkola Lawrence K. Saul Presented by: Afsaneh Shirazi.

Bayesian Networks, Winter Yoav Haimovitch & Ariel Raviv 1.

Bayes Networks Markov Networks Noah Berlow. Bayesian -> Markov (Section 4.5.1) Given B, How can we turn into Markov Network? The general idea: – Convert.

CSE 5522: Survey of Artificial Intelligence II: Advanced Techniques Instructor: Alan Ritter TA: Fan Yang.

Chapter 15 Probabilistic Reasoning over Time. Chapter 15, Sections 1-5 Outline Time and uncertainty Inference: ltering, prediction, smoothing Hidden Markov.

TERM REVIEW. Class Announcements  Three P’s Dead see do-b-schools-still-teach-the-famed-4ps-of-marketing-when-three-

Overview of Inference Algorithms for Bayesian Networks Wei Sun, PhD Assistant Research Professor SEOR Dept. & C4I Center George Mason University, 2009.

CS774. Markov Random Field : Theory and Application Lecture 06 Kyomin Jung KAIST Sep

Lec 1: March 28th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 5: April 11th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 12: May 4th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 11: May 2nd, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 2: March 30th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

COMS W1004 Introduction to Computer Science June 25, 2008.

Lec 6: April 13th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 3: April 4th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 9: April 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 16: May 25th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

5/25/2005EE562 EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005.

CS 188: Artificial Intelligence Spring 2007 Lecture 14: Bayes Nets III 3/1/2007 Srini Narayanan – ICSI and UC Berkeley.

Lec 17: May 30th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 10: April 26th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 4: April 6th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 7: April 18th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Bayesian Networks Alan Ritter.

CPSC 422, Lecture 18Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 18 Feb, 25, 2015 Slide Sources Raymond J. Mooney University of.

. Introduction to Bayesian Networks Instructor: Dan Geiger Web page:

CSE 590ST Statistical Methods in Computer Science Instructor: Pedro Domingos.

CS 188: Artificial Intelligence Fall 2009 Lecture 19: Hidden Markov Models 11/3/2009 Dan Klein – UC Berkeley.

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 14: May 18th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

CIS 410/510 Probabilistic Methods for Artificial Intelligence Instructor: Daniel Lowd.

1 Bayesian Networks Chapter ; 14.4 CS 63 Adapted from slides by Tim Finin and Marie desJardins. Some material borrowed from Lise Getoor.

Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

Lec 15: May 22nd, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

CSE 515 Statistical Methods in Computer Science Instructor: Pedro Domingos.

Computing & Information Sciences Kansas State University Lecture 28 of 42 CIS 530 / 730 Artificial Intelligence Lecture 28 of 42 William H. Hsu Department.

QUIZ!!  T/F: The forward algorithm is really variable elimination, over time. TRUE  T/F: Particle Filtering is really sampling, over time. TRUE  T/F:

Probabilistic Graphical Models David Madigan Rutgers University

EE462 MLCV Lecture (1.5 hours) Segmentation – Markov Random Fields Tae-Kyun Kim 1.

1 Variable Elimination Graphical Models – Carlos Guestrin Carnegie Mellon University October 11 th, 2006 Readings: K&F: 8.1, 8.2, 8.3,

Introduction to Bioinformatics Biostatistics & Medical Informatics 576 Computer Sciences 576 Fall 2008 Colin Dewey Dept. of Biostatistics & Medical Informatics.

Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 28 of 41 Friday, 22 October.

Readings: K&F: 11.3, 11.5 Yedidia et al. paper from the class website

Computing & Information Sciences Kansas State University Lecture 13 of 42 CIS 530 / 730 Artificial Intelligence Lecture 13 of 42 William H. Hsu Department.

Data and Applications Security Developments and Directions Dr. Bhavani Thuraisingham The University of Texas at Dallas Introduction to the Course January.

An Introduction to Variational Methods for Graphical Models

Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa.

1 Chapter 15 Probabilistic Reasoning over Time. 2 Outline Time and UncertaintyTime and Uncertainty Inference: Filtering, Prediction, SmoothingInference:

1 Use graphs and not pure logic Variables represented by nodes and dependencies by edges. Common in our language: “threads of thoughts”, “lines of reasoning”,

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Elements of a Discrete Model Evaluation.

CPSC 422, Lecture 17Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 17 Oct, 19, 2015 Slide Sources D. Koller, Stanford CS - Probabilistic.

1 CMSC 671 Fall 2001 Class #20 – Thursday, November 8.

Pattern Recognition and Machine Learning

1 Find as many examples as you can of w, x, y, z so that w is accepted by this DFA, w = x y z, y ≠ ε, | x y | ≤ 7, and x y n z is in L for all n ≥ 0.

Introduction on Graphic Models

Machine Learning – Lecture 18

Dynamic Programming & Hidden Markov Models. Alan Yuille Dept. Statistics UCLA.

Computing & Information Sciences Kansas State University Wednesday, 04 Oct 2006CIS 490 / 730: Artificial Intelligence Lecture 17 of 42 Wednesday, 04 October.

1 Variable Elimination Graphical Models – Carlos Guestrin Carnegie Mellon University October 15 th, 2008 Readings: K&F: 8.1, 8.2, 8.3,

Computing & Information Sciences Kansas State University Friday, 13 Oct 2006CIS 490 / 730: Artificial Intelligence Lecture 21 of 42 Friday, 13 October.

CS 188: Artificial Intelligence Spring 2007

CSE 515 Statistical Methods in Computer Science

Bayesian Networks Independencies Representation Probabilistic

Readings: K&F: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7 Markov networks, Factor graphs, and an unified view Start approximate inference If we are lucky… Graphical.

Readings: K&F: 11.3, 11.5 Yedidia et al. paper from the class website

Generalized Belief Propagation

Chapter 14 February 26, 2004.

Presentation transcript:

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring, 2006 Graphical Models Jeff A. Bilmes Lecture 8 Slides April 20 th, 2006

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 2 READING: –Chapter 11,12,15 in Jordan’s book Reminder: TA discussions and office hours: –Office hours: Thursdays 3:30-4:30, Sieg Ground Floor Tutorial Center –Discussion Sections: Fridays 9:30-10:30, Sieg Ground Floor Tutorial Center Lecture Room Reminder: take-home Midterm: May 5 th -8 th, you must work alone on this. Announcements

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 3 L1: Tues, 3/28: Overview, GMs, Intro BNs. L2: Thur, 3/30: semantics of BNs + UGMs L3: Tues, 4/4: elimination, probs, chordal I L4: Thur, 4/6: chrdal, sep, decomp, elim L5: Tue, 4/11: chdl/elim, mcs, triang, ci props. L6: Thur, 4/13: MST,CI axioms, Markov prps. L7: Tues, 4/18: Mobius, HC-thm, (F)=(G) L8: Thur, 4/20: phylogenetic trees, HMMs L9: Tue, 4/25 L10: Thur, 4/27 L11: Tues, 5/2 L12: Thur, 5/4 L13: Tues, 5/9 L14: Thur, 5/11 L15: Tue, 5/16 L16: Thur, 5/18 L17: Tues, 5/23 L18: Thur, 5/25 L19: Tue, 5/30 L20: Thur, 6/1: final presentations Class Road Map

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 4 L1: Tues, 3/28: L2: Thur, 3/30: L3: Tues, 4/4: L4: Thur, 4/6: L5: Tue, 4/11: L6: Thur, 4/13: L7: Tues, 4/18: L8: Thur, 4/20: Team Lists, short abstracts I L9: Tue, 4/25: L10: Thur, 4/27: short abstracts II L11: Tues, 5/2 L12: Thur, 5/4: abstract II + progress L13: Tues, 5/9 L14: Thur, 5/11: 1 page progress report L15: Tue, 5/16 L16: Thur, 5/18: 1 page progress report L17: Tues, 5/23 L18: Thur, 5/25: 1 page progress report L19: Tue, 5/30 L20: Thur, 6/1: final presentations L21: Tue, 6/6 4-page papers due (like a conference paper). Final Project Milestone Due Dates Team lists, abstracts, and progress reports must be turned in, in class and using paper (dead tree versions only). Final reports must be turned in electronically in PDF (no other formats accepted). Progress reports must report who did what so far!!

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 5 Factorization property on MRF, (F) When (F) = (G) = (L) = (P) inclusion-exclusion Möbius Inversion lemma Hammersley/Clifford theorem, when (G) => (F) Factorization and decomposability Factorization and junction tree Directed factorization (DF), and (G) Markov blanket Bayesian networks, moralization, and ancestral sets Summary of Last Time

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 6 d-Separation, (DL), (DO), and equivalence of all Markov properties on BNs. Phylogenetic Trees and Chordal Models Mixture Models Hidden Markov Models (HMMs) Forward (  ) recursion and elimination Backwards (  ) recursion and elimination Outline of Today’s Lecture

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 7 Books and Sources for Today M. Jordan: Chapters 11,12,15. Lauritzen, chapter 3. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, T. McKee “Topics in Intersection Graph Theory”

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 8 Preservation of (DF) in ancestral sets

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 9 Example (DF) – (G)

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 10 Example (DF) – (G)

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 11 d-Separation revisited

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 12 d-Separation revisited

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 13 All Together Now

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 14 What else can chordal graphs do?

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 15 Phylogenetic Tree: example species characters resulting phylogenetic tree

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 16 Perfect Phylogeny

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 17 Examples

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 18 Phylogenetic Trees

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 19 Examples: G T, G I 1,1 1,2 1,3 3,1 3,2 3,3 2,1 2,3 2,2

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 20 Phylogenetic Trees

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 21 Phylogenetic Trees

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 22 Intersection Graphs

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 23 Intersection Graphs, Chordality, Phylogeny

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 24 Summary But triangulated graphs (really ``trees'') have many other properties as well. We are interested in them since they are exactly the class of models on which we can perform exact inference, which is the topic we will next spend some time on. Next topic: Morphing from mixture models to HMMs

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 25 Mixture Models

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 26 Mixture Models

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 27 Inference on Hidden Markov Models Hidden Markov Models (HMMs) are a ubiquitously used model in speech recognition, natural language processing, bioinformatics, financial markets, and many time-series problems. HMMs are rich enough to be interesting, but simple enough so that they are a perfect example to start with when performing exact inference. HMMs can be described either with a BN or an MRF –so this means they must be decomposable Since HMMs are already triangulated (after moralization if necessary), there is no triangulation step. Moreover, since the clique sizes are small, HMMs are easy to deal with (compexity only O(TN 2 )

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 28 HMMs and Bayesian Networks

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 29 HMMs and Markov Random Fields

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 30 HMMs

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 31 HMMs

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 32 HMMs, elimination orders, and forward recursion

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 33 HMMs, elimination orders, and forward recursion

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 34 HMMs, elimination orders, and backward recursion

Lec 8: April 20th, 2006EE512 - Graphical Models - J. BilmesPage 35 HMMs, elimination orders, and backward recursion