Belief Propagation Kai Ju Liu March 9, 2006. Statistical Problems Medicine Finance Internet Computer vision.

Slides:

Advertisements

Similar presentations

Mean-Field Theory and Its Applications In Computer Vision1 1.

Advertisements

Bayesian Belief Propagation

Linear Time Methods for Propagating Beliefs Min Convolution, Distance Transforms and Box Sums Daniel Huttenlocher Computer Science Department December,

Markov Networks Alan Ritter.

Discrete Optimization Lecture 4 – Part 3 M. Pawan Kumar Slides available online

Exact Inference. Inference Basic task for inference: – Compute a posterior distribution for some query variables given some observed evidence – Sum out.

Introduction to Markov Random Fields and Graph Cuts Simon Prince

Exact Inference in Bayes Nets

Loopy Belief Propagation a summary. What is inference? Given: –Observabled variables Y –Hidden variables X –Some model of P(X,Y) We want to make some.

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.

EE462 MLCV Lecture Introduction of Graphical Models Markov Random Fields Segmentation Tae-Kyun Kim 1.

Probabilistic Inference Lecture 1

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Belief Propagation on Markov Random Fields Aggeliki Tsoli.

CS774. Markov Random Field : Theory and Application Lecture 04 Kyomin Jung KAIST Sep

1 Graphical Models in Data Assimilation Problems Alexander Ihler UC Irvine Collaborators: Sergey Kirshner Andrew Robertson Padhraic Smyth.

Global Approximate Inference Eran Segal Weizmann Institute.

Abstract We present a model of curvilinear grouping using piecewise linear representations of contours and a conditional random field to capture continuity.

1 Computer Vision Research  Huttenlocher, Zabih –Recognition, stereopsis, restoration, learning  Strong algorithmic focus –Combinatorial optimization.

Conditional Random Fields

Belief Propagation, Junction Trees, and Factor Graphs

Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship.

Understanding Belief Propagation and its Applications Dan Yuan June 2004.

© 2006 by Davi GeigerComputer Vision April 2006 L1.1 Binocular Stereo Left Image Right Image.

Measuring Uncertainty in Graph Cut Solutions Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University.

Computer vision: models, learning and inference

A Trainable Graph Combination Scheme for Belief Propagation Kai Ju Liu New York University.

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

Computer vision: models, learning and inference

CSC2535 Spring 2013 Lecture 2a: Inference in factor graphs Geoffrey Hinton.

Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Yong-Joong Kim Dept. of Computer Science Yonsei.

Some Surprises in the Theory of Generalized Belief Propagation Jonathan Yedidia Mitsubishi Electric Research Labs (MERL) Collaborators: Bill Freeman (MIT)

12/07/2008UAI 2008 Cumulative Distribution Networks and the Derivative-Sum-Product Algorithm Jim C. Huang and Brendan J. Frey Probabilistic and Statistical.

Undirected Models: Markov Networks David Page, Fall 2009 CS 731: Advanced Methods in Artificial Intelligence, with Biomedical Applications.

Markov Random Fields Probabilistic Models for Images

Inference Complexity As Learning Bias Daniel Lowd Dept. of Computer and Information Science University of Oregon Joint work with Pedro Domingos.

14 October, 2010LRI Seminar 2010 (Univ. Paris-Sud)1 Statistical performance analysis by loopy belief propagation in probabilistic image processing Kazuyuki.

1 Markov Random Fields with Efficient Approximations Yuri Boykov, Olga Veksler, Ramin Zabih Computer Science Department CORNELL UNIVERSITY.

CS774. Markov Random Field : Theory and Application Lecture 02

Approximate Inference: Decomposition Methods with Applications to Computer Vision Kyomin Jung ( KAIST ) Joint work with Pushmeet Kohli (Microsoft Research)

Learning low-level vision Computer Examples by Michael Ross.

1 Mean Field and Variational Methods finishing off Graphical Models – Carlos Guestrin Carnegie Mellon University November 5 th, 2008 Readings: K&F:

Probabilistic Graphical Models seminar 15/16 ( ) Haim Kaplan Tel Aviv University.

Belief Propagation and its Generalizations Shane Oldenburger.

Exact Inference in Bayes Nets. Notation U: set of nodes in a graph X i : random variable associated with node i π i : parents of node i Joint probability:

CPSC 422, Lecture 17Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 17 Oct, 19, 2015 Slide Sources D. Koller, Stanford CS - Probabilistic.

Markov Random Fields & Conditional Random Fields

Christopher M. Bishop, Pattern Recognition and Machine Learning 1.

Efficient Belief Propagation for Image Restoration Qi Zhao Mar.22,2006.

Markov Random Fields in Vision

Perfect recall: Every decision node observes all earlier decision nodes and their parents (along a “temporal” order) Sum-max-sum rule (dynamical programming):

10 October, 2007 University of Glasgow 1 EM Algorithm with Markov Chain Monte Carlo Method for Bayesian Image Analysis Kazuyuki Tanaka Graduate School.

Extending Expectation Propagation for Graphical Models

Sublinear Computational Time Modeling in Statistical Machine Learning Theory for Markov Random Fields Kazuyuki Tanaka GSIS, Tohoku University, Sendai,

Markov Random Fields with Efficient Approximations

Exact Inference Continued

Prof. Adriana Kovashka University of Pittsburgh April 4, 2017

Graduate School of Information Sciences, Tohoku University

Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 18

Uncertainty in AI.

CSCI 5822 Probabilistic Models of Human and Machine Learning

Generalized Belief Propagation

Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 17

Graduate School of Information Sciences, Tohoku University

Markov Random Fields Presented by: Vladan Radosavljevic.

Graduate School of Information Sciences, Tohoku University

Expectation-Maximization & Belief Propagation

Readings: K&F: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7 Markov networks, Factor graphs, and an unified view Start approximate inference If we are lucky… Graphical.

Lecture 3: Exact Inference in GMs

Presentation transcript:

Belief Propagation Kai Ju Liu March 9, 2006

Statistical Problems Medicine Finance Internet Computer vision

Inference Problems Given data B, infer A: p(A|B) Computer vision –Given image, find objects –Given two images, resolve 3D object –Given multiple images, track object

Conditional Probability Given event B, what is probability of A? Independence: p(A|B)=p(A) AB

Bayes’ Rule

e.g. ColdWeekday Party Hangover Marginal Probability: 8-sum Joint Probability:

e.g. (cont.) ColdWeekday Party Hangover Marginal Probability: 8-sum Localize probabilities:

Approach Define variables and connections Calculate marginal probabilities efficiently Find most likely configuration

Pairwise Markov Random Field Basic structure: vertices, edges

Pairwise Markov Random Field Basic structure: vertices, edges Vertex i has set of possible states X i and observed value y i Compatibility between states and observed values, Compatibility between neighboring vertices i and j,

Pairwise MRF: Probabilities Joint probability: Marginal probability: –Advantage: allows average over ambiguous states –Disadvantage: complexity exponential in number of vertices

Belief Propagation

Beliefs replace probabilities: Messages propagate information:

Belief Propagation Example

BP: Questions When can we calculate beliefs exactly? When do beliefs equal probabilities? When is belief propagation efficient? Answer: Singly-Connected Graphs (SCG’s) Graphs without loops Messages terminate at leaf vertices Beliefs equal probabilities Complexity in previous example reduced from 13S 5 to 24S 2

BP on Loopy Graphs Messages do not terminate Possible approximate solutions –Standard belief propagation –Generalized belief propagation BP-TwoGraphs: Goals Utilize advantages of SCG’s Be accurate and efficient on loopy graphs

BP-TwoGraphs: SCG’s Calculate beliefs on each set of SCG’s: – Select maximum beliefs from both sets – Consider loopy graph with n vertices Select two sets of SCG’s that approximate the graph –

BP-TwoGraphs: Vision SCG’s Rectangular grid of pixel vertices H i : horizontal graphs G i : vertical graphs

Image Segmentation add noise segment

Image Segmentation: Results

Real Image Segmentation

Real Image Segmentation: Training

Real Image Segmentation: Results

Stereo Vision