ED-BP: Belief Propagation via Edge Deletion UCLA Automated Reasoning Group Arthur Choi, Adnan Darwiche, Glen Lenker, Knot Pipatsrisawat Last updated 07/15/2010:

Slides:

Advertisements

Similar presentations

Scaling Up Graphical Model Inference

Advertisements

Exact Inference. Inference Basic task for inference: – Compute a posterior distribution for some query variables given some observed evidence – Sum out.

Join-graph based cost-shifting Alexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten University of California Irvine Introduction Mini-Bucket Elimination.

Exact Inference in Bayes Nets

MPE, MAP AND APPROXIMATIONS Lecture 10: Statistical Methods in AI/ML Vibhav Gogate The University of Texas at Dallas Readings: AD Chapter 10.

Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.

Markov Networks.

Belief Propagation on Markov Random Fields Aggeliki Tsoli.

CS774. Markov Random Field : Theory and Application Lecture 04 Kyomin Jung KAIST Sep

Bayesian network inference

Carnegie Mellon Focused Belief Propagation for Query-Specific Inference Anton Chechetka Carlos Guestrin 14 May 2010.

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graph.

Global Approximate Inference Eran Segal Weizmann Institute.

Message Passing Algorithms for Optimization

Belief Propagation, Junction Trees, and Factor Graphs

Understanding Belief Propagation and its Applications Dan Yuan June 2004.

Efficient Quantum State Tomography using the MERA in 1D critical system Presenter : Jong Yeon Lee (Undergraduate, Caltech)

Learning Programs Danielle and Joseph Bennett (and Lorelei) 4 December 2007.

A Differential Approach to Inference in Bayesian Networks - Adnan Darwiche Jiangbo Dang and Yimin Huang CSCE582 Bayesian Networks and Decision Graphs.

On the Power of Belief Propagation: A Constraint Propagation Perspective Rina Dechter Bozhena Bidyuk Robert Mateescu Emma Rollon.

Genome Evolution. Amos Tanay 2009 Genome evolution: Lecture 8: Belief propagation.

Lec 18: May 31st, 2006EE512 - Graphical Models - J. BilmesPage 1 Jeff A. Bilmes University of Washington Department of Electrical Engineering EE512 Spring,

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

Ahsanul Haque *, Swarup Chandra *, Latifur Khan * and Charu Aggarwal + * Department of Computer Science, University of Texas at Dallas + IBM T. J. Watson.

CS774. Markov Random Field : Theory and Application Lecture 13 Kyomin Jung KAIST Oct

Ahsanul Haque *, Swarup Chandra *, Latifur Khan * and Michael Baron + * Department of Computer Science, University of Texas at Dallas + Department of Mathematical.

Undirected Models: Markov Networks David Page, Fall 2009 CS 731: Advanced Methods in Artificial Intelligence, with Biomedical Applications.

GraphLab: how I understood it with sample code Aapo Kyrola, Carnegie Mellon Univ. Oct 1, 2009.

Belief Propagation. What is Belief Propagation (BP)? BP is a specific instance of a general class of methods that exist for approximate inference in Bayes.

Belief Propagation Revisited Adnan Darwiche. Graphical Models Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.

Readings: K&F: 11.3, 11.5 Yedidia et al. paper from the class website

Two Approximate Algorithms for Belief Updating Mini-Clustering - MC Robert Mateescu, Rina Dechter, Kalev Kask. "Tree Approximation for Belief Updating",

Experimental Study of Directed Feedback Vertex Set Problem With Rudolf Fleischer and Liwei Yuan Fudan University, Shanghai Xi Wu.

Daphne Koller Message Passing Belief Propagation Algorithm Probabilistic Graphical Models Inference.

Variable and Value Ordering for MPE Search Sajjad Siddiqi and Jinbo Huang.

1 Mean Field and Variational Methods finishing off Graphical Models – Carlos Guestrin Carnegie Mellon University November 5 th, 2008 Readings: K&F:

Stochastic greedy local search Chapter 7 ICS-275 Spring 2009.

Belief Propagation and its Generalizations Shane Oldenburger.

Using Combinatorial Optimization within Max-Product Belief Propagation

Wei Sun and KC Chang George Mason University March 2008 Convergence Study of Message Passing In Arbitrary Continuous Bayesian.

COSC 5341 High-Performance Computer Networks Presentation for By Linghai Zhang ID:

Exact Inference in Bayes Nets. Notation U: set of nodes in a graph X i : random variable associated with node i π i : parents of node i Joint probability:

Single-Pass Belief Propagation

Join-graph based cost-shifting Alexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten University of California Irvine Introduction Mini-Bucket Elimination.

Tightening LP Relaxations for MAP using Message-Passing David Sontag Joint work with Talya Meltzer, Amir Globerson, Tommi Jaakkola, and Yair Weiss.

04/21/2005 CS673 1 Being Bayesian About Network Structure A Bayesian Approach to Structure Discovery in Bayesian Networks Nir Friedman and Daphne Koller.

Markov Networks: Theory and Applications Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208

30 November, 2005 CIMCA2005, Vienna 1 Statistical Learning Procedure in Loopy Belief Propagation for Probabilistic Image Processing Kazuyuki Tanaka Graduate.

Many-Pairs Mutual Information for Adding Structure to Belief Propagation Approximations Arthur Choi and Adnan Darwiche University of California, Los Angeles.

Compiling Bayesian Networks Using Variable Elimination

Belief Propagation and Approximate Inference: Compensating for Relaxations Arthur Choi.

StatSense In-Network Probabilistic Inference over Sensor Networks

Bucket Renormalization for Approximate Inference

Encoding CNFs to Enhance Component Analysis

Arthur Choi and Adnan Darwiche UCLA

Generalized Belief Propagation

Bucket Renormalization for Approximate Inference

Graduate School of Information Sciences, Tohoku University

Arthur Choi and Adnan Darwiche UCLA

Physical Fluctuomatics 7th~10th Belief propagation

Expectation-Maximization & Belief Propagation

Probabilistic image processing and Bayesian network

Approximate Inference by Sampling

Approximating the Partition Function by Deleting and then Correcting for Model Edges Arthur Choi and Adnan Darwiche University of California, Los Angeles.

Readings: K&F: 11.3, 11.5 Yedidia et al. paper from the class website

Markov Networks.

Mean Field and Variational Methods Loopy Belief Propagation

Iterative Join Graph Propagation

Presentation transcript:

ED-BP: Belief Propagation via Edge Deletion UCLA Automated Reasoning Group Arthur Choi, Adnan Darwiche, Glen Lenker, Knot Pipatsrisawat Last updated 07/15/2010: See the speaker notes on each slide for commentary (PowerPoint only).

Adnan Knot Glen Arthur

ED-BP: Idea Delete Edges Compensate loopy BP

Characterizing Belief Propagation ED-BP characterization: in a tree: – MAR: BP marginals – PR: Bethe – MPE: BP max-marginals XiXi XiXi XjXj XjXj (Xi)(Xi) (Xj)(Xj) Pr(X i = x) = Pr(X j = x) =  (X i = x)  (X j = x)/z ij

Edge Deletion exact delete edges ?

Edge Recovery loopy BP exact recover edges … …

Edge Recovery: Old Idea loopy BP recover edges … … [CD06]: target quality: use mutual information

Edge Recovery: New Idea loopy BP recover edges … … Challenge UAI-10: encourage convergence, residual recovery

ED-BP: Residual Recovery Recover edges based on how close they are to convergence ED-BP characterization: Ongoing: try residuals as in residual BP XiXi XiXi XjXj XjXj (Xi)(Xi) (Xj)(Xj) Pr(X i = x) = Pr(X j = x) =  (X i )  (X j )/z ij

Exact Solvers Exact inference in a simplified network: ED-BP can use any black box inference engine – currently using vanilla Hugin and Shenoy-Shafer jointree algorithms – not currently using Ace, or other advanced inference engines …

Overall Results SolverScore edbr vgogate libDai PR Task: 20 Seconds SolverScore edbq libDai vgogate MAR Task: 20 Seconds

Overall Results SolverScore vgogate edbp libDai PR Task: 20 Minutes SolverScore ijgp edbq libDai MAR Task: 20 Minutes

Overall Results SolverScore vgogate edbr libDai PR Task: 1 Hour SolverScore ijgp edbr libDai MAR Task: 1 Hour

Overall Results SolverScore vgogate edbr libDai PR Task: 1 Hour SolverScore ijgp edbr libDai MAR Task: 1 Hour Congratulations Vibhav

More Slides on the ED-BP Solver

ED-BP: The Solver Based on UAI'08 solver, new MPE version Numerous improvements – pre-processing – initial spanning tree – internal inference engine for exact reasoning – edge recovery led to biggest impact in performance

ED-BP: The Solver Pre-processing – lightweight – RSat: infer fixed values from network zero’s Initial spanning tree – random spanning tree – max spanning tree (mutual information) Black box engine for exact inference – jointree algorithms: shenoy-shafer versus hugin – in the future: compilation to ACs (Ace)

Arthur Choi, Hei Chan, and Adnan Darwiche. On Bayesian Network Approximation by Edge Deletion. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI), Arthur Choi and Adnan Darwiche. An Edge Deletion Semantics for Belief Propagation and its Practical Impact on Approximation Quality. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI), Arthur Choi and Adnan Darwiche. A Variational Approach for Approximating Bayesian Networks by Edge Deletion. In Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence (UAI), Arthur Choi, Mark Chavira, and Adnan Darwiche. Node Splitting: A Scheme for Generating Upper Bounds in Bayesian Networks. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI), Arthur Choi and Adnan Darwiche. Approximating the Partition Function by Deleting and then Correcting for Model Edges. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI), Arthur Choi and Adnan Darwiche. Focusing Generalizations of Belief Propagation on Targeted Queries. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI), Arthur Choi and Adnan Darwiche. Many-Pairs Mutual Information for Adding Structure to Belief Propagation Approximations. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI), Arthur Choi and Adnan Darwiche. Approximating MAP by Compensating for Structural Relaxations. In Proceedings of the Twenty-Third Annual Conference on Neural Information Processing Systems (NIPS), Arthur Choi, Trevor Standley, and Adnan Darwiche. Approximating Weighted Max-SAT Problems by Compensating for Relaxations. In Proceedings of the 15th International Conference on Principles and Practice of Constraint Programming (CP), References