Download presentation
Presentation is loading. Please wait.
1
Arthur Choi and Adnan Darwiche UCLA {aychoi,darwiche}@cs.ucla.edu
An Edge Deletion Semantics for Belief Propagation and its Practical Impact on Approximation Quality Slides used for oral presentation at AAAI-06. Updated 09/21/2006. Arthur Choi and Adnan Darwiche UCLA
2
The Idea A C B D A B C D Approximate inference: Exact inference in an approximate model Approximate model: by deleting edges
3
The Idea A C B D A B Y X C D Approximate inference: Exact inference in an approximate model Approximate model: by deleting edges BP as approximate models Semantics lead to improved approximations.
4
The Idea Original Network Approximate Network
5
Deleting an Edge U X
6
Deleting an Edge: The Clone
U U' X
7
Deleting an Edge: The Soft Evidence
U New edge parameters for each new query. s' U' X
8
Deleting Edges: Specifying the Approximation
How do we parametrize edges? Compensate for the missing edge Quality of approximation Which edges do we delete? Computational complexity
9
Belief Propagation in Polytrees
C D E BP is exact in polytrees.
10
Belief Propagation in Polytrees
U C D X E BP is exact in polytrees.
11
Belief Propagation in Polytrees
U C D X E BP is exact in polytrees.
12
Belief Propagation in Polytrees
U C D X E BP is exact in polytrees.
13
Belief Propagation in Loopy Networks
U C D X E IBP is approximate in loopy networks.
14
Iterative Belief Propagation in Loopy Networks
U C D X E Iteration t = 0, Initialization
15
Iterative Belief Propagation in Loopy Networks
U C D X E Iteration t = 1
16
Iterative Belief Propagation in Loopy Networks
U C D X E Iteration t = 2
17
Iterative Belief Propagation in Loopy Networks
U C D X E Iteration tc Convergence
18
Iterative Belief Propagation
Successfully applied in varied applications: Error-correcting codes Computer vision Satisfiability Etc. U X
19
Iterative Belief Propagation
Successfully applied in varied applications: Error-correcting codes Computer vision Satisfiability Etc. U What does IBP do? Perspective based on statistical physics … Perspective based on deleting edges … X
20
Deleting Edges: Specifying the Approximation
How do we parametrize edges? Compensate for the missing edge Quality of approximation Which edges do we delete? Computational complexity
21
Parametrizing Edges: ED-BP (Edge Deletion-Belief Propagation)
ED-BP: Choose parameters that satisfy U s' U' Used as update equations: Initialize parameters Iterate until we reach a fixed point X
22
Parametrizing Edges Iteratively: ED-BP
Iteration t = 0 Initialization
23
Parametrizing Edges Iteratively: ED-BP
Iteration t = 1
24
Parametrizing Edges Iteratively: ED-BP
Iteration t = 2
25
Parametrizing Edges Iteratively: ED-BP
Iteration tc Convergence
26
Iterative Belief Propagation as Edge Deletion
Theorem: IBP corresponds to ED-BP ED-BP: Iteration t Iteration t
27
Iterative Belief Propagation as Edge Deletion
IBP is a disconnected approximation. IBP in the original network IBP is any polytree approximation.
28
Splitting the Network into Two
Theorem: If deleting an edge splits a network into two independent subnetworks, marginals are exact in each subnetwork. U U' s' X What if deleting an edge doesn’t split a network into two?
29
Which Edges Do We Delete?
30
Summary: ED-BP (Edge Deletion-Belief Propagation)
How do we parametrize edges? Local conditions: Global conditions: Agreement on parent marginals. Agreement on strengths of evidence. Which edges do we delete? Recover edges using mutual information.
31
ED-BP: Improving on the Quality of IBP
Belief Propagation Exact Inference
32
ED-BP: Improving on the Quality of IBP
Belief Propagation Exact Inference
33
ED-BP: Potentially Bad Approximations
Belief Propagation Unimproved, but costly, approximation, Exact Inference
34
ED-BP: Improving on the Convergence Rate
35
ED-BP: Improving on Running Time
36
ED-BP: Global KL-Divergence
Belief Propagation Exact Inference Choi & Darwiche, presented at UAI-06.
37
ED-BP: Global KL-Divergence
Belief Propagation See UAI-06 Exact Inference Choi & Darwiche, presented at UAI-06.
38
ED-BP: Global KL-Divergence
Belief Propagation See UAI-06 Exact Inference Choi & Darwiche, presented at UAI-06.
39
Summary: ED-BP (Edge Deletion-Belief Propagation)
How do we parametrize edges? Characterizes BP: As a fully disconnected approximation, As a class of polytree approximations. Which edges do we delete? Recover edges using mutual information. Are there other ways to delete edges? Yes! Based on the KL-Divergence (see UAI-06).
Similar presentations
