1. Belief Propagation in Large, Highly Connected Graphs for 3D Part-Based Object Recognition Frank DiMaio and Jude Shavlik Computer Sciences Department University of Wisconsin – Madison USA

2. Part-Based Object Recognition A part-based model describes an object using a pairwise Markov Field (Felzenszwalb et al 2000, Sudderth et al 2004, Isard 2003) Object described using Undirected part graph G=(V,E) Vertex potential functions Edge potential functions

3. Part-Based Object Recognition Probability of a configuration U={u i } – given an image I – is the product of potential functions For part-based object recognition Skeletal graph for tightly coupled parts Occupancy graph ensures no other parts collide in 3D space

4. Inferring Part Locations with Belief Propagation (BP) Want to find part configuration maximizing product of potential functions Use belief propagation (BP) to approximate marginal distributions Iterative, message-passing method (Pearl 1988) A message, m i→j, from part i to part j indicates where i currently expects to find j

5. Belief Propagation Example b( torso | image) b( head | image) b( left arm | image) b( left leg | image) b( right arm | image) b( right leg | image)

6. Belief Propagation Example m head→torso (torso) b( torso | image) b( head | image) m R.leg → torso m L.leg → torso m R.arm → torso m L.arm → torso

7. Belief Propagation Example b( torso | image) b( head | image) b( left arm | image) b( left leg | image) b( right arm | image) b( right leg | image)

8. What if the Graph has Thousands of Parts? In a graph with N parts and E edges BP running time and memory requirements O(E) Skeleton graph typically sparse – O(N) edges Occupancy graph fully connected – O(N 2 ) edges In very large graphs, O(N 2 ) runtime intractable AggBP (our system) approximates O(N 2 ) occupancy messages using O(N) messages

9. Message Approximation Illustrated 2 3 571 6 8 3 6 5 4

10. 2 3 571 6 8 4 Accumulator

11. Experiment I: Density Map Interpretation GLU TYR PHE THR LEU GLN ILE ARG GLY ARG GLU ARG PHE … GLY 31 ALA 30 GLN 29 ALA 28 LYS 26 LEU 25 GLU 24 LEU 23 GLU 21 ASN 20 LEU 19 GLU 18 ARG 17 PHE 16 MET 15 GLU 14 PHE 13 ARG 12 GLU 11 ARG 10 GLY 9 ARG 8 ILE 7 GLN 6 LEU 5 THR 4 PHE 3 TYR 2 GLU 1 ALA 22 ASP 27

12. LoopyBP vs. AggBP: Runtime Number of Parts Normalized Runtime 0 5 10 15 20 25 30 152535455565758595 AggBP LoopyBP

13. LoopyBP vs. AggBP: Accuracy BP iteration 0 2 4 6 8 10 05 1520 AggBP LoopyBP RMS Error

14. Experiment II: Synthetic Graph Generator increase branching factor allow spatial overlap vary radii [skeleton graph]

15. LoopyBP vs. AggBP: Accuracy 0125 stdev(part size) RMS Error 0 2 4 6 024 LoopyBP AggBP

16. Conclusions AggBP makes belief propagation tractable in large, highly connected graphs For part-based modeling, runtime and storage is reduced from O(N 2 ) to O(N) AggBP’s solutions on two datasets are as good as standard BP’s in less time

17. Acknowledgements Dr. George Phillips NLM Grant 1R01 LM008796 NLM Grant 1T15 LM007359