1. Discrete Optimization Lecture 4 – Part 2 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvn.ecp.fr/personnel/pawan/

2. MRF V1V1 d1d1 V2V2 d2d2 V3V3 d3d3 V4V4 d4d4 V5V5 d5d5 V6V6 d6d6 V7V7 d7d7 V8V8 d8d8 V9V9 d9d9 A is conditionally independent of B given C if there is no path from A to B when C is removed

3. MRF V1V1 d1d1 V2V2 d2d2 V3V3 d3d3 V4V4 d4d4 V5V5 d5d5 V6V6 d6d6 V7V7 d7d7 V8V8 d8d8 V9V9 d9d9 V a is conditionally independent of V b given V a ’s neighbors

4. Pairwise MRF V1V1 d1d1 V2V2 d2d2 V3V3 d3d3 V4V4 d4d4 V5V5 d5d5 V6V6 d6d6 V7V7 d7d7 V8V8 d8d8 V9V9 d9d9 Z is known as the partition function Unary Potential ψ 1 (v 1,d 1 ) Pairwise Potential ψ 56 (v 5,v 6 ) Probability P(v,d) = Π a ψ a (v a,d a ) Π (a,b) ψ ab (v a,v b ) Z

5. Inference max v P(v) Maximum a Posteriori (MAP) Estimation min v Q(v)Energy Minimization P(v a = l i ) = Σ v P(v)δ(v a = l i ) Computing Marginals P(v a = l i, v b = l k ) = Σ v P(v)δ(v a = l i )δ(v b = l k ) P(v) = exp(-Q(v))/Z

6. Outline Belief Propagation on Chains Belief Propagation on Trees Loopy Belief Propagation

7. Overview VaVa VbVb VcVc VdVd Compute the marginal probability for V d P(v) = P(v a |v b )P(v b |v c )P(v c |v d )P(v d ) Compute (unnormalized) distribution Ψ a (v a )Ψ ab (v a,v b )ΣvaΣva Function m(v b )

8. Overview VaVa VbVb VcVc VdVd Compute the marginal probability for V d P(v) = P(v a |v b )P(v b |v c )P(v c |v d )P(v d ) Compute (unnormalized) distribution Ψ b (v b )Ψ bc (v b,v c )m(v b )ΣvbΣvb Function m(v c )

9. Overview VaVa VbVb VcVc VdVd Compute the marginal probability for V d P(v) = P(v a |v b )P(v b |v c )P(v c |v d )P(v d ) Compute (unnormalized) distribution Ψ c (v c )Ψ cd (v c,v d )m(v c )ΣvcΣvc (Unnormalized) Marginals !!

10. Overview VaVa VbVb VcVc VdVd Compute the marginal probability for V c P(v) = P(v a |v b )P(v b |v c )P(v c |v d )P(v d ) P(v) = P(v a |v b )P(v b |v c )P(v d |v c )P(v c ) Several common terms !!

11. Overview VaVa VbVb VcVc VdVd Compute the marginal probability for V b P(v) = P(v a |v b )P(v b |v c )P(v c |v d )P(v d ) P(v) = P(v a |v b )P(v b |v c )P(v d |v c )P(v c ) P(v) = P(v a |v b )P(v c |v b )P(v d |v c )P(v b )

12. Overview VaVa VbVb VcVc VdVd Compute the marginal probability for V a P(v) = P(v a |v b )P(v b |v c )P(v c |v d )P(v d ) P(v) = P(v a |v b )P(v b |v c )P(v d |v c )P(v c ) P(v) = P(v a |v b )P(v c |v b )P(v d |v c )P(v b ) P(v) = P(v b |v a )P(v c |v b )P(v d |v c )P(v a )

13. Belief Propagation on Chains Compute exact marginals Avoids re-computing common terms

14. Two Variables VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 Unary Potentials ψ a (l i ) Pairwise Potentials ψ ab (l i,l k )

15. Two Variables VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 Marginal Probability P(v b = l j ) = Σ i ψ a (l i )ψ b (l j )ψ ab (l i,l j )/Z

16. Two Variables VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 Un-normalized Marginal Probability P’(v b = l j ) = Σ i ψ a (l i )ψ b (l j )ψ ab (l i,l j )/Z

17. Two Variables VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 Un-normalized Marginal Probability P’(v b = l j ) = Σ i ψ a (l i )ψ b (l j )ψ ab (l i,l j )

18. Two Variables VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 Un-normalized Marginal Probability P’(v b = l j ) = ψ b (l j )Σ i ψ a (l i )ψ ab (l i,l j )

19. Two Variables VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 2 x 3

20. Two Variables VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 2 x 3+ 5 x 1 M ab;0 11

21. Two Variables VaVa VbVb 2 5 41 3 2 x 1 VaVa VbVb 2 5 2 3 1 11

22. Two Variables 2 x 1 11 VaVa VbVb 2 5 2 3 1 VaVa VbVb 2 5 41 3 + 5 x 3 M ab;1 17

23. Two Variables 11 VaVa VbVb 2 5 2 3 1 17 Marginal Probability P’(v b = l j ) = ψ b (l j )Σ i ψ a (l i )ψ ab (l i,l j ) VaVa VbVb 2 5 41 3

24. Two Variables 11 VaVa VbVb 2 5 2 3 1 17 Marginal Probability P’(v b = l j ) = ψ b (l j )M ab;j VaVa VbVb 2 5 41 3 P’(v b = l 0 ) = 22P’(v b = l 1 ) = 68

25. Two Variables 11 VaVa VbVb 2 5 2 3 1 17 Marginal Probability P(v b = l j ) = ψ b (l j )M ab;j /Z VaVa VbVb 2 5 41 3 P’(v b = l 0 ) = 22P’(v b = l 1 ) = 68 Z = Σ j P’(v b = l j ) = 90

26. Two Variables 11 VaVa VbVb 2 5 2 3 1 17 VaVa VbVb 2 5 41 3 P(v b = l 0 ) = 0.244…P(v b = l 1 ) = 0.755… = 90 O(h 2 )!! Marginal Probability P(v b = l j ) = ψ b (l j )M ab;j /Z Z = Σ j P’(v b = l j )

27. Two Variables 11 VaVa VbVb 2 5 2 3 1 17 VaVa VbVb 2 5 41 3 P(v b = l 0 ) = 0.244…P(v b = l 1 ) = 0.755… O(h 2 )!! Same as brute-force

28. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 P’(v c = l k ) Σ j Σ i ψ a (l i )ψ b (l j )ψ c (l k )ψ ab (l i,l j )ψ bc (l j,l k )

29. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 P’(v c = l k ) ψ c (l k )Σ j Σ i ψ a (l i )ψ b (l j )ψ ab (l i,l j )ψ bc (l j,l k )

30. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )Σ i ψ a (l i )ψ ab (l i,l j )ψ bc (l j,l k )

31. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )ψ bc (l j,l k )Σ i ψ a (l i )ψ ab (l i,l j ) M ab;j 11 17

32. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )ψ bc (l j,l k )M ab;j 11 17 M bc;k

33. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 46 2 1 3 3 2 P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )ψ bc (l j,l k )M ab;j 11 17

34. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 46 2 1 3 3 2 P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )ψ bc (l j,l k )M ab;j 11 17 4 x 2 x 11

35. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 46 2 1 3 3 2 P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )ψ bc (l j,l k )M ab;j 11 17 4 x 2 x 11+ 2 x 2 x 17

36. Three Variables VaVa VbVb 2 5 2 3 1 VcVc 46 2 1 3 3 2 P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )ψ bc (l j,l k )M ab;j 11 17 4 x 2 x 11+ 2 x 2 x 17 156

37. Three Variables P’(v c = l k ) ψ c (l k )Σ j ψ b (l j )ψ bc (l j,l k )M ab;j 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11146

38. Three Variables P’(v c = l k ) ψ c (l k )M bc;k 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 NOTE: M bc;k “includes” M ab;j 146

39. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 P(v c = 0) = 0.35 P(v c = 1) = 0.65 Z = 156 x 3 + 146 x 6 = 1344 146

40. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 O(nh 2 )Better than brute-force 146

41. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 What about P(v b = l j )? 146

42. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 P’(v b = l j ) Σ k Σ i ψ a (l i )ψ b (l j )ψ c (l k )ψ ab (l i,l j )ψ bc (l j,l k ) 146

43. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 P’(v b = l j ) ψ b (l j )Σ k Σ i ψ a (l i )ψ c (l k )ψ ab (l i,l j )ψ bc (l j,l k ) 146

44. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 P’(v b = l j ) ψ b (l j )Σ k ψ c (l k )Σ i ψ a (l i )ψ ab (l i,l j )ψ bc (l j,l k ) 146

45. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 P’(v b = l j ) ψ b (l j )Σ k ψ c (l k )ψ bc (l j,l k )Σ i ψ a (l i )ψ ab (l i,l j ) M ab;j 146

46. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 P’(v b = l j ) ψ b (l j )M ab;j Σ k ψ c (l k )ψ bc (l j,l k ) M cb;j NOTE: M cb;j does not “include” M bc;k 146

47. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 P’(v b = l j ) ψ b (l j )M ab;j M cb;j 24 12146

48. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 P(v b = 0) = 0.39 P(v b = 1) = 0.61 Z = 11 x 12 x 4 + 17 x 24 x 2 = 1344

49. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 O(nh 2 )Better than brute-force

50. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 What about P(v a = l i )?

51. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 P’(v a = l i ) Σ j Σ k ψ a (l i )ψ b (l j )ψ c (l k )ψ ab (l i,l j )ψ bc (l j,l k )

52. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 P’(v a = l i ) ψ a (l i )Σ j Σ k ψ b (l j )ψ c (l k )ψ ab (l i,l j )ψ bc (l j,l k )

53. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 P’(v a = l i ) ψ a (l i )Σ j ψ b (l j )Σ k ψ c (l k )ψ ab (l i,l j )ψ bc (l j,l k )

54. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 P’(v a = l i ) ψ a (l i )Σ j ψ b (l j )ψ ab (l i,l j )Σ k ψ c (l k )ψ bc (l j,l k ) M cb;j

55. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 P’(v a = l i ) ψ a (l i )Σ j ψ b (l j )ψ ab (l i,l j )M cb;j M ba;i NOTE: M ba;i “includes” M cb;j

56. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 P’(v a = l i ) ψ a (l i )M ba;i 192

57. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 192 P(v a = 0) = 0.71 P(v b = 1) = 0.29 Z = 192 x 2 + 192 x 5 = 1344

58. Three Variables 17156 VaVa VbVb 2 5 2 3 1 VcVc 461 2 1 3 3 2 3 11 24 12146 192 O(nh 2 )Better than brute-force

59. Belief Propagation on Chains Start from left, go to right For current edge (a,b), compute M ab;k = Σ i ψ a (l i )ψ ab (l i,l k )Π n≠b M na;i Repeat till the end of the chain Start from right, go to left M ab;k = Σ i ψ a (l i )ψ ab (l i,l k )Π n≠b M na;i Repeat till the end of the chain

60. Belief Propagation on Chains P’(v a = l i,v b = l j ) = ? Normalize to compute true marginals P’(v a = l i ) = ? ψ a (l i )ψ b (l j )ψ ab (l i,l j )Π n≠b M na;i Π n≠a M nb;j ψ a (l i )Π n M na;i

61. Outline Belief Propagation on Chains Belief Propagation on Trees Loopy Belief Propagation Pearl, 1988

62. Belief Propagation on Trees VcVc VdVd VaVa VbVb Σ k Σ j Σ i ψ a (l i )ψ b (l j )ψ c (l k )ψ d (l o )ψ ac (l i,l k )ψ bc (l j,l k )ψ cd (l k,l o ) P’(v d = l o )

63. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )Σ k Σ j Σ i ψ a (l i )ψ b (l j )ψ c (l k )ψ ac (l i,l k )ψ bc (l j,l k )ψ cd (l k,l o ) P’(v d = l o )

64. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )Σ k ψ c (l k )Σ j Σ i ψ a (l i )ψ b (l j )ψ ac (l i,l k )ψ bc (l j,l k )ψ cd (l k,l o ) P’(v d = l o )

65. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )Σ k ψ c (l k )ψ cd (l k,l o )Σ j Σ i ψ a (l i )ψ b (l j )ψ ac (l i,l k )ψ bc (l j,l k ) P’(v d = l o )

66. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )Σ k ψ c (l k )ψ cd (l k,l o )Σ j ψ b (l j )Σ i ψ a (l i )ψ ac (l i,l k )ψ bc (l j,l k ) P’(v d = l o )

67. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )Σ k ψ c (l k )ψ cd (l k,l o )Σ j ψ b (l j )ψ bc (l j,l k )Σ i ψ a (l i )ψ ac (l i,l k ) P’(v d = l o ) M ac;k

68. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )Σ k ψ c (l k )ψ cd (l k,l o )Σ j ψ b (l j )ψ bc (l j,l k )M ac;k P’(v d = l o ) M bc;k M ac;k M bc;k

69. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )Σ k ψ c (l k )ψ cd (l k,l o )M bc;k M ac;k P’(v d = l o ) M ac;k M bc;k M cd;o

70. Belief Propagation on Trees VcVc VdVd VaVa VbVb ψ d (l o )M cd;o P’(v d = l o ) M ac;k M bc;k M cd;o

71. Belief Propagation on Trees VcVc VdVd VaVa VbVb P’(v c = l k ) M ac;k M bc;k M cd;o M dc;k ψ c (l k )M ac;k M bc;k M dc;k

72. Belief Propagation on Trees VcVc VdVd VaVa VbVb P’(v b = l j ) M ac;k M bc;k M cd;o M dc;k M cb;j ψ b (l j )M cb;j

73. Belief Propagation on Trees VcVc VdVd VaVa VbVb P’(v a = l i ) M ac;k M bc;k M cd;o M dc;k M cb;j M ca;i ψ a (l i )M ca;i

74. Belief Propagation on Trees Start from leaf, go towards root For current edge (a,b), compute M ab;k = Σ i ψ a (l i )ψ ab (l i,l k )Π n≠b M na;i Repeat till the root is reached Start from root, go towards leaves M ab;k = Σ i ψ a (l i )ψ ab (l i,l k )Π n≠b M na;i Repeat till the leafs are reached

75. Belief Propagation on Trees P’(v a = l i,v b = l j ) = ? Normalize to compute true marginals P’(v a = l i ) = ? ψ a (l i )ψ b (l j )ψ ab (l i,l j )Π n≠b M na;i Π n≠a M nb;j ψ a (l i )Π n M na;i

76. Outline Belief Propagation on Chains Belief Propagation on Trees Loopy Belief Propagation Pearl, 1988; Murphy et al., 1999

77. Loopy Belief Propagation Initialize all messages to 1 In some order of edges, update messages M ab;k = Σ i ψ a (l i )ψ ab (l i,l k )Π n≠b M na;i Until Convergence Rate of changes in messages < threshold

78. Loopy Belief Propagation VaVa VbVb VdVd VcVc M ab M bc M bc contains M ab M cd M da M cd contains M bc M da contains M cd Overcounting!!

79. Loopy Belief Propagation Initialize all messages to 1 In some order of edges, update messages M ab;k = Σ i ψ a (l i )ψ ab (l i,l k )Π n≠b M na;i Until Convergence Rate of changes in messages < threshold Not Guaranteed !!

80. Loopy Belief Propagation B’ ab (i,j) = Normalize to compute beliefs B a (i), B ab (i,j) B’ a (i) = ψ a (l i )ψ b (l j )ψ ab (l i,l j )Π n≠b M na;i Π n≠a M nb;j ψ a (l i )Π n M na;i At convergence Σ j B ab (i,j) = B a (i)