1. Improved Moves for Truncated Convex Models M. Pawan Kumar Philip Torr

2. Aim Efficient, accurate MAP for truncated convex models V1V1 V2V2 ……… …………… …………… …………VnVn Random Variables V = { V 1, V 2, …, V n } Edges E define neighbourhood

3. Aim VaVa VbVb lili lklk  ab;ik Accurate, efficient MAP for truncated convex models  ab;ik = w ab min{ d(i-k), M }  ab;ik i-k w ab is non-negative Truncated Linear i-k  ab;ik Truncated Quadratic d(.) is convex  a;i  b;k

4. Motivation Low-level Vision Smoothly varying regions Sharp edges between regions min{ |i-k|, M} Boykov, Veksler & Zabih 1998 Well-researched !!

5. Things We Know NP-hard problem - Can only get approximation Best possible integrality gap - LP relaxation Manokaran et al., 2008 Solve using TRW-S, DD, PP Slower than graph-cuts Use Range Move - Veksler, 2007 None of the guarantees of LP

6. Real Motivation Gaps in Move-Making Literature LP Move- Making Potts Truncated Linear Truncated Quadratic 2 Multiplicative Bounds 2 + √2 O(√M) Chekuri et al., 2001

7. Real Motivation Gaps in Move-Making Literature LP Move- Making Potts Truncated Linear Truncated Quadratic 2 Multiplicative Bounds 2 2 + √2 2M O(√M)- Boykov, Veksler and Zabih, 1999

8. Real Motivation Gaps in Move-Making Literature LP Move- Making Potts Truncated Linear Truncated Quadratic 2 Multiplicative Bounds 2 2 + √2 4 O(√M)- Gupta and Tardos, 2000

9. Real Motivation Gaps in Move-Making Literature LP Move- Making Potts Truncated Linear Truncated Quadratic 2 Multiplicative Bounds 2 2 + √2 4 O(√M)2M Komodakis and Tziritas, 2005

10. Real Motivation Gaps in Move-Making Literature LP Move- Making Potts Truncated Linear Truncated Quadratic 2 Multiplicative Bounds 2 2 + √2 O(√M) 2 + √2 O(√M)

11. Outline Move Space Graph Construction Sketch of the Analysis Results

12. Move Space VaVa VbVb Initialize the labelling Choose interval I of L’ labels Each variable can Retain old label Choose a label from I Choose best labelling Iterate over intervals

13. Outline Move Space Graph Construction Sketch of the Analysis Results

14. Two Problems VaVa VbVb Choose interval I of L’ labels Each variable can Retain old label Choose a label from I Choose best labelling Large L’ => Non-submodular Non-submodular

15. First Problem VaVa VbVb Submodular problem Ishikawa, 2003; Veksler, 2007

16. First Problem VaVa VbVb Non-submodular Problem

17. First Problem VaVa VbVb Submodular problem Veksler, 2007

18. First Problem VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn

19. First Problem VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn

20. First Problem VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn

21. First Problem VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn

22. First Problem VaVa VbVb Model unary potentials exactly a m+1 a m+2 anan t b m+1 b m+2 bnbn

23. First Problem VaVa VbVb Similarly for V b a m+1 a m+2 anan t b m+1 b m+2 bnbn

24. First Problem VaVa VbVb Model convex pairwise costs a m+1 a m+2 anan t b m+1 b m+2 bnbn

25. First Problem VaVa VbVb Overestimated pairwise potentials Wanted to model  ab;ik = w ab min{ d(i-k), M } For all l i, l k  I Have modelled  ab;ik = w ab d(i-k) For all l i, l k  I

26. Second Problem VaVa VbVb Choose interval I of L’ labels Each variable can Retain old label Choose a label from I Choose best labelling Non-submodular problem !!

27. Second Problem - Case 1 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn s ∞∞ Both previous labels lie in interval

28. Second Problem - Case 1 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn s ∞∞ w ab d(i-k)

29. Second Problem - Case 2 VaVa VbVb Only previous label of V a lies in interval a m+1 a m+2 anan t b m+1 b m+2 bnbn s ∞ ubub

30. Second Problem - Case 2 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn u b : unary potential of previous label of V b M s ∞ ubub

31. Second Problem - Case 2 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn M w ab d(i-k) s ∞ ubub

32. Second Problem - Case 2 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn M w ab ( d(i-m-1) + M ) s ∞ ubub

33. Second Problem - Case 3 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn Only previous label of V b lies in interval

34. Second Problem - Case 3 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn s uaua ∞ u a : unary potential of previous label of V a M

35. Second Problem - Case 4 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn Both previous labels do not lie in interval

36. Second Problem - Case 4 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn s uaua ubub P ab : pairwise potential for previous labels ab P ab M M

37. Second Problem - Case 4 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn w ab d(i-k) s uaua ubub ab P ab M M

38. Second Problem - Case 4 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn w ab ( d(i-m-1) + M ) s uaua ubub ab P ab M M

39. Second Problem - Case 4 VaVa VbVb a m+1 a m+2 anan t b m+1 b m+2 bnbn P ab s uaua ubub ab P ab M M

40. Graph Construction VaVa VbVb Find st-MINCUT. Retain old labelling if energy increases. a m+1 a m+2 anan b m+1 b m+2 bnbn t ITERATE

41. Outline Move Space Graph Construction Sketch of the Analysis Results

42. Analysis VaVa VbVb Current labelling f(.) Q C ≤ Q’ C VaVa VbVb Global Optimum f*(.) QPQP Previous labelling f’(.) VaVa VbVb

43. Analysis VaVa VbVb Current labelling f(.) Q C ≤ Q’ C VaVa VbVb Partially Optimal f’’(.) Previous labelling f’(.) VaVa VbVb Q’ 0 ≤

44. Analysis VaVa VbVb Current labelling f(.) Q P - Q’ C VaVa VbVb Partially Optimal f’’(.) Previous labelling f’(.) VaVa VbVb Q P - Q’ 0 ≥

45. Analysis VaVa VbVb Current labelling f(.) Q P - Q’ C VaVa VbVb Partially Optimal f’’(.) Local Optimal f’(.) VaVa VbVb Q P - Q’ 0 ≤ 0

46. Analysis VaVa VbVb Current labelling f(.) VaVa VbVb Partially Optimal f’’(.) Local Optimal f’(.) VaVa VbVb Q P - Q’ 0 ≤ 0 Take expectation over all intervals

47. Analysis Truncated Linear Q P ≤ 2 + max 2M, L’ L’M Q* L’ = M4Gupta and Tardos, 2000 L’ = √2M 2 + √2 Truncated Quadratic Q P ≤ O(√M) Q* L’ = √M

48. Outline Move Space Graph Construction Sketch of the Analysis Results

49. Synthetic Data - Truncated Linear Faster than TRW-S Comparable to Range Moves With LP Relaxation guarantees Time (sec) Energy

50. Synthetic Data - Truncated Quadratic Faster than TRW-S Comparable to Range Moves With LP Relaxation guarantees Time (sec) Energy

51. Stereo Correspondence Disparity Map Unary Potential: Similarity of pixel colour Pairwise Potential: Truncated convex

52. Stereo Correspondence AlgoEnergy1Time1Energy2Time2 Swap367820018.48370726820.25 Exp367795011.7336878748.79 TRW-S3677578131.653679563332.94 BP3789486272.065180705331.36 Range368684497.233679552141.78 Our3613003120.143679552191.20 Teddy

53. Stereo Correspondence AlgoEnergy1Time1Energy2Time2 Swap367820018.48370726820.25 Exp367795011.7336878748.79 TRW-S3677578131.653679563332.94 BP3789486272.065180705331.36 Range368684497.233679552141.78 Our3613003120.143679552191.20 Teddy

54. Stereo Correspondence AlgoEnergy1Time1Energy2Time2 Swap64522728.8670912020.04 Exp6349319.527233609.78 TRW-S63472094.86651696226.07 BP662108170.672155759244.71 Range63472039.7565169680.40 Our63472066.1365169680.70 Tsukuba

55. Summary Moves that give LP guarantees Similar results to TRW-S Faster than TRW-S because of graph cuts

56. Questions Not Yet Answered Move-making gives LP guarantees –True for all MAP estimation problems? Huber function? Parallel Imaging Problem? Primal-dual method? Solving more complex relaxations?

57. Questions? Improved Moves for Truncated Convex Models Kumar and Torr, NIPS 2008 http://www.robots.ox.ac.uk/~pawan/