2. Active Contours / Planes Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision http://robots.stanford.edu/cs223b Some slides taken from: Julien Jomier

3. Sebastian Thrun Stanford University CS223B Computer Vision Outline n Snakes n Planes

4. Sebastian Thrun Stanford University CS223B Computer Vision Introduction (1) The active contour model, or snake, is defined as an energy-minimizing spline. Active contours results from work of Kass et.al. in 1987. Active contour models may be used in image segmentation and understanding. The snake’s energy depends on its shape and location within the image. Snakes can be closed or open

5. Sebastian Thrun Stanford University CS223B Computer Vision Example

6. Sebastian Thrun Stanford University CS223B Computer Vision Introduction (2) Aorta segmentation using active contours

7. Sebastian Thrun Stanford University CS223B Computer Vision Introduction (3) First an initial spline (snake) is placed on the image, and then its energy is minimized. Local minima of this energy correspond to desired image properties. Unlike most other image models, the snake is active, always minimizing its energy functional, therefore exhibiting dynamic behavior. Also suitable for analysis of dynamic data or 3D image data.

8. Sebastian Thrun Stanford University CS223B Computer Vision Kass Algorithm The snake is defined parametrically as v(s)=[x(s),y(s)], where s  [0,1] is the normalized arc length along the contour. The energy functional to be minimized may be written as E cont = snake continuity E curv = snake curvature E image = image forces (e.g., edge attraction)

9. Sebastian Thrun Stanford University CS223B Computer Vision Internal Energy Continuity: Curvature:

10. Sebastian Thrun Stanford University CS223B Computer Vision Image Forces n Dark/Bright Lines n Edges

11. Sebastian Thrun Stanford University CS223B Computer Vision Trade-offs  determine trade-off

12. Sebastian Thrun Stanford University CS223B Computer Vision Numerical Algorithm n Select N initial locations p 1,…, p N n Update until convergence

13. Sebastian Thrun Stanford University CS223B Computer Vision Snakes, Done Right n Define Spline over p 1,…, p N n Optimize criterion for all points on spline n Allow for corners n (optimization becomes tricky, fills entire literature)

14. Sebastian Thrun Stanford University CS223B Computer Vision Applications Main Applications are: Segmentation Tracking Registration http://www.markschulze.net/snakes/

15. Sebastian Thrun Stanford University CS223B Computer Vision Examples (1) Julien Jomier

16. Sebastian Thrun Stanford University CS223B Computer Vision Examples (2) Julien Jomier

17. Sebastian Thrun Stanford University CS223B Computer Vision Examples (3) Heart Julien Jomier

18. Sebastian Thrun Stanford University CS223B Computer Vision Examples (4) 3D Segmentation of the Hippocampus Julien Jomier

19. Sebastian Thrun Stanford University CS223B Computer Vision Examples (5) Julien Jomier

20. Sebastian Thrun Stanford University CS223B Computer Vision Example (6) Julien Jomier

21. Sebastian Thrun Stanford University CS223B Computer Vision Example (7) Julien Jomier

22. Sebastian Thrun Stanford University CS223B Computer Vision Problems with snakes Snakes sometimes degenerate in shape by shrinking and flattening. Stability and convergence of the contour deformation process unpredictable. Solution: Add some constraints Initialization is not straightforward. Solution: Manual, Learned, Exhaustive

23. Sebastian Thrun Stanford University CS223B Computer Vision References M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. In Proc. 1st ICCV, pages 259-268, June 1987. London, UK. Yongjik Kim. A summary of Implicit Snake Formulation. Jorgen Ahlberg. Active Contours in Three Dimensions. M. Bertalmio, G. Sapiro and G. Randall. Morphing Active Contours. IEEE PAMI, Vol 22, No 7, July 2000

24. Sebastian Thrun Stanford University CS223B Computer Vision Outline n Snakes n Planes

25. Sebastian Thrun Stanford University CS223B Computer Vision Finding Planes n Lines in Image (e.g., Hough transform) n Planes in 3D space (e.g., stereo reconstruction, SFM) n Problems: –Number of planes –Parameters of planes –Data association: which point belongs to which plane –Outlier removal (noise, non-flat surfaces)

26. Sebastian Thrun Stanford University CS223B Computer Vision Range Data in Multi-Planar Environment Taken with Laser Range Finder, but similar for

27. Sebastian Thrun Stanford University CS223B Computer Vision Basic Idea ** ****

28. Sebastian Thrun Stanford University CS223B Computer Vision Mathematical Model: Expectation Maximization n 3D Model: Planar surface in 3D Distance point-surface surface normal  y x z displacement 

29. Sebastian Thrun Stanford University CS223B Computer Vision Mixture Measurement Model Case 1: Measurement z i caused by plane  j  Case 2: Measurement z i caused by something else

30. Sebastian Thrun Stanford University CS223B Computer Vision Measurement Model with Correspondences correspondence variables C : }

31. Sebastian Thrun Stanford University CS223B Computer Vision Expected Log-Likelihood Function …after some simple math mapping with known data association probabilistic data association

32. Sebastian Thrun Stanford University CS223B Computer Vision E-Step Calculate expectations with fixed model  : (normalize so that )

33. Sebastian Thrun Stanford University CS223B Computer Vision M-Step n Maximize subject to n Is equivalent to minimizing subject to

34. Sebastian Thrun Stanford University CS223B Computer Vision M-Step: Solve via Lagrange Multipliers n Define n And observe that

35. Sebastian Thrun Stanford University CS223B Computer Vision M-Step: Solve via Lagrange Multipliers Solve for  : n Substitute back: n Is of the linear form: Solution: two Eigenvectors of A with smallest Eigenvalues.

36. Sebastian Thrun Stanford University CS223B Computer Vision Determining Number of Surfaces J =1 First model component * * J =1 E-Step * * J =1 M-Step * * J =3 Add model components J =3 E-Step J =3 M-step J =1 Prune model J =3 Add model components J =3 E/M Steps * J =2 Prune model

37. Sebastian Thrun Stanford University CS223B Computer Vision Choosing the “Right” Number of Planes: AIC J=2J=3J=5J=0J=1J=4 increased data likelihoodincreased prior probability

38. Sebastian Thrun Stanford University CS223B Computer Vision Model Selection Approximately every 20 iterations of EM: n Start new surfaces –Near any set of collinear measurements n Terminate unsupported surfaces –If not supported by enough measurements –If density of measurements too low –If two planes are too close to each other

39. Sebastian Thrun Stanford University CS223B Computer Vision Results

40. Sebastian Thrun Stanford University CS223B Computer Vision Online Robotic Mapping @ CMU

41. Sebastian Thrun Stanford University CS223B Computer Vision Online Robotic Mapping @ Stanford