1. Level Set Methods and Fast Marching Methods Wen Hongwei

2. Reference “Level Set and PDE Methods for Computer Graphics” Leopold K¨uhschelm Sebastian Zambal“Level Set and PDE Methods for Computer Graphics” Leopold K¨uhschelm Sebastian Zambal Institute of Computer Graphics Vienna University of Technolgy Vienna / Austria > >

3. Problem Definition In 2-D, Curve (Interface) evolution over time.In 2-D, Curve (Interface) evolution over time. In 3-D, Surface (Interface) evolution.In 3-D, Surface (Interface) evolution. Example: Waves in the ocean.Example: Waves in the ocean. Waves can be uniquely modeled if we track the boundary between air and water (the interface).Waves can be uniquely modeled if we track the boundary between air and water (the interface).

4. How to Model and Implement Model the physical process and interface motion using partial derivative equations (PDE).Model the physical process and interface motion using partial derivative equations (PDE). Level Set Methods are used for the implementation of curve/interface evolution under various forces.Level Set Methods are used for the implementation of curve/interface evolution under various forces. Challenges:Challenges: Numerical errors due to discretization.Numerical errors due to discretization. Estimation of derivatives around discontinuities.Estimation of derivatives around discontinuities. Numerical stability of the implementation.Numerical stability of the implementation.

5. Curve representation A closed curve cannot be represented as a 1-D function y = f(x)A closed curve cannot be represented as a 1-D function y = f(x) A parametric representation can be used:A parametric representation can be used: C = {x(p), y(p)}C = {x(p), y(p)} p is in [0,1]p is in [0,1] C c

6. Curve Representation Explicit (parametric) representationExplicit (parametric) representation Uses a parametric representationUses a parametric representation Sample the curve at fixed intervals.Sample the curve at fixed intervals. Problem: During evolution, curve changes, so does the parameterization.Problem: During evolution, curve changes, so does the parameterization. Implicit representation (Level set methods)Implicit representation (Level set methods) Embed the curve C into a 2-D function z = f(x,y)Embed the curve C into a 2-D function z = f(x,y)

7. Implicit Methods Curve is embedded as the 0 level set.Curve is embedded as the 0 level set. C = {(x,y) | f(x,y) = 0}C = {(x,y) | f(x,y) = 0} C c

8. Implicit Methods A better embedding function is smootherA better embedding function is smoother C c

9. Level Set Methods Main idea:Main idea: Evolve the embedding function f(x,y)Evolve the embedding function f(x,y) Keep track of its zero level setKeep track of its zero level set This can thought as the diffusion of a 2-D surface.This can thought as the diffusion of a 2-D surface.

10. Level Set Methods Example: Propagation of two flamesExample: Propagation of two flames

11. Level Set Methods The level set approach: The level set approach: Define problem in one higher dimensionDefine problem in one higher dimension Define level set function =  (= 0)Define level set function z =  (x,y,t = 0) where the () plane contains the contour, and where the (x,y) plane contains the contour, and = signed Euclidean distance transform value (negative means inside closed contour, positive means outside contour) z = signed Euclidean distance transform value (negative means inside closed contour, positive means outside contour)

12. How to Move the Contour? Move the level set function,  (), so that it rises, falls, expands, etc. Move the level set function,  (x,y,t), so that it rises, falls, expands, etc. Contour = cross section at = 0, i.e., Contour = cross section at z = 0, i.e., {(x,y) |  (x,y,t) = 0}

13. Level Set Surface The zero level set (in blue) at one point in time as a slice of the level set surface (in red) The zero level set (in blue) at one point in time as a slice of the level set surface (in red)

14. Level Set Surface Later in time the level set surface (red) has moved and the new zero level set (blue) defines the new contour Later in time the level set surface (red) has moved and the new zero level set (blue) defines the new contour

15. Level Set Surface

16. How to Move the Level Set Surface? 1.Define a velocity field,, that specifies how contour points move in time 1.Define a velocity field, F, that specifies how contour points move in time Based on application-specific physics such as time, position, normal, curvature, image gradient magnitudeBased on application-specific physics such as time, position, normal, curvature, image gradient magnitude 2.Build an initial value for the level set function,  (=0), based on the initial contour position 2.Build an initial value for the level set function,  (x,y,t=0), based on the initial contour position 3.Adjust  over time; contour at time t defined by  () = 0 3.Adjust  over time; contour at time t defined by  (x(t), y(t), t) = 0 Hamilton-Jacobi equation

17. Level Set Formulation Constraint: level set value of a point on the contour with motion x(t) must always be 0 Constraint: level set value of a point on the contour with motion x(t) must always be 0  (x(t), t) = 0 By the chain rule By the chain rule  t +  (x(t), t) · x(t) = 0 (1) Since F supplies the speed in the outward normal direction Since F supplies the speed in the outward normal direction x(t) · n = F, where n =  / |  | (2) Hence evolution equation for  is Hence evolution equation for  is  t + F|  | = 0 (3)

18. Speed Function The speed function F is the surface normal direction, control the movement of the curve, including: The speed function F is the surface normal direction, control the movement of the curve, including: image-related items (such as gradient information),image-related items (such as gradient information), the geometric shape of the curve (such as the curvature of the curve),the geometric shape of the curve (such as the curvature of the curve), the additional propagation terms.the additional propagation terms.

19. Example: Segmentation Digital Subtraction Angiogram Digital Subtraction Angiogram based on image gradient and contour curvature F based on image gradient and contour curvature

20. Example (cont.) Initial contour specified manually Initial contour specified manually

21. Narrow Band Central idea: we are interested on the motion of the zero-level set and not for the motion of each iso-phote of the surfaceCentral idea: we are interested on the motion of the zero-level set and not for the motion of each iso-phote of the surface Extract the latest positionExtract the latest position Define a band within a certain distanceDefine a band within a certain distance Update the level set functionUpdate the level set function Check new position with respectCheck new position with respect the limits of the band the limits of the band Update the position of the bandUpdate the position of the band regularly, and re-initialize the implicit function regularly, and re-initialize the implicit function

22. Narrow Band Significant decrease on the computational complexity, in particular when implemented efficiently and can account for any type of motion flowsSignificant decrease on the computational complexity, in particular when implemented efficiently and can account for any type of motion flows

23. Fast Marching Method J. Sethian, 1996 J. Sethian, 1996 Special case that assumes the velocity field,, never changes sign. That is, contour is either always expanding (F>0) or always shrinking (F 0) or always shrinking (F<0) Convert problem to a stationary formulation on a discrete grid where the contour is guaranteed to cross each grid point at most once Convert problem to a stationary formulation on a discrete grid where the contour is guaranteed to cross each grid point at most once

24. Fast Marching Method Compute () = time at which the contour crosses grid point () Compute T(x,y) = time at which the contour crosses grid point (x,y) At any height, t, the surface gives the set of points reached at time t At any height, t, the surface gives the set of points reached at time t

25. Fast Marching Algorithm Compute T using the fact that Compute T using the fact that Distance = rate × timeDistance = rate × time In 1D: 1 = F × dT/dx (1)In 1D: 1 = F × dT/dx (1) In 2D: 1 = F × |  T | (2)In 2D: 1 = F × |  T | (2) Contour at time t = Contour at time t = {(x,y) | T(x,y) = t} {(x,y) | T(x,y) = t}

26. Fast Marching Algorithm Construct the arrival time surface () incrementally:Construct the arrival time surface T(x,y) incrementally: 1.Build the initial contour 2.Incrementally add on to the existing surface the part that corresponds to the contour moving with speed (in other words, repeatedly pick a point on the fringe with minimum value) 2.Incrementally add on to the existing surface the part that corresponds to the contour moving with speed F (in other words, repeatedly pick a point on the fringe with minimum T value) 3.Iterate until goes to 0 3.Iterate until F goes to 0 Builds level set surface by “scaffolding” the surface patches farther and farther away from the initial contourBuilds level set surface by “scaffolding” the surface patches farther and farther away from the initial contour

27. Fast Marching Update “downwind” (i.e., unvisited neighbors) Compute new possible values

28. Fast Marching Expand point on the fringe with minimum value Update neighbors “downwind”

29. Fast Marching Expand point on the fringe with minimum value Update neighbors “downwind”

30. Summary Fast Marching methods are: Very efficient – also in 3D Very efficient – also in 3D Of an ’open architecture’ in terms of speed functions Of an ’open architecture’ in terms of speed functions Dealing with sharp corners and changes in topology Dealing with sharp corners and changes in topology Widely applicable: image segmentation, robotics, seismic travel times, shortest path finding etc. Widely applicable: image segmentation, robotics, seismic travel times, shortest path finding etc. In essence a smart way to solve the Eikonal equation In essence a smart way to solve the Eikonal equation

31. Results: Segmentation using Fast Marching No level set tuning No level set tuning

32. Results: Vein Segmentation No level set tuning With level set tuning No level set tuning With level set tuning

33. Results: Vein Segmentation (continued) Original Fast Marching +Level Set only Level Set Tuning Original Fast Marching +Level Set only Level Set Tuning

34. Results: Segmentation using Fast Marching No level set tuning No level set tuning

35. Results: Brain Image Segmentation # of iterations = 9000 # of iterations = 12000 Fast marching only, no level set tuning # of iterations = 9000 # of iterations = 12000 Fast marching only, no level set tuning

36. Results: Brain Segmentation (continued) Without level set tuning With level set tuning

37. Conclusion Level Set methods are used for implementing curve evolution or diffusion.Level Set methods are used for implementing curve evolution or diffusion. Level Set Methods can also be used for surface evolutionLevel Set Methods can also be used for surface evolution Applications in Computer Graphics.Applications in Computer Graphics. Volume segmentation in 3-D images.Volume segmentation in 3-D images. Matlab toolbox for trying some examples of level set methods can be downloaded from:Matlab toolbox for trying some examples of level set methods can be downloaded from: http://vision.ece.ucsb.edu/download.htmlhttp://vision.ece.ucsb.edu/download.htmlhttp://vision.ece.ucsb.edu/download.html