1. Segmentation of Tree like Structures as Minimisation Problem applied to Lung Vasculature Pieter Bruyninckx

2. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Overview Introduction –What is vessel segmentation –Clinical applications (lungs) Possible approaches –Single Vessel –Tracking – Bottom-up  –Vessel Enhancement Filter Methodological Approach –Initialisation Segmentation of whole lung Initial vesselness Initial orientation + uncertainty –Optimisation Energy function Optimisation methods –Results Discussion –Current problems (bronchi, speed) Speed up through multiresolution? –Future Work Validation

3. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Overview Introduction –What is vessel segmentation –Lung anatomy and pathology State of the Art Minimisation Approach Discussion

4. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Introduction Vessel segmentation –Extracting vessels from an image (3D) –Multiple representations possible Hard: Centerline, border Soft: Probability Use general properties –Intensity range –Tubular shape –Tree like structure Connectedness Bifurcations  Pathological cases?

5. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Lung anatomy and pathology General Structure –Vessels –Bronchi Common pathology –Mosaic Perfusion Pulmonary embolism Small airways disease Emphysema

6. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Overview Introduction State of the Art –Top-down (initialisation) Single Vessel Tracking –Bottom-up (no initialisation) Vessel Enhancement Filter Tschirren Minimisation Approach Discussion

7. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Single Vessel Start point and end point –Shortest distance problem Methods –Shortest path Limitations –Whole vessel tree? [Wink2002]

8. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Top-down: Single Vessel Initialisation –Start and end-point Iteration –Optimal path –Minimal energy –Intensity, smoothness Challenges –Whole vessel tree? [Wink2002]

9. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Top-down: Tracking Initialisation –Start point (tree root) Iteration –Centerline tracking –More advanced Region growing Wave front propagation Level sets Challenges –Handling bifurcations –Mathematical framework [Wink2000]

10. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Wave Front Propagation Object: front –Moves outwards –Varying speed Limitations –Freezing needed –Line-structure lost Improvements –Freezing criterion More advanced: Level sets No Freezing With freezing [Deschamps2002]

11. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Bottom-up: Vessel Enhancement Filters Soft segmentation = vessel enhancement: –Improve hard segmentation –Improve visualization No initialisation Single iteration –Eigenvalues/vectors Hessian Vesselness and orientation –Multiresolution approach –Anisotropic filtering Challenges –Physical units? –Sensitive to noise (second derivatives) [Frangi1998]

12. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Bottom-up: Tschirren Initialisation –Detection of probably- vessel voxels (intensity) –Compute orientation –Classify: vessel, junction, nodule Iteration –Join points into a tree Challenge –Rather ad hoc method [Tschirren2005]

13. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Overview Introduction State of the Art Minimisation Approach –Pre-processing –Initialisation –Iteration –Results Discussion

14. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Design Decisions Bottom-up Approach –Complex vessel structure –No missing branches Minimisation problem –Allow for sound mathematical framework No multiresolution –Tends to deform vessels

15. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Minimisation Approach Initial vesselness (intensity only) Patient scan Initial vessel orientation (with uncertainty) Final vesselness and orientation through energy optimization Energy: function of neighbourhood and local vessel orientation and vesselness Lung Segmentation

16. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Lung Segmentation Goal –Separate lung from other tissue –Allow for intensity-based vessel segmentation –Increased efficiency Implementation –Simple ad hoc algorithm

17. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Minimisation Approach Initial vesselness (intensity only) Patient scan Initial vessel orientation (with uncertainty) Final vesselness and orientation through energy optimization Energy: function of neighbourhood and local vessel orientation and vesselness Lung Segmentation

18. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Vesselness Initialisation Based on intensities only First step –Determine average and standard deviation of local background (10x10x10 mm³) Second step –Express vesselness as a ‘probability’  [0,1] (function of background ,  and intensity)

19. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Vesselness Initalisation

20. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Vesselness Initialisation

21. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Minimisation Approach Initial vesselness (intensity only) Patient scan Initial vessel orientation (with uncertainty) Final vesselness and orientation through energy optimization Energy: function of neighbourhood and local vessel orientation and vesselness Lung Segmentation

22. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Initial Vessel Orientation Estimate –Orientation –Uncertainty Method –Compute gradients  perpendicular orientation –Hessian approach also possible

23. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Minimisation Approach Initial vesselness (intensity only) Patient scan Initial vessel orientation (with uncertainty) Final vesselness and orientation through energy optimization Energy: function of neighbourhood and local vessel orientation and vesselness Lung Segmentation

24. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Iteration: Energy Function Energy Function f –Energy ~ Vessel-likeness  internal energy E E( ) < E( ) –Energy ~ ‘distance’ to original  distance D D(, ) < D(, )

25. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Iteration: Energy Function n i v

26. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Iteration: Energy Function d(, )

27. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Iteration: Energy Function Distance function D –Extension of d –g n : main directions (orthogonal) ( )

28. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Iteration: Energy Function No uncertainty No certainty Some uncertainty

29. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Optimisation: Method Possible approaches –Simulated annealing (SA) Slow –Local alignment Fast Global optimum? Proposed solution –Local everywhere –SA at ‘difficult’ locations  Best of both worlds If difficult locations can be found (high local energy)

30. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Results: 2D originalα = 0.1α = 0.4α = 0.6α = 0.8 α = 0.9 α = 0.95α = 0.99

31. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Results: 2D (detail) original α = 0.99

32. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Visualisation: 3D Result: soft segmentation Visualisation –Volume rendering (Voxar3D) –Intensity = vesselness –(no orientation information) –Window/level (can't see the wood for the trees)

33. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Results: 3D(Overview)

34. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Results: 3D(Detail)

35. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Overview Introduction State of the Art Minimisation Approach Discussion –Current challenges –Future Work

36. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Current Problems Bronchi –Tubular structures –Walls are likely to be enhanced –Multiresolution needed? Speed –Processing time > 24 h Rewrite code partially in C More efficient optimisation

37. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Current Problem: Discrete weights Current weights –Influence of all neighbours  Vessels get wider Discrete weights –Look at two neighbors –Smaller vessels possible –Better delineation –Efficient local alignment?

38. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Future Work: Liver and Bronchi Liver vasculature –Complex tree like structure Arterial and venous –More ‘noisy’ background Bronchi –Complex tree like structure –Intensity based classification difficult Low intense centre surrounded by high intense wall (partial volume artefacts) –Multiresolution?

39. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Future work Multiresolution –Separation: Vessel wall  Vessel –Improve speed? Model extensions –Synchronous segmentation: Vessels and Bronchi –Bifurcation detection

40. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Future Work Post processing –Multitude of information Vesselness Orientation (3 degrees of freedom / voxel) –Advanced visualisation –Improved hard segmentation algorithms Validation

41. Pieter.Bruyninckx@uz.kuleuven.ac.be Vessel segmentation Introduction State of the Art Minimisation Approach Discussion Thank you for your attention Questions ?