1. 2 University of Iowa Iowa City, IA 52246 2 University of Iowa Iowa City, IA 52246 William E. Higgins 1,2 Anthony J. Sherbondy 1 James P. Helferty 1 Atilla P. Kiraly 1 Geoffrey McLennan 2 Eric A. Hoffman 2 Janice Z Turlington 1 William E. Higgins 1,2 Anthony J. Sherbondy 1 James P. Helferty 1 Atilla P. Kiraly 1 Geoffrey McLennan 2 Eric A. Hoffman 2 Janice Z Turlington 1 RSNA 2000 Chicago, IL 26 November 2000 RSNA 2000 Chicago, IL 26 November 2000 1 Penn State University University Park, PA 16802 16802

2. 1.Introduce the feasibility of PC-based virtual endoscopy for both 3D CT assessment and live bronchoscopy. 2.Demonstrate the concept of a multimedia case study for CT-based report generation and bronchoscopic guidance. 3.Describe a method for linking 3D CT data to live bronchoscopic video. 1.Introduce the feasibility of PC-based virtual endoscopy for both 3D CT assessment and live bronchoscopy. 2.Demonstrate the concept of a multimedia case study for CT-based report generation and bronchoscopic guidance. 3.Describe a method for linking 3D CT data to live bronchoscopic video. (1 of 32)

3. 1.Overview of virtual bronchoscopy and our system (Virtual Navigator) 2.Stage-1 CT-only Analysis: Human case 3.Stage-2 Bronchoscopy examples: a.Human case b.Phantom and animal studies 1.Overview of virtual bronchoscopy and our system (Virtual Navigator) 2.Stage-1 CT-only Analysis: Human case 3.Stage-2 Bronchoscopy examples: a.Human case b.Phantom and animal studies (2 of 32)

4.  virtual copy of chest anatomy Input:Input: high-resolution 3D radiologic chest image  permits unlimited “exploration”  no risk to patient  permits unlimited “exploration”  no risk to patient Explore:Explore: the virtual anatomy using computer (3 of 32)

5.  Permit CT-only analysis No link to follow-on live bronchoscopy No link to follow-on live bronchoscopy  Limited quantitative path planning to interesting sites  Do not provide complete examination package  Often require expensive computers  Permit CT-only analysis No link to follow-on live bronchoscopy No link to follow-on live bronchoscopy  Limited quantitative path planning to interesting sites  Do not provide complete examination package  Often require expensive computers (4 of 32)

6. (5 of 32)  Complete CT examination  Guide live bronchoscopy  Automate steps in CT assessment  Inexpensive, PC-based  Complete CT examination  Guide live bronchoscopy  Automate steps in CT assessment  Inexpensive, PC-based

7.  Build with Graphics/Processing Tools 3D CT assessment Guide bronchoscopy  Supplemental plan  Multimedia report (6 of 32)

8. 3.Reporting Abstractions  Snapshots  Plots  Movies  Case Notes  Measurements 3.Reporting Abstractions  Snapshots  Plots  Movies  Case Notes  Measurements 1.Data Sources  3D CT Image  Bronchoscopic Video Video 1.Data Sources  3D CT Image  Bronchoscopic Video Video 2.Data Abstractions  Root Site  Key Sites  Paths  Tree 2.Data Abstractions  Root Site  Key Sites  Paths  Tree (7 of 32)

9. VirtualscopeVirtualscope 3D Surface Tool Study Manager Projection Tools (8 of 32)

10. Sliding Slab Depth Tools Plot Tool Oblique Cross Sections Slicer Tools (MPR Views) (9 of 32)

11. CT-Video Live Match Tool Cube Tool Tube Viewer Notes Tool (10 of 32)

12. Examine case and plan bronchoscopy Virtual CT guidance of live bronchoscopy Stage 1: CT Assessment Stage 2: BronchoscopyBronchoscopy (11 of 32)

13. 1. Create new Case Study. 2. Compute guidance data. 3. Build complete Case Study. 1. Create new Case Study. 2. Compute guidance data. 3. Build complete Case Study. Stage 1: CT Assessment 1. Load Case Study. 2. Set up graphical tools. 3. Perform virtual-guided bronchoscopy. 1. Load Case Study. 2. Set up graphical tools. 3. Perform virtual-guided bronchoscopy. Stage 2: BronchoscopyBronchoscopy (12 of 32)

14. a.Build Case Study registry B B Create Case Create Case b.Set up for computing guidance data S S Set up Set up 1.Create new Case Study. 2. Automatically compute guidance data. 3. Build complete Case Study. A A B B (13 of 32)

15. Determine global 3D interior detail Travel through virtual airway paths. View structure relations within their environment. Study interior path dynamics. Extract paths through airways. Render airway tree with paths, traverse pathways. (14 of 32)

16. Patient underwent EBCT scan (Electron Beam): single 20-sec breath-holdsingle 20-sec breath-hold 133 contiguous slices133 contiguous slices Reconstructed 3D CT image: Slice = 512X512 voxelsSlice = 512X512 voxels Slice thickness =1.5mmSlice thickness =1.5mm axial-plane [x-y] resolution = 0.586mm.axial-plane [x-y] resolution = 0.586mm. Patient underwent EBCT scan (Electron Beam): single 20-sec breath-holdsingle 20-sec breath-hold 133 contiguous slices133 contiguous slices Reconstructed 3D CT image: Slice = 512X512 voxelsSlice = 512X512 voxels Slice thickness =1.5mmSlice thickness =1.5mm axial-plane [x-y] resolution = 0.586mm.axial-plane [x-y] resolution = 0.586mm. Virtual Navigator shows Characteristics of collapseCharacteristics of collapse Condition of extended bronchial pathwaysCondition of extended bronchial pathways Virtual Navigator shows Characteristics of collapseCharacteristics of collapse Condition of extended bronchial pathwaysCondition of extended bronchial pathways (15 of 32)

17. (16 of 32) MPR Views Indicate Global 3D position MPR Views Indicate Global 3D position Depth- weighted Slab shows geometry Depth- weighted Slab shows geometry Same 3D site focused on by all tools.

18. (17 of 32) Selected 3D site (BLUE DOT) Highlighted on Five different tools at once. Selected 3D site (BLUE DOT) Highlighted on Five different tools at once.

19. Bottom to top view of collapse Site #99 Near carina, leaving collapse Approaching collapse Site #20 Site #47 Within tracheal collapse Site #86 Leaving trachea (18 of 32)

20. drop in area Approaching collapse Site #20 Within tracheal collapse Site #47 Plot clearly shows section where blockage occurs (19 of 32)

21. Pathology documented by captured Depth-Slab snapshots. Extent of collapse shown in rendered Airway tree. Extent of collapse shown in rendered Airway tree. (20 of 32)

22. Virtual Navigator shows Details of existing stentDetails of existing stent Basis for intervention analysisBasis for intervention analysis (laser therapy was performed) Virtual Navigator shows Details of existing stentDetails of existing stent Basis for intervention analysisBasis for intervention analysis (laser therapy was performed) Patient underwent EBCT scan (Electron Beam): single 20-sec breath-holdsingle 20-sec breath-hold 133 contiguous slices133 contiguous slices Reconstructed 3D CT image: Slice = 512X512 voxelsSlice = 512X512 voxels Slice thickness =1.5mmSlice thickness =1.5mm axial-plane [x-y] resolution = 0.586mm.axial-plane [x-y] resolution = 0.586mm. Patient underwent EBCT scan (Electron Beam): single 20-sec breath-holdsingle 20-sec breath-hold 133 contiguous slices133 contiguous slices Reconstructed 3D CT image: Slice = 512X512 voxelsSlice = 512X512 voxels Slice thickness =1.5mmSlice thickness =1.5mm axial-plane [x-y] resolution = 0.586mm.axial-plane [x-y] resolution = 0.586mm. (21 of 32)

23. (22 of 32) Stent visible In this and other views Stent visible In this and other views Same 3D site focused on by all tools.

24. (23 of 32) Stent encroaching on main carina

26. 3. Perform virtual-guided bronchoscopy. 2. Set up graphical tools. 1. Load Case Study. L L Load Case Load Case S S Graphical Tools Graphical Tools P P Virtual Guidance Virtual Guidance (24 of 32)

27. 2.LIVE bronchoscope video Video Match Tool shows a matched point between 3.Corresponding videobronchoscopy ( ROI superimposed) Virtual data guides airway traversal. airway traversal. Virtual data guides airway traversal. airway traversal. Coronal Projection shows extracted airway tree 1.CT rendering of airway region ( ROI rendered) 1.CT rendering of airway region ( ROI rendered) (25 of 32)

28. 1.Phantom 2.Animal 3.Human 1.Phantom 2.Animal 3.Human

29. Rubber phantom Experimental set-up: physician was blind to phantom physician was blind to phantom Experimental set-up: physician was blind to phantom physician was blind to phantom (26 of 32)

30. Extracted tree and paths Matched video frame with ROI RegisteredvirtualshotRegisteredvirtualshot (27 of 32)

31. Physician #1 (trial 1) Physician #1 (trial 2) Physician #2 Distance (mm) Time sec. Distance (mm) Time sec. Distance (mm) Time sec. 2.1812.6131.739.6722.0110.91 1.098.8650.978.7890.895.325  Average biopsy error: 1.98 mm  Average match time: 11.065 sec.  Average biopsy error: 1.98 mm  Average match time: 11.065 sec. Average Std Dev Distance and time measured to match each ROI target. Distance measured from line extrapolated from the needle direction to metal bead edge. Note: (28 of 32)

32. Registered virtual shot Registered LiveEndoscopicvideoLiveEndoscopicvideo Matched video frame with ROI (29 of 32)

33. Snapshots are misaligned to compensate for differing placement during CT scanning. Note: Actual site after guided dye marker placement. Planned site from CT analysis. (30 of 32)

34. (31 of 32)

35. (32 of 32) Bronchoscope video matched to rendered CT during live procedure.

36. National Center for Research Resources of the NIH (grant RR11800) National Center for Research Resources of the NIH (grant RR11800) Whitaker Foundation National Cancer Institute of the NIH (grant CA74325) National Cancer Institute of the NIH (grant CA74325) (End)(End)