1. Surgical Planning Laboratory Brigham and Women’s Hospital Boston, Massachusetts USA a teaching affiliate of Harvard Medical School The AMIGO Project: An Advanced Multi-Modality Image-Guided Operating Suite Randy Ellis, Ph.D.

2. ©2005 Surgical Planning Laboratory, ARR Slide 2 AMIGO: Phase II Layout

3. ©2005 Surgical Planning Laboratory, ARR Slide 3 AMIGO: Central Operating Room

4. ©2005 Surgical Planning Laboratory, ARR Slide 4 AMIGO: Phase I View to 3T MRI

5. ©2005 Surgical Planning Laboratory, ARR Slide 5 AMIGO: Phase II View to PET/CT

6. ©2005 Surgical Planning Laboratory, ARR Slide 6 AMIGO Applications, Phase I (2005-2006) BRAIN: FUS (MRI, fMRI, DTI, T-MRI) BRAIN: Open Surgery (MRI, fMRI, DTI, US) BREAST: MRI-Guided Lumpectomy (MRI,3D-US,4D MRI of contrast uptake) BRAIN: Skull Base Surgery (MRI, fMRI, DTI, CT, US, Endoscope)

7. ©2005 Surgical Planning Laboratory, ARR Slide 7 AMIGO Applications, Phase II (2006-2007) BREAST CANCER: FUS LIVER THERMAL ABLATION KIDNEY THERMAL ABLATION CERVICAL CANCER BRACHYTHERAPY BONE METASTATSIS THERMAL ABLATION BONE TUMOR FUS VERTEBROPLASTY PROSTATE BIOPSY PROSTATE BRACHYTHERAPY PROSTATE FUS

8. ©2005 Surgical Planning Laboratory, ARR Slide 8 AMIGO Applications, Phase II (2007-2008) SPINE SURGERY NEUROVASCULAR INTERVENTIONS CRANIOFACIAL SURGERY LIVER ENDSOCOPIC SURGERY KIDNEY FUS LUNG THERMAL ABLATION LUNG BRONCHOSCOPY JOINT ENDOSCOPIC SURGERY CARDIAC EP ABLATION AVM EMBOLIZATION, AVM FUS

9. ©2005 Surgical Planning Laboratory, ARR Slide 9 Image Intensive Environment Preoperative images: – diagnostic & planning Intraoperative images: – Updating anatomical changes – Functional monitoring Postoperative images: – Verification

10. ©2005 Surgical Planning Laboratory, ARR Slide 10 Imaging Modalities Unite modalities in a patient coordinate frame: MR for soft tissues PET for metabolic imaging CT for bone surfaces Fluoroscopy for updates 3D ultrasound for soft tissues Microscopes/endoscopes for sight 1D signals (EEG, ECG, etc.)

11. ©2005 Surgical Planning Laboratory, ARR Slide 11 Integrated Imaging and Intervention + Image/patient registration problem disappears + Real-time monitoring of procedures - Visualization challenges + Multi-system assessment and treatment

12. ©2005 Surgical Planning Laboratory, ARR Slide 12 Interoperability is Key - Information management Maintain registration across rooms Multiple software systems – GE/NAV for established procedures – 3D Slicer, etc. for novel applications Multiple localization technologies – Primarily EM & optical – Robot also localizes

13. ©2005 Surgical Planning Laboratory, ARR Slide 13 The AMIGO Navigator: An Overview of the Software Development Process Randy Ellis, Ph.D.

14. ©2005 Surgical Planning Laboratory, ARR Slide 14 Software is Expensive and Complex 31% were abandoned before delivery Over 57% were a minimum of 189% over budget Since then: overall, limited progress Source: The Standish Group An extensive 1994 study of thousands of US industrial projects found:

15. ©2005 Surgical Planning Laboratory, ARR Slide 15 Our Guiding Principles Open Source: – Collaborative process – Free to commercialization Robust: – Use NAMIC/Kitware infrastructure – Automated testing of all changes Extensible: – Gain from ongoing Slicer activities – Expect to add new imaging modalities – Expect new interventional procedures

16. ©2005 Surgical Planning Laboratory, ARR Slide 16 Application-Oriented Architecture Base (common) software MRI Biopsy app. Neurosurgery app.Breast Surgery app.

17. ©2005 Surgical Planning Laboratory, ARR Slide 17 Storyboards for Applications Define the basic approach to the interventional procedure Prototype screen appearances (paper cartoons or PowerPoint are useful) Specify screen-to-screen actions Implement “happy-day” scenario Elaborate alternative workflows (e.g., error handling, new image acquisitions) Result: fast development of well tested software Engineer and clinicians work together:

18. ©2005 Surgical Planning Laboratory, ARR Slide 18 Technical Overview Requirements drive the process Application-specific: – Rapid specification of ideal workflow – Catch design problems early – Elaborate necessary extensions only, not all possible variants Test-oriented: – The specification produces tests before code is written – Repeat all tests whenever changes are made Result: robust, trustable systems

19. ©2005 Surgical Planning Laboratory, ARR Slide 19 Example Application: CT-Guided Brain Tumor Biopsy

20. ©2005 Surgical Planning Laboratory, ARR Slide 20 Clinical Problem: Biopsy in CT Scanner Idea: Perform brain biopsy using 2D slices Technique: Attach fiducials to head holder Scan holder and patient Select slices showing structures at risk Superimpose needle trajectory on slices Verify biopsy with new CT acquisition(s)

21. ©2005 Surgical Planning Laboratory, ARR Slide 21 CT Biopsy Phase 1: Procedure Workflow 1.Prepare patient (including fiducials) 2.Transfer patient to CT scanner 3.Acquire the CT scan (?inject?, image) 4.Load scan into Navigator 5.Software calculates registration 6.Software displays needle trajectory in CT 7.Perform the biopsy 8.Close the patient

22. ©2005 Surgical Planning Laboratory, ARR Slide 22 CT Biopsy Phase 2: Software Actions 1.Load instrument descriptions 2.Load CT scan 3.Detect fiducials 4.Calculate registration 5.Select key CT slices 6.Track the needle guide 7.Overlay needle trajectory on slices

23. ©2005 Surgical Planning Laboratory, ARR Slide 23 CT Biopsy Phase 3: Procedure Steps 1.Screen to select scan from disk 2.Screen to give registration progress 3.Screen to select CT slices 4.Screen to display navigation Additional Requirements: – Forward/backward buttons – Exit button – Tool status display (e.g., visibility)

24. ©2005 Surgical Planning Laboratory, ARR Slide 24 LOADSELECTNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display CT Biopsy Storyboard: File Selection All we need here is a simple dialog box

25. ©2005 Surgical Planning Laboratory, ARR Slide 25 LOADSELECTNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display CT Biopsy Storyboard : Registration Need a progress bar during registration, then report RMS error after registration

26. ©2005 Surgical Planning Laboratory, ARR Slide 26 LOADSELECTNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display CT Biopsy Storyboard: Selection Idea #1 Show original slices in here SELECT SLICE 1/2/3/4  Scroll bar 

27. ©2005 Surgical Planning Laboratory, ARR Slide 27 LOADSELECTNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display CT Biopsy Storyboard: Selection Idea #2 Show original slices in here Drag/Drop into here  Scroll bar 

28. ©2005 Surgical Planning Laboratory, ARR Slide 28 LOADSELECTNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display CT Biopsy Storyboard: 2D Navigation Slice #1 Slice #3 Slice #2 Slice #4 Show the needle trajectory on top of each slice

29. ©2005 Surgical Planning Laboratory, ARR Slide 29 CT Biopsy Phase 5: Implement and Test 1.Create scans with/without fiducials (to test registration algorithms) 2.Build phantoms to measure navigation accuracy 3.Power cycle computer to test system robustness Unplug/plug tracking system, etc.

30. ©2005 Surgical Planning Laboratory, ARR Slide 30 CT Biopsy Phase 6: Next Iteration Next round, we may need to: Calibrate the biopsy needle intraoperatively Navigate the biopsy needle Use needle-oriented reslicing Migrate system to use MRI scans (new application: software re-use) Control MRI scanner for in-scanner biopsy (new application: software re-use)

31. ©2005 Surgical Planning Laboratory, ARR Slide 31 Example Application: Ventriculoscopy for Brain Tumor Biopsy

32. ©2005 Surgical Planning Laboratory, ARR Slide 32 Clinical Problem: Deep Brain Biopsy Idea: Use flexible scope to biopsy from within a brain ventricle Technique: Same setup as for CT biopsy navigation Show 3D location of scope inside volumetric image Needs calibration of optical image to tracking device

33. ©2005 Surgical Planning Laboratory, ARR Slide 33 Deep-Biopsy Phase 1: Procedure Workflow 1.Prepare patient (including fiducials) 2.Transfer patient to CT or MRI scanner 3.Acquire the 3D scan (inject & image) 4.Load scan into Navigator 5.Software calculates registration 6.Calibrate ventriculoscope 7.Software displays scope in volumetric scan 8.Perform the biopsy 9.Close the patient

34. ©2005 Surgical Planning Laboratory, ARR Slide 34 Deep-Biopsy Phase 2: Software Actions 1.Load instrument descriptions 2.Load volumetric (MRI/PET/CET) scan 3.Detect fiducials 4.Calculate registration 5.Calibrate ventriculoscope to tracker device 6.Confirm calibration using a sharp probe 7.Track the ventriculoscope 8.Show scope in volumetric (MRI/PET/CET) scan

35. ©2005 Surgical Planning Laboratory, ARR Slide 35 Deep-Biopsy Phase 3: Procedure Steps 1.Screen to select scan from disk 2.Screen to give registration progess 3.Screen to calibrate ventriculoscope 4.Screen to verify calibration 5.Screen to display navigation Additional Requirements: – Forward/backward/exit buttons – “Tabs” to other tasks  label “forward”

36. ©2005 Surgical Planning Laboratory, ARR Slide 36 LOADCALIBRATEVERIFYNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display Deep-Biopsy Storyboard: File Selection All we need here is a simple dialog box ACCEPT

37. ©2005 Surgical Planning Laboratory, ARR Slide 37 LOADCALIBRATEVERIFYNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display Deep-Biopsy Storyboard: Registration Need a progress bar during registration, then report RMS error after registration ACCEPT

38. ©2005 Surgical Planning Laboratory, ARR Slide 38 LOADCALIBRATEVERIFYNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display Deep-Biopsy Storyboard: Calibrate Vscope Show live video feed in here CALIBRATE

39. ©2005 Surgical Planning Laboratory, ARR Slide 39 LOADCALIBRATEVERIFYNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display Deep-Biopsy Storyboard: Verification 3D probe model on top of live video feed here ACCEPT

40. ©2005 Surgical Planning Laboratory, ARR Slide 40 LOADCALIBRATEVERIFYNAVIGATEREGISTER LOGO GOES HERE EXIT Tool Status Display Deep-Biopsy Storyboard: 3D Navigation Live video feed from scope Scope field-of- view cone inside volume Forward rendering from scope

41. ©2005 Surgical Planning Laboratory, ARR Slide 41 Deep-Biopsy Phase 5: Implement and Test 1.Create scans with/without fiducials (to test registration algorithms) 2.Calibrate optics to tracker device 3.Verify calibration with sharp probe 4.Build phantoms to measure navigation accuracy 5.Power cycle computer to test system robustness Unplug/plug tracking system, etc.

42. ©2005 Surgical Planning Laboratory, ARR Slide 42 Deep-Biopsy Phase 6: Next Iteration Next round, we may need to: Calibrate the biopsy needle Navigate the biopsy needle Use needle-oriented reslicing Acquire/integrate intraoperative MRI (new application: software re-use) Migrate to laparoscopic ultrasound (new application: software re-use)