1. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology IGT System Engineering Peter Kazanzides Assistant Research Professor Department of Computer Science Johns Hopkins University October 19, 2006

2. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology My Background 1989-1990 Postdoctoral research at IBM on ROBODOC 1990-2002 Co-Founder of Integrated Surgical Systems –Commercial development of ROBODOC® System –Commercial sales in Europe (CE Mark) –Clinical trials in U.S. and Japan 2002-present Research faculty at JHU CISST ERC –Lead engineering infrastructure –Develop image-guided robots

3. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Outline What is System Engineering? Major activities: –Requirements –Risk Analysis –Architecture –Modeling / Simulation –Verification and Validation Case study: Image-guided robot for rodent research Current work: Surgical Assistant Workstation Summary Three challenges and opportunities for assistance Four

4. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology What is System Engineering? Customer Needs State the Problem Investigate Alternatives Model the system Integrate Assess Performance Launch System Outpu t Re-evaluate The Systems Engineering Process from A. T. Bahill and B. Gissing, “Re-evaluating systems engineering concepts using systems thinking”, IEEE Transaction on Systems, Man and Cybernetics, Part C: Applications and Reviews, 28 (4), 516-527, 1998. INCOSE: International Council on Systems Engineering (www.incose.org) Re-evaluate Spans the entire development process Considers the entire system, including hardware and software (interdisciplinary) The SIMILAR Process

5. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology System Engineering Activities Requirements Risk analysis Architecture Modeling / simulation Verification and Validation Level of effort will vary based on factors such as: –University or industry project –Plans for clinical evaluation –Cohesiveness of developers and users

6. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Requirements Informal (undocumented) is fine for early prototypes Documented requirements necessary for: –Any system for clinical use –Any development involving multiple/distributed parties (e.g., university researchers, industry, clinicians) Requirements may not be necessary for toolkits

7. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Risk Analysis Should be performed by cross-functional team, including application expert Failure Mode Effects (and Criticality) Analysis (FMEA/FMECA) –Bottom up analysis: for each component failure, determine (potential) system failure –Most useful in design phase (proactive) Fault Tree Analysis (FTA) –Top down analysis: trace each system failure down to components –Most useful for after-the-fact analysis

8. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Architecture Often refers to software, but can include hardware elements Simple definition: “What the pieces are and how they work together” From Software Engineering Institute (SEI) at CMU: Software architecture is the set of design decisions which, if made incorrectly, may cause your project to be cancelled – Eoin Wood

9. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Architecture: 4+1 View From: P. Kruchten, “The 4+1 View Model of Architecture”, IEEE Software, 12(6), Nov 1995. Other approaches include RM-ODP, Zachman framework

10. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Modeling and Simulation Create models of system to guide development Model Driven Architecture (MDA) ® * –Model is enduring asset –Perform simulation/testing with model –Generate code from model –Is the technology (tool set) there yet? “Processes, Methodologies, and Tools used for the Development of a Model Driven Architecture Based Open Software Framework for Distributed Medical Devices”, Amen Ra Mashariki, Ph.D. proposal, Morgan State University * Registered trademark of OMG

11. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Verification and Validation (V&V) NASA SATC: “differences between verification and validation are unimportant except to the theorist” Two primary V&V activities: –Reviews (inspections, walkthroughs) –Testing (e.g., against requirements)

12. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Case Study: Image-Guided Robot for Rodent Research Initial application: correlate pO 2 measurements with PET values to validate non-invasive method for locating hypoxic tumor regions PET pixel value pO 2 probe reading

13. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Image-Guided Robot for Rodent Research: Requirements Distributed team: –Developers at JHU (Baltimore) –Customers (users) at Memorial Sloan Kettering Cancer Center (New York City) Requirement were critical: –First meeting: Sept 2003 –Three major revisions –Final version approved: March 2004 –System installed at MSKCC: Jan 2005

14. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Image-Guided Robot for Rodent Research: Physical Architecture PC (Windows) Servo Control and Amps (Galil) DMC-2143 Robot 3D Slicer Ethernet

15. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology GUI Image-Guided Robot for Rodent Research: Development Architecture vtkRodent Region Growing Registration Executable mskccRobot Galil driverCISST 3D Slicer VTK TCL/TK Interpreter File I/O CISST LAPACK Ethernet wxPython wxWidgets Python Interpreter (IRE) File I/O Open source Proprietary to vendor Application-specific Application Environment Testing Environment Test Scripts

16. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Foundation libraries cisstCommon cisstVector cisstNumerical cisstInteractive Interactive Research Environment (IRE) CISST Software Package Real Time Support cisstOSAbstraction cisstDeviceInterface cisstRealtime Interventional Devices cisstStereoVision cisstTracker cisstRobot … Open Source www.cisst.org/cisst

17. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Image-Guided Robot for Rodent Research: Verification & Validation Well-tested toolkits (Slicer, VTK, CISST,…) System accuracy tests with phantom: –PET Fiducial Localization Error: 0.26 mm –Robot Fiducial Localization Error: 0.18 mm –Target Registration Error: 0.29 mm Customer acceptance testing “Design and Validation of an Image-Guided Robot for Small Animal Research,” MICCAI, Copenhagen, Denmark, Oct 2006.

18. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Intuitive API Intuitive Comm. Interface JHU Robot Interfaces API Emulator CISST – ISI “wrapper” Optimization Virtual Fixtures Video Subsystem Visualization Subsystem Research Applications and Subsystems Haptics Task modeling Skill Assessment Remote telesurgery Etc. Stereo processing Tool tracking Image registration Interactive visualization Ultrasound Endoscopes Microscopes HMD; Stereo TV ISI License Surgical Assistant Workstation

19. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Surgical Assistant Workstation NSF supplement to CISST ERC –Started Sept 2006 Collaborative effort between JHU and Intuitive Surgical Currently working on formal specifications –Functional specification –Architecture views (4+1)

20. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Summary System engineering integrates multiple disciplines over the development life cycle Key activities include requirements, modeling, architecture, verification & validation –What’s needed depends on development scenario But, there are challenges in all of these activities…

21. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology DICOM Interfaces / Middleware HL7 Input Device(s) Display(s) Electronic Records Anatomical Atlas Patient Database Real-Time Data Distribution Intraoperative Imaging (e.g., X-ray, ultrasound, microscope video) Image Acquisition Image Processing Image Overlay Trajectory Control Servo Control Medical RobotNavigation or Tracking System Visualization/GUI (Navigation System) Target Positions Preoperative Data and Models DICOM System Configuration Procedural Logic (e.g., Task Graph) Data Logging Visualization/GUI Application Controller Challenge #1: HW and SW for IGT Systems Preoperative Images

22. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Challenge #1: HW and SW for IGT Systems Intraoperative imaging –Hardware exists (X-ray, US, video, …, CT, MRI) –Need better data access (e.g., US RF) Navigation/tracking –Hardware exists –Software toolkits recently available –Research interfaces to commercial systems emerging Robots –Mostly custom hardware –Software toolkits not yet available –Research interface to daVinci emerging Application control / Workflow –Software toolkits available (Slicer, SIGN, IGSTK, …)

23. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology DICOM Interfaces / Middleware HL7 Input Device(s) Display(s) Electronic Records Anatomical Atlas Patient Database Real-Time Data Distribution Intraoperative Imaging (e.g., X-ray, ultrasound, microscope video) Image Acquisition Image Processing Image Overlay Trajectory Control Servo Control Medical RobotNavigation or Tracking System Visualization/GUI (Navigation System) Target Positions Preoperative Data and Models DICOM System Configuration Procedural Logic (e.g., Task Graph) Data Logging Visualization/GUI Application Controller Challenge #2: Architecture and Interface Standards Preoperative Images Standards needed

24. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Challenge #2: Architecture and Interface Standards Need standards for: –Technical interoperability: delivery of messages/data between subsystems (e.g., networks, middleware) –Semantic interoperability: ability to process messages (e.g., common data structures, commands) Existing efforts: –DICOM for medical images –ISO 11073 (IEEE 1073) –CANOpen –Medical device plug & play (CIMIT) Examples of tracking system interfaces: –SIGN / OpenTracker –IGSTK / Atamai –cisstTracker / cisTracker (JHU) –Orion (Vanderbilt)

25. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Challenge #3: Tools for modeling, simulation, and testing Can we predict system performance before implementation? Can we do automated nightly testing of software that interacts with the real world? –Are simulations good enough?

26. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Challenge #4: System engineering in a research environment Define a System Engineering process that: –Facilitates clinical translation –Enables efficient interactions with industry –Does not delay the pace of research

27. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology How can NCIGT (or ???) help? 1.Support database of IGT hardware and software solutions 2.Promote the development and adoption of standards for IGT 3.Develop realistic modeling and simulation environments for system evaluation and testing 4.Provide guidance documents for performing system engineering