1. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Development of Open Source Software for C omputer- A ssisted I ntervention Systems Peter Kazanzides, Anton Deguet, Ankur Kapoor, Ofri Sadowsky, Andy LaMora, Russell Taylor Johns Hopkins University www.cisst.org October 30, 2005

2. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Outline Background and Motivation Where are we going? CISST Libraries –Foundation libraries –Real time support Development process and tools

3. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology Multi-institution, multi-disciplinary center –Johns Hopkins University + Medical Institutions –MIT + Brigham & Women’s Hospital –CMU + UPMC –Others: Morgan State, Georgetown, Harvard, Penn, Columbia Funding –Year 7: Core NSF Grant = $3.9M; Total = ~$8.3M –Years 1-10: Core NSF Grant = $30M; Total = ~$55M University researchers, clinicians, industry Research, Systems, Education, Outreach

4. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Historical Background ERC-developed software was captured in: –CIS library Common tools, such as logging Vectors, matrices, transformations Interface to tracking systems Numerical methods, registration, … –MRC library Common interface to different robots Essentially a “wrapper” around API for hardware that provides low-level control

5. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Robot Controller Architecture Supervisory/Trajectory Control (~100 Hz) Application (non-real-time) Hardware Read Sensors Compute Joint Goals Compute Goal on Trajectory Interpolate Setpoint Compute Control Read Sensors Servo Control (~1000 Hz) Application API MRC cisstMRC

6. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Motivation for CISST Package Improve process, design, testing, portability, maintainability for open source release and to facilitate clinical use: –Programming standards –Design reviews –Portable build tools –Automated testing –User documentation Enable the development of real-time software for physical devices such as robots and trackers

7. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Motivation for Real Time Support Motivated by transition from motion controller boards (with processor and vendor’s software) to I/O boards (no processor) and research software Motion Controller Boards (intelligent hardware) I/O Boards (non-intelligent)

8. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Example: Teleoperation of Snake Robot LoPoMoCo Control PC (RTAI/Linux) I/O and Amps LoPoMoCo Control PC (RTAI/Linux) Master Robots Slave Robots

9. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Example: Image-Guided Robot for Rodent Research PC (Windows) Servo Control and Amps (Galil) DMC-2143 Robot 3D Slicer

10. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Where are we going? C++ Software Libraries cisst libraries other libraries Frameworks Based on system complexity Component of larger system Binary components hardware interfaces research algorithms static linking frozen spots hot spots Distributed Interface dynamic loading

11. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology CISST Libraries (Mostly) Stable –Open source* Sept 2005 cisstInteractive ~Nov 2005 Beta version –Open source ~Jan 2006 Work in process –cisstTracker ~Jan 2006 –cisstMRC ~June 2006 Foundation libraries cisstCommon cisstVector cisstNumerical cisstInteractive Real Time Support cisstOSAbstraction cisstDeviceInterface cisstRealtime Interventional Devices cisstTracker cisstMRC … * www.cisst.org, current license based on Slicer 2.x, goal is Slicer 3.0

12. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology cisstVector Vectors, Matrices and Transformations Extensive use of C++ templates (metaprogramming) Fixed size and dynamic –Fixed size especially suited for real-time code Operations on slices and sub-regions References to vectors and matrices –Improves interoperability with other libraries Matrices in row-major or column-major format C++ wrapping of NetLib (numerical methods, including CLAPACK, MINPACK)

13. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Fixed Size Vectors and Matrices Templated by: –Element type (int, double, etc.) –Dimension (number of rows, columns, etc.) Efficiency considerations (for templated dimension): –Loop is easy, but not efficient for small vectors: int Sum() { sum = 0 for (i=0; i < _size; i++) sum += data[i]; return sum; } –Recursive function also not efficient: int RecursiveSum(int size){ return (size == 1) ? data[0] : RecursiveSum(size-1) + data[size-1]; } int Sum() { return RecursiveSum(_size); } –Recursive template (template metaprogramming) is efficient: Compiler “unwraps” recursive template into straight-line code

14. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Fixed Size Vectors and Matrices Operations provided by “recursive engines”: –Classify operations by: Number of operands Type of operands Type of result Storage location for result –Small number of engines handle all operations Example: same engine for addition, subtraction, …

15. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Python Interface Automatic wrapping of C++ for Python (Swig) Object registry to share objects between languages Can load cisst libraries into Python shell Can start Python shell from C++ program GUI features provided by cisstInteractive (using wxWidgets)

16. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Device Interface Device Hierarchies : Tracker Tool GetPosition Robot Tool GetPosition MoveToPosition Should be able to use robot in place of a tracker Can lead to complexity!

17. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Device Interface Our solution: query with string to obtain command object (Command Pattern) Tracker Tool Provides Robot Tool Provides ddiDeviceInterface Configure Provides GetMethodByName … Initialize (e.g., from XML) Return list of implemented operations (strings) Return command object for specified operation (string) Abstract Base Class { “GetPosition” } { “GetPosition”, “MoveToPosition” }

18. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Device Interface · · · dev = ptr to device dev  Configure(…); cmd = dev  GetMethodByName(“GetPosition”); · · · public: void Startup(); void Run(); }; · · · cmd  Execute(data); · · · class myTask : public rtsTask { private: ddiDeviceInterface* dev; ddiCommand* cmd; Non-real-time Real-time

19. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Real Time Support 1.Devices and Tasks should be interchangeable: –Example: Servo control via an intelligent device or via a software task –Solution: Task class derived from device class, but also includes a device ddiDeviceInterface rtsTask

20. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Real Time Support 2.Maintain time history of important state data –State Data Table (SDT), indexed by time and data id 3.Task communication with Command Pattern: –Read from Task SDT or Device –Write to Task Mailbox or Device –Command object can handle remote communications High-level task (low frequency) Low-level task (high frequency) Device Interface mailbox SDT mailbox SDT Hardware Command objects

21. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology cisstRealTime

22. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Development Tools CTest Build Environment LaTeX Doxygen SWIG Compile Library Binaries (static & dynamic, e.g., cisstVector, cisstCommon) Test Programs Applications Optional Interpreter (IRE) Wrapped Source Formatted Documentation (e.g., pdf, html) (e.g., VC++, gcc/make) Test Results Link CVSTrac (bug/feature requests) CMake CVS Repository (source control) CMake Build Instructions Documentation Libraries Test App Applications Scripts CppUnit Dart2 (dashboard) PyUnit Link

23. Copyright © CISST ERC, 2005 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology Development Summary Tools adequately manage implementation, (unit) testing and maintenance phases of development –Automated testing will be hard for physical devices Much documentation is manually created (not enforced by the process) –Requirements, risk analysis, (high-level) design, validation –User guide, tutorial, quickstart We needed Dart sooner than we expected! –gcc is getting pickier! –Windows static/shared libraries (dll export)