© CISST ERC, 2012 An Improved GUI and Vision- based Guidance for the Robo-ELF Team Members: Jon Kriss Collaborators: Renata Smith Mentors: Kevin Olds, Dr. Russ Taylor, Dr. Jeremy Richmon

© CISST ERC, 2012 Background Approximately 25,000 annual cases of throat cancer in the United State, resulting in approximately 6000 deaths Surgical treatments are commonly used because of the risks to sensitive tissue caused by radiation and chemotherapy Endoscopic intra-airway surgery is much less invasive BackgroundGoalApproachConclusion

© CISST ERC, 2012 Background Robotic EndoLaryngeal Flexible Scope (Robo- ELF) system for endoscope manipulation 2011 CIS II project to build and test robot Cadaver experiments in fall 2011 (Olds, et al.) BackgroundGoalApproachConclusion

© CISST ERC, 2012 Current System TaskManager RobotTask Robot Control Safety Features Galil Controller cisst wrapper GUI PC Galil Heartbeat Digital/Analog I/O Robot Motors/Encoders Potentiometers Limit Switches Joysticks EStop Endoscope HD Video Video Capture Card Video Display/ Recording BackgroundGoalApproachConclusion

© CISST ERC, 2012 Current System BackgroundGoalApproachConclusion

© CISST ERC, 2012 Motivation Current Robo-ELF system provides a great improvement over standard laryngeal surgery techniques Drawbacks of current system – Non-intuitive control mechanism – Lack of a useful intra-operative GUI Ultimate goal of project is clinical human trials BackgroundGoalApproachConclusion

© CISST ERC, 2012 Goal 1)Complete regulatory requirements for clinical trials with current system – Software documentation and review – Risk analysis and validation 2)Develop a robust GUI including vision-based guidance – Provide useful tools to the surgeon – Provide an intuitive control method BackgroundGoalApproachConclusion

© CISST ERC, 2012 Requirements Complete system documentation including user manual, risk analysis and validation Complete system updates Integrate computer vision code into robot control code Design and build a GUI with recommendations from surgeons BackgroundGoalApproachConclusion

© CISST ERC, 2012 Current Progress Full system documentation is partially complete already Software design review underway System updates to fulfill requirements in progress Computer vision code for vision-based control already written BackgroundGoalApproachConclusion

© CISST ERC, 2012 Approach Work Required for FDA submission – Documentation and testing of safety features – Software design review – Risk assessment and validation, FMEA – User manual Physical setup instructions Cleaning procedures Computer setup/calibration instructions Explanation of error codes BackgroundGoalApproachConclusion

© CISST ERC, 2012 Approach Desired features of GUI – Point-and-click navigation – Show visually current position of robot – Display current state of important inputs Joint limits Error status, Warnings – Interaction with preoperative data/images BackgroundGoalApproachConclusion

© CISST ERC, 2012 Proposed System BackgroundGoalApproachConclusion TaskManager RobotTask Robot Control Safety Features Galil Controller cisst wrapper GUI Robot state/error display PC Galil Heartbeat Digital/Analog I/O Robot Motors/Encoders Potentiometers Limit Switches EStop Endoscope HD Video Video Capture Card Video Recording/Display Vision-based Control Joysticks

© CISST ERC, 2012 Deliverables Minimum – Submit current system plan for FDA approval Expected – Point-and-click visual navigation – Robust intra-operative GUI Maximum – Port visual navigation to iPad BackgroundGoalApproachConclusion

© CISST ERC, 2012 Dependencies DependencyPlan to ResolveResolve by Draping procedureRenata is working on itMarch 5 Mechanical changesRenata is working on itMarch 12 Software walkthroughSchedule with lab personnelMarch 26 User manualRenata is working on itApril 2 Design review and FMEASchedule meeting with lab personnel and Dr. Richmon April 2 Surgeon’s desired features Meet with Dr. RichmonMarch 5 BackgroundGoalApproachConclusion

© CISST ERC, 2012 BackgroundGoalApproachConclusion Timeline Deliverables 24-Feb2-Mar9-Mar16-Mar23-Mar2-Apr6-Apr13-Apr20-Apr27-Apr4-May10-May Implement and test safety features Improve Error Logging and Handling Software Documentation Software Risk Analysis Software Walkthrough and Review Software for FDA approval finished FMEA Ready for full design review User Manual Vision-based navigation Vision- based nav finished svlFilter for vision code Integrate vision into control code New intra-operative GUI New GUI finished Design for new GUI Visual representation of robot pose Full display of robot status Final Report Course Complete In progress Complete Major Milestone

© CISST ERC, 2012 Milestones Finish software requirements for FDA – March 16 Complete Design Review for Robo-ELF – April 2 Vision-based Navigation – April 13 New GUI – May 4 BackgroundGoalApproachConclusion

© CISST ERC, 2012 Management Plan Reassess deliverables and timeline after each milestone is complete Meeting schedule – Weekly meetings with Kevin Olds, Renata Smith – Meet with Dr. Taylor at least every other week – Monthly meetings with Dr. Richmon BackgroundGoalApproachConclusion

© CISST ERC, 2012 References Olds, K., Hillel, A. T., Cha, E., Curry, M., Akst, L. M., Taylor, R. H. and Richmon, J. D. (2011), Robotic endolaryngeal flexible (Robo-ELF) scope: A preclinical feasibility study. The Laryngoscope, 121: 2371–2374 Reilink, R.; Stramigioli, S.; Misra, S.;, "Image-based flexible endoscope steering," Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on, vol., no., pp , Oct McLeod, I.K., Mair, E.A., Melder, P.C. Potential applications of the da Vinci minimally invasive surgical robotic system in otolaryngology. Ear Nose Throat J 84, , Buckingham, R.O.; Graham, A.C.;, "Computer controlled redundant endoscopy," Systems, Man, and Cybernetics, 1996., IEEE International Conference on, vol.3, no., pp vol.3, Oct 1996

© CISST ERC, 2012 Questions?