Video Overlay Advanced Computer Integrated Surgery ( ) Jeff Hsin, Cyrus Moon, Anand Viswanathan Final Presentation

Overview of Video Overlay Video Overlay: The ability to accurately superimpose computer generated images onto related objects in real-time video.

Software Flow Chart Video Software Network Software Tracking Software Graphics Software Tracker Input Camera Input Overlay Image Cyrus Jeff Anand

Final Plan OriginalUpdatedCurrent Tracker code: –testing 3/18 3/18 Networking Code: –testing3/ /13* 5/6* Graphics Code: –diagram frames & transformations 3/11 3/11 –framework 3/25 3/25 –primitive tracking/rendering: 3/25 4/15* 4/15* –tool object creation 3/25 3/25 –tool calibration -- 4/? –video input -- 4/15* 5/10* –camera calibration -- ? 5/6* –working video overlay system 4/7 4/22* 5/10** Blue – Completed * – Changed Date ** – Twice Changed Date

Results/Work Accomplished Functional and tested Optotracker class Working network software (UDP transportation of frame data) Functional and tested vqs class Working graphics framework Integrated Tracking/Video/Graphics software Working pointer/camera calibration software Functional Video Overlay System

Coordinate Graph (Video Overlay) Patient markers Model (from Imaging) World (Tracker) Tool 1Tool k.... Camera Lens Screen *Dynamic Transforms (determined by Optotrak)

Possible Improvements (Remaining Work to be Done) Network Code: –Client/Server Model of OptoTracker class –fix remaining bugs of CIS Libraries (CISNetwork) –Usage of multithreading –or: Robust network code--using CORBA instead Improved Pointer Calibration Program Fully tested/debugged Camera Calibration Code Curve Rendering Function Registration Program More Debugging and Quality Testing Video Distortion Correction

Significance of Work Use of video overlay would make some surgeries more accurate and consistent. –Video overlay could be integrated into a broad range of surgical procedures Code from this project could be used as basic frameworks for future projects in the ERC involving the OptoTrak, Video Input, Networking, and/or GLVU. Code could be modified/appended to work with Image Overlay. Aid in debugging of CIS Libraries has made them more robust, functional

Problems Encountered Figuring out other people’s code: –Lack of good documentation Compiling/Linking problems –includes, libraries, dependencies Code Integration (incompatibilities) –Networking: SPREAD  CORBA  TCP  UDP –CC and g++ Optotracker problems?—Anand needs to remember 

What We have Learned Issues associated with large scale code development –manipulation/use of large amounts of code is difficult; Integrating them is even harder Optotrak API,CIS Libraries, VL libraries, glvu libraries –compilation/compatibility issues CORBA and CIS libraries different compilers do different things (CC and g++ don’t mix) what a makefile is (finally!).dll vs..lib creation

What We Have Learned (cont’d) Always Check on dependencies Organization = efficiency, less wasted time/stress –Know what to do / when to do it –‘If you don’t write it down, it never happened’ One can never have too many backup plans The skeleton in the ERC is real –"You Morons!“ -- Randy G. (to team Video Overlay & co.)

What we would have done differently Better planning/research earlier on in the project Bugged advisors earlier/gotten help as soon as possible Setting of smaller, daily/weekly goals More organization Cyrus punching paper towels 

Special Thanks to : Advisors: -Dr. Taylor-Dr. Cohen ERC people: -Anton Deguet-Dr. Rajesh Kumar-Andy Bzostek -Sadi Yigit-Adam Morris - rest of the ERC Staff for moral support Jeff Lee - Beta Tester for OptoTracker class