HAPTIC RENDERING OF CUTTING TOOLS INTERACTING WITH 3D VOLUMETRIC MODELS WITH APPLICATION TO DENTAL SURGERY SIMULATION Examination Committee :Prof. Peter Haddawy (Chairman) Examination Committee : Prof. Peter Haddawy (Chairman) Dr. Matthew Dailey, Dr. Manukid Parnichkun Dr. Matthew Dailey, Dr. Manukid Parnichkun Domain Expert : Dr. Siriwan Suebnukarn Presenter :Kugamoorthy Gajananan

Outline Introduction  Related Works  Methodology  Results  Conclusion 

Background(1) Traditional Healthcare Training Methods [E.g. Clinical and Surgical Skills in Dentistry] – Mannequins (Plastic Models) – Live patient Introduction [1]

Background(2) Drawbacks -Traditional Training Methods Introduction Expensive EquipmentsLack of Cases Ethical ConcernsBusy Experts

Background(3) Virtual Reality (VR) Simulators IntroductionOperatingRoom SurgicalEquipments Patients [2] VR based medical simulation provides a highly realistic surgical environment – Natural Interaction is guaranteed

Virtual Reality - 3D Volumetric Model [Tuwien, 2008] [Nationmaster,2008] Introduction

Haptic Device Introduction [4]

Haptic Example Demo Introduction [5]

Benefits of using a simulator No incremental cost No Risk Reusable,Repeatable and Always available Maximum teaching effectiveness(More cases). Gather very precise data about the procedures. – Assesment of skills. – Feedback in terms of teaching. Continuous monitoring of competence possible Introduction

Available Prototypes Related Works (a).PerioSim (b). PerioSim (c). VRDTS(d).IDSS a. c. b. d.

Available Research Studies a [Kim et al.2005] b. [Eriksson et al.2006] c. [Yau et al.2006] d. [Wang et al.2003] Related Works

The Goal and Objectives Real Time Manipulation of 3D Volumetric Data [Cutting/Drilling Simulation] – Volumetric data representation – Different shapes of virtual tools – Force feedback computation algorithms for different shapes of virtual tools – Formal evaluation of the simulator

Demo Methodology

Snap Shot - Capsule Shape Tool Methodology

Snap Shot - Cylinder Shape Tool Methodology

Snap Shot - Cone Shape Tool Methodology

Snap Shot - Sphere Shape Tool Methodology

Graphic Rendering Haptic Rendering Dental Simulator’s Architectural Design Collision Detection Force Computation Methodology Surface Model Simulation 1000 Hz 30 Hz Volume Model Haptic Interface

Main Technical Challenges Real time cutting simulation – Maintaining threads’ refresh rates at required level Compact data structure – Hybrid data structure : Volumetric and Surface Representation of Tooth Model Efficient collision detection algorithm Fast Force feedback computation for different shapes of tool Haptic Hz Graphic - 30 Hz Methodology

Evaluation Methodology Dentists from Faculty of Dentistry in Thammasat University

Evaluation Results Realism of Visual DisplayNMeanStandard deviation Tooth Model Realism Crown Roots Root Canal orifices D Instruments Realism Capsule Cylindrical Conical Spherical N - Number of participants Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7. Results

Evaluation Results Realism of FeelNMeanStandard deviation Tooth Anatomy Different Stiffness Cutting Force Cutting Time D Instruments Realism Capsule Cutting Cylindrical Cutting Conical Cutting Spherical Cutting N - Number of participants Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7. Results

Evaluation Results Usefulness and EasinessNMeanStandard deviation Usefulness Learning Dental Skills Assessing Dental Skills Teaching Tooth Preparation Easiness Comfortable Felt Focussed Friendly User Interface N - Number of participants Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7. Results

Key Research Contributions ( To be published) Cylinder bur - roundCylinder bur Taper cylinder Round bur Force Feedback algorithms for different shapes of tool Conclusion

Key Research Contributions(2) Visual Display (Haptic integration with PolyVox Surface extraction Library) Formal Evaluation Conclusion

Potential Use Thammasat University Dental Students Training tool for Thammasat University Dental Students Phattanapon Rhienmora Intelligent Dental Tutoring - Ongoing PhD research by Phattanapon Rhienmora (Prof.Peter Haddawy’s research group) Conclusion

Future Innovations General Techniques – Tooth Filling – Bone Drilling (E.g Skull ) Conclusion

Thank you for your attention !

References [1] [2] Phattanapon Rhienmora’s proposal [3] [4] [5]

Acknowledgement Thank you Professor Peter Haddawy, Dr.Matthew Dailey, Dr.Manukid Parnichkun, Dr.Siriwan Suebnukarn for your guidence, advices and thouhtful discussions. Thank you Dr.Sumantha Guha for your guidence in Visualization. Thank you Dr.Chris Sewell for supporting my effort. Thank you Dr.David Williams for your great help. Thank you “Pat” for your valuable ideas for the force computation, continuous support, suggestions, critics and discussions.

Possible Publications

Additional Information

Data Acquisition [Rhienmora, 2008] Design and Implementation Tooth Model - Patient’s CT scan data Tooth Resolutions : a. 109 * 84 * 97 (0,100,150,255) b. 103 * 94 * 161 (0 – 255) c. 128 * 128 * 256 (0 – 255)

Data Representation(1) Volumetric Model – Voxels (256 * 256 * 256) – Pad up zeros on the orginal size (128 * 128 * 256) Design and Implementation [Ooeygui, 2008] Block Volume

Data Representation(2) Surface Model – PolyVox devides the volume model into regions – PolyVox reconstruct surface mesh for each region Volume Surface Marching Cube Algorithm [Rhienmora, 2008]

Data Representation(3) Tool Models – Volumetric Sample Points Design and Implementation

Haptic Rendering Collision Detection Design and Implementation

Haptic Rendering Force Computation –Underlying model Spring-damper model F = k.∆x – b.v k  stiffness b  damping constant

Haptic Rendering(1) Force Vector Computation - Force Models for Cylinder,Capsule, Cone Shapes Design and Implementation

Haptic Rendering(2) Force Computation - Magnitude – Magnitude based on voxel density Averaging the density values of the voxels with in the local area that a tool model intersects. Local area refers to the voxels collided by the immersed volume sample points. This implies the fact that stiffness calculated by averaging the stiffness values of the voxels with in the local area. Design and Implementation

Haptic Rendering(3) Force Computation - Magnitude – Nonlinear magnitude computation Based on the number of collided sample points of the tool. Non-linear function of the number of immersed sample points. Resultant Force – The direction of force (unit vector) computed by force response algorithm for different shapes of the tool. – This force vector scaled up by the magnitude calculated. Resultant force smoothed and filtered – Moving Window (size 100 ) Technique Design and Implementation

Technical Challenges Synchronization of Haptic and Graphic threads Stability Impact of number of sample points of tool models Resolution of volumetric data Limitation of the device Design and Implementation

Tools and Libraries OpenHaptics Toolkit -OpenHaptics SDK (HDAPI) PolyVox Technology – Extract Surface Mesh Design and Implementation

Evaluation Objectives – Simulator realism for images of tooth model and instrument – Simulator realism for the feel of the tooth and instruments while cutting – the perceived realism of Tooth cutting – Ratings for simulator usefulness and easiness Evaluation

Conclusions Done Evaluation Done

Key Research Contributions Volumetric Sample point technique used to implement different force models for different shapes of dental tool. Force feedback model weights the contribution of the intersected voxels' density to compute the force feedback's magnitude Force feedback model considers the number of immersed sample points to compute the force feedback's magnitude PolyVox integrated for the first time with haptic – very much improved graphic rendering,Hyprid Data Structure Design of a formal evaluation

Future Work Whole Tool (Handle + Tip) Collision detection. More advanced approaches for material removal. Volume feedback (Physical contact sound). Undo/redo function. Tooth filling. Control the mirror using another haptic device.

Commercial Systems [3] DentSim (DenX USA) [3] Related Works

Force computation Implement force models for different shapes of virtual dental tool Introduction [Petersik et al., 2003] & [Nicolasasenjoosorio, 2008]