1. HAPTIC RENDERING OF CUTTING TOOLS INTERACTING WITH 3D VOLUMETRIC MODELS WITH APPLICATION TO DENTAL SURGERY SIMULATION Examination Committee :Prof. Peter Haddawy (Chairman) Examination Committee : Prof. Peter Haddawy (Chairman) Dr. Matthew Dailey, Dr. Manukid Parnichkun Dr. Matthew Dailey, Dr. Manukid Parnichkun Domain Expert : Dr. Siriwan Suebnukarn Presenter :Kugamoorthy Gajananan

2. Outline Introduction  Related Works  Methodology  Results  Conclusion 

3. Background(1) Traditional Healthcare Training Methods [E.g. Clinical and Surgical Skills in Dentistry] – Mannequins (Plastic Models) – Live patient Introduction [1]

4. Background(2) Drawbacks -Traditional Training Methods Introduction Expensive EquipmentsLack of Cases Ethical ConcernsBusy Experts

5. Background(3) Virtual Reality (VR) Simulators IntroductionOperatingRoom SurgicalEquipments Patients [2] VR based medical simulation provides a highly realistic surgical environment – Natural Interaction is guaranteed

6. Virtual Reality - 3D Volumetric Model [Tuwien, 2008] [Nationmaster,2008] Introduction

7. Haptic Device Introduction [4]

8. Haptic Example Demo Introduction [5]

9. Benefits of using a simulator No incremental cost No Risk Reusable,Repeatable and Always available Maximum teaching effectiveness(More cases). Gather very precise data about the procedures. – Assesment of skills. – Feedback in terms of teaching. Continuous monitoring of competence possible Introduction

10. Available Prototypes Related Works (a).PerioSim (b). PerioSim (c). VRDTS(d).IDSS a. c. b. d.

11. Available Research Studies a [Kim et al.2005] b. [Eriksson et al.2006] c. [Yau et al.2006] d. [Wang et al.2003] Related Works

12. The Goal and Objectives Real Time Manipulation of 3D Volumetric Data [Cutting/Drilling Simulation] – Volumetric data representation – Different shapes of virtual tools – Force feedback computation algorithms for different shapes of virtual tools – Formal evaluation of the simulator

13. Demo Methodology

14. Snap Shot - Capsule Shape Tool Methodology

15. Snap Shot - Cylinder Shape Tool Methodology

16. Snap Shot - Cone Shape Tool Methodology

17. Snap Shot - Sphere Shape Tool Methodology

18. Graphic Rendering Haptic Rendering Dental Simulator’s Architectural Design Collision Detection Force Computation Methodology Surface Model Simulation 1000 Hz 30 Hz Volume Model Haptic Interface

19. Main Technical Challenges Real time cutting simulation – Maintaining threads’ refresh rates at required level Compact data structure – Hybrid data structure : Volumetric and Surface Representation of Tooth Model Efficient collision detection algorithm Fast Force feedback computation for different shapes of tool Haptic - 1000 Hz Graphic - 30 Hz Methodology

20. Evaluation Methodology Dentists from Faculty of Dentistry in Thammasat University

21. Evaluation Results Realism of Visual DisplayNMeanStandard deviation Tooth Model Realism Crown Roots Root Canal orifices 10 4.80 5.20 1.32 0.92 0.84 3D Instruments Realism Capsule Cylindrical Conical Spherical 10 5.30 5.20 5.10 0.82 0.63 0.88 0.74 N - Number of participants Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7. Results

22. Evaluation Results Realism of FeelNMeanStandard deviation Tooth Anatomy Different Stiffness Cutting Force Cutting Time 10 4.40 4.10 5.30 1.32 0.92 0.84 3D Instruments Realism Capsule Cutting Cylindrical Cutting Conical Cutting Spherical Cutting 10 4.80 4.70 1.14 1.03 1.06 N - Number of participants Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7. Results

23. Evaluation Results Usefulness and EasinessNMeanStandard deviation Usefulness Learning Dental Skills Assessing Dental Skills Teaching Tooth Preparation Easiness Comfortable Felt Focussed Friendly User Interface 10 5.60 4.90 5.60 4.40 4.60 4.90 0.72 1.28 0.52 1.07 0.84 0.99 N - Number of participants Very poor - 1, Poor - 2, Fair - 3, Good - 4, Very good - 5, Excellent - 6, Exceptional - 7. Results

24. Key Research Contributions ( To be published) Cylinder bur - roundCylinder bur Taper cylinder Round bur Force Feedback algorithms for different shapes of tool Conclusion

25. Key Research Contributions(2) Visual Display (Haptic integration with PolyVox Surface extraction Library) Formal Evaluation Conclusion

26. Potential Use Thammasat University Dental Students Training tool for Thammasat University Dental Students Phattanapon Rhienmora Intelligent Dental Tutoring - Ongoing PhD research by Phattanapon Rhienmora (Prof.Peter Haddawy’s research group) Conclusion

27. Future Innovations General Techniques – Tooth Filling – Bone Drilling (E.g Skull ) Conclusion

28. Thank you for your attention !

29. References [1] http://www.images.google.com [2] Phattanapon Rhienmora’s proposal [3] http://www.denx.com/ [4] http://www.sensable.com/ [5] http://www.reachin.se/

30. Acknowledgement Thank you Professor Peter Haddawy, Dr.Matthew Dailey, Dr.Manukid Parnichkun, Dr.Siriwan Suebnukarn for your guidence, advices and thouhtful discussions. Thank you Dr.Sumantha Guha for your guidence in Visualization. Thank you Dr.Chris Sewell for supporting my effort. Thank you Dr.David Williams for your great help. Thank you “Pat” for your valuable ideas for the force computation, continuous support, suggestions, critics and discussions.

31. Possible Publications

32. Additional Information

33. Data Acquisition [Rhienmora, 2008] Design and Implementation Tooth Model - Patient’s CT scan data Tooth Resolutions : a. 109 * 84 * 97 (0,100,150,255) b. 103 * 94 * 161 (0 – 255) c. 128 * 128 * 256 (0 – 255)

34. Data Representation(1) Volumetric Model – Voxels (256 * 256 * 256) – Pad up zeros on the orginal size (128 * 128 * 256) Design and Implementation [Ooeygui, 2008] Block Volume

35. Data Representation(2) Surface Model – PolyVox devides the volume model into regions – PolyVox reconstruct surface mesh for each region Volume Surface Marching Cube Algorithm [Rhienmora, 2008]

36. Data Representation(3) Tool Models – Volumetric Sample Points Design and Implementation

37. Haptic Rendering Collision Detection Design and Implementation

38. Haptic Rendering Force Computation –Underlying model Spring-damper model F = k.∆x – b.v k  stiffness b  damping constant

39. Haptic Rendering(1) Force Vector Computation - Force Models for Cylinder,Capsule, Cone Shapes Design and Implementation

40. Haptic Rendering(2) Force Computation - Magnitude – Magnitude based on voxel density Averaging the density values of the voxels with in the local area that a tool model intersects. Local area refers to the voxels collided by the immersed volume sample points. This implies the fact that stiffness calculated by averaging the stiffness values of the voxels with in the local area. Design and Implementation

41. Haptic Rendering(3) Force Computation - Magnitude – Nonlinear magnitude computation Based on the number of collided sample points of the tool. Non-linear function of the number of immersed sample points. Resultant Force – The direction of force (unit vector) computed by force response algorithm for different shapes of the tool. – This force vector scaled up by the magnitude calculated. Resultant force smoothed and filtered – Moving Window (size 100 ) Technique Design and Implementation

42. Technical Challenges Synchronization of Haptic and Graphic threads Stability Impact of number of sample points of tool models Resolution of volumetric data Limitation of the device Design and Implementation

43. Tools and Libraries OpenHaptics Toolkit -OpenHaptics SDK (HDAPI) PolyVox Technology – Extract Surface Mesh Design and Implementation

44. Evaluation Objectives – Simulator realism for images of tooth model and instrument – Simulator realism for the feel of the tooth and instruments while cutting – the perceived realism of Tooth cutting – Ratings for simulator usefulness and easiness Evaluation

45. Conclusions Done Evaluation Done

46. Key Research Contributions Volumetric Sample point technique used to implement different force models for different shapes of dental tool. Force feedback model weights the contribution of the intersected voxels' density to compute the force feedback's magnitude Force feedback model considers the number of immersed sample points to compute the force feedback's magnitude PolyVox integrated for the first time with haptic – very much improved graphic rendering,Hyprid Data Structure Design of a formal evaluation

47. Future Work Whole Tool (Handle + Tip) Collision detection. More advanced approaches for material removal. Volume feedback (Physical contact sound). Undo/redo function. Tooth filling. Control the mirror using another haptic device.

49. Commercial Systems [3] DentSim (DenX USA) [3] Related Works

50. Force computation Implement force models for different shapes of virtual dental tool Introduction [Petersik et al., 2003] & [Nicolasasenjoosorio, 2008]