1. 1 07/11/07 Using an Event-Based Approach to Improve the Multimodal Rendering of 6DOF Virtual Contact Jean Sreng, Florian Bergez, Jérémie Le Garrec, Anatole Lécuyer, Claude Andriot jean.sreng@inria.fr

2. 207/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

3. 307/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

4. 407/11/07 Context Virtual contact between objects –Virtual prototyping –Assembly/Maintenance simulations –Video games –…–…

5. 507/11/07 Context Multimodal rendering of contact –Visual –Audio –Haptic Some limitations –Computational resources –Sensor resolution 500 Hz1 kHz48 kHz60Hz

6. 607/11/07 Context To overcome these limitations : –Sound synthesis algorithms (modal synthesis) [van den doel et. al.] –Event-based haptic [kuchenbecker et. al.] –…–… However : –Often tied to a specific physical engine –Demonstrated in a simple context

7. 707/11/07 We propose… An event-based approach : –Integrate various techniques Visual Audio 6DOF Haptic –Based only on position information Not to be tied to a specific physical engine Can be easily adapted to an existing simulation

8. 807/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

9. 907/11/07 Contact information (states) Continuous, “time independent” states : –Free motion (no contact) –Friction (contact) Free motionFriction

10. 1007/11/07 Contact information (events) Discrete events describing the “evolution” : –Impact –Detachment Free motionFriction ImpactDetachment

11. 1107/11/07 Contact information (states and events) The classic continuous information : –Is essential to graphic and closed-loop haptic rendering –Cannot convey efficiently the transient nature of an impact The different states and events information can be used to improve the rendering of : –Transient phenomena (impact) –Continuous phenomena (friction)

12. 1207/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

13. 1307/11/07 Computation of contact states and events The contact condition : The events are defined by : –Impact : –Detachment : Local linear velocity Normal Friction ImpactDetachment

14. 1407/11/07 Velocity computation To estimate the velocity : Some issues : –Time stepping –Approximation errors

15. 1507/11/07 Computation of contact states and events On the timestep –Impact event : –Friction state : –Detachment event :

16. 1607/11/07 Computation of contact states and events In a nutshell… Friction ImpactDetachment

17. 1707/11/07 Outline Context Contact states and events Computation Multimodal feedback –Visual –Audio –6DOF Haptic Conclusion OUTLINE

18. 1807/11/07 Multimodal feedback The generated states and events can be used with specific rendering techniques Implementation on an existing 6DOF haptic simulation

19. 1907/11/07 Visual feedback Particle effect : –Impact event –Friction state –Detachment event Pencil effect on friction state : –Color : Force –Thickness : Velocity

20. 2007/11/07 Audio feedback Modal synthesis of impact event and friction state :

21. 2107/11/07 Haptic feedback 6DOF event-based haptic feedback of impact event –Traditional closed loop feedback –High frequency force pattern Elementary wrench associated to an impact point

22. 2207/11/07 Haptic feedback

23. 2307/11/07 Conclusion We proposed : –An event-based approach to improve the rendering of contact : Impact Friction Detachment –A computation method based only on object’s positions –A multimodal implementation on a complex virtual prototyping simulation Visual Audio 6DOF Haptic

24. 2407/11/07 Further work A formal user evaluation –Compare with traditional feedback –Complex industrial context Test some other feedback : –Detachment event –6DOF haptic texture

25. 2507/11/07 ? Questions Thank you for listening Questions ?