1. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic Rendering Max Smolens COMP 259 March 26, 2003

2. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL What is haptics? Using the sense of touch to interact with computers and virtual environments

3. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL What is haptic rendering? The process of computing and generating forces in response to use interactions with virtual objects

4. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Why use haptics? Increases the information flow between the computer and the user Intrinsically bilateral ♦ When we push on an object, it pushes back on us

5. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Why use haptics? (2) Our sensing of forces is closely tied to our visual system and sense of three-dimensional space Information and intent can be conveyed in a physically direct and primal way

6. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic Applications Medicine ♦ Surgical simulators for training ♦ Manipulating robots for minimally invasive surgery ♦ Telemedicine, remote diagnosis ♦ Accessibility for the disabled

7. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic Applications (2) Entertainment ♦ Video games, simulators that enable the user to feel and manipulate objects in the environment Education ♦ Feel phenomena at a variety of spatial and temporal scales ♦ Studying complex data sets

8. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic Applications (3) Industry ♦ CAD systems ♦ Virtual prototyping Assembly and disassembly can guide final design ♦ Shape sculpting Expressive, free-form shape generation and modification

9. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic Applications (4) The arts ♦ Virtual painting, sculpting ♦ Virtual musical instruments

10. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic interaction

11. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Human haptics Mechanical, sensory, motor and cognitive components Two classes of sensory information: ♦ Tactile ♦ Kinesthetic

12. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Human haptics (2) Tactile information ♦ From skin in contact with an object ♦ Spatial and temporal variations of forces within the contact region ♦ Slipping, fine textures, small shapes, and softness

13. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Human haptics (3) Kinesthetic information ♦ Net forces along with position and motion of limbs ♦ Coarse properties of object ♦ Large shapes, spring-like compliances

14. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Human haptics (4) Kinesthetic resolution: ♦ 2 degrees for fingers and wrist ♦ 1 degree for shoulder Force exerted by a finger: ♦ 50 to 100 N maximum ♦ 5-15 N typically during exploration and manipulation

15. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic interfaces

16. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL What makes a good interface? Must work with human abilities and limitations Approximations of real-world haptic interactions determined by limits of human performance

17. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL A good haptic interface Free motion must feel free ♦ Low back-drive inertia and friction ♦ No motion constraints Ergonomics and comfort ♦ Pain, discomfort and fatigue will detract from the experience

18. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL A good haptic interface (2) Suitable range, resolution and bandwidth ♦ User should not be able to go through rigid objects by exceeding force range ♦ No unintended vibrations ♦ Solid objects must feel stiff

19. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic rendering Two parts: collision detection, response

20. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Two types of interactions Point-based haptic interactions ♦ Only end point of device, or haptic interface point (HIP), interacts with virtual object ♦ When moved, collision detection algorithm checks to see if the end point is inside the virtual object ♦ Depth calculated as distance between HIP and closest surface point

21. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Two types of interactions (2) Ray-based haptic interactions ♦ Probe of haptic device modeled as a line- segment whose orientation matters ♦ Can touch multiple objects simultaneously ♦ Torque interactions

22. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Collision detection Detect collisions between haptic probe and virtual objects Bounding volume hierarchies, spatial partitioning H-COLLIDE, hybrid technique: ♦ Partition virtual workspace as uniform grid ♦ For each grid cell containing primitives, computes OBBTrees

23. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Simple collision response Haptic rendering of 3D sphere

24. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Simple collision response (2) Reaction force calculated using the linear spring law F=kx ♦ k: stiffness of object ♦ x: depth of penetration Direction of force along surface normal

25. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Penalty methods Subdivide object and associate each subvolume with a surface Determine feedback force directly from penetration Works well for simple geometric shapes

26. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Penalty methods (2) There are some problems Two possible paths to reach same location, which path was taken?

27. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Penalty methods (3) Force summation for multiple objects Compute net force by adding ♦ Correct for perpendicular surfaces ♦ For obtuse angle, force vector becomes too large ♦ When almost parallel, force vector too large by a factor of 2

28. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Penalty methods (4) Problems with thin objects ♦ If pushed halfway through an object, will be pulled through the rest of the way

29. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Solution? God-object Zilles, Salisbury (1995) Cannot stop HIP from penetrating virtual objects Define additional variables to represent the virtual location of the haptic interface (god-object, IHIP, proxy)

30. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL God-object (2) In free space, HIP and IHIP are collocated When HIP moves into an object, the IHIP remains on the surface IHIP computed such that its distance from the HIP is minimized Correct force vector is unambiguous

31. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL God-object (3) Infinite surface: ♦ Active if the old IHIP is a positive distance from the surface and the HIP is a negative distance from the surface Finite extent: ♦ If a line traced from the old IHIP to new HIP passes through the facet, then consider the facet active

32. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL God-object (4) When touching convex portion of an object, only one surface should be active at a time

33. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL God-object (5) When touching concave portion of an object, multiple surfaces can be active ♦ 2 surfaces: constrain IHIP to a line ♦ 3 surfaces: constrain IHIP to a point IHIP might cross another surface before HIP ♦ Solution: iterate the process, until no new constraints found

34. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL God-object (6) Location computation using Lagrange multipliers ♦ x, y, z: coordinates of IHIP ♦ x p, y p, z p : coordinates of HIP ♦ Constraints added as planes

35. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL God-object (7) Minimize L by setting its six partial derivatives equal to 0, solvable with 65 multiplies and divides

36. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Rendering surface details Smoothing Friction Textures

37. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Force shading Render objects as smooth and continuous, even if underlying representation is not Compute force vector for each vertex, interpolate over polygonal surfaces (like Phong shading)

38. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Surface friction Without friction, virtual objects feel “icy-smooth” Coulomb friction: sticking and sliding Forces tangential to surface, direction opposite of motion

39. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic texturing Force perturbation ♦ Modify the direction and magnitude of the force vector ♦ Max and Becker (1994):

40. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic texturing (2) Image-based: ♦ Construct texture field from 2D image data ♦ Map heights onto the object surface Procedural: ♦ Generate synthetic texture fields using mathematical functions

41. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Haptic texturing (3)

42. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Challenges Graphics update rate must be between 20-30 Hz Haptic update rate must be around 1kHz Decouple simulation and haptic loops using multiple processors or a dedicated machine

43. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL 6-DOF haptics challenges Detect all surface contact instead of just at a single point Calculate a reaction force and torque at every point or region of contact Maintain the 1kHz refresh rate

44. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Examples

45. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL References Basdogan, C., Srinivasan, M.A. “Haptic rendering in virtual environments.” http://network.ku.edu.tr/~cbasdogan/- Papers/VRbookChapter.pdfhttp://network.ku.edu.tr/~cbasdogan/- Papers/VRbookChapter.pdf Chen, E. “Six degree-of-freedom haptic system for desktop virtual prototyping applications.” Proc. First International Workshop on Virtual Reality and Prototyping, p. 97-106, 1999. Gregory, A., Lin, M., Gottschalk, S. and Taylor, R. “A Framework for Fast and Accurate Collision Detection for Haptic Interaction.” Proc. of the IEEE Virtual Reality (VR 99), p. 38-45, 1999. Mark, W. et al. “Adding force feedback to graphics systems: issues and solutions.” Proc. ACM SIGGRAPH 1996. Massie, Thomas H. and Kenneth Salisbury. “The PHANTOM haptic interface: a device for probing virtual objects.” Proc ASME Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 1994. McNeely, W., Puterbaugh K., and Troy, J. “Six degree-of-freedom haptic rendering using voxel sampling.” Proc. ACM SIGGRAPH 1999.

46. The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL References (2) Ruspini, Kolarov and Khatib. “The haptic display of complex graphical environments.” Proc. ACM SIGGRAPH 1997. Salisbury, J.K. et al. “Haptic rendering: programming touch interaction with virtual objects.” Proc. ACM SIGGRAPH 1995. Salisbury, J.K. and Srinivasan, M.A. “Phantom-based haptic interaction with virtual objects.” IEEE Computer Graphics and Applications, 17(5), p. 6-10. Salisbury, J.K. “Making graphics physically tangible.” Communications of the ACM, 42(8), p. 74-81. Srinivasan, M.A. and Basdogan, C. “Haptics in virtual environments: taxonomy, research status, and challenges.” Computers & Graphics, 21(4), p. 393-404. Zilles, C.B. and Salisbury, J.K. “A constraint-based god-object method for haptic display.” Proc. IEE/RSJ International Conference on IntelligentRobots and Systems, Human Robot Interaction, and Cooperative Robots, Vol. 3, p. 146-151, 1995.