1. Spatiotemporal Information Processing No.4 3 components of Virtual Reality-3 Display System Kazuhiko HAMAMOTO Dept. of Information Media Technology, School of Information and Telecommunication Eng., Tokai University, Japan

2. Today’s Contents  Somatic sense Skin sense Deep sense Haptic sense  Display system – skin sense The principle of tactile primary color  Display system – deep sense  Display system – haptic sense

3. The relationship among 3 components (review) Person real spacevirtual space Simulation system Display system Sensing system computer

4. Somatic sense (1/4)  Cutaneous sense (or sense of touch or skin sense) A sense by some sensory receptors which lie beneath the surface of the skin tactile sense, temperature sense, pain sense

5. Somatic sense (2/4)  Deep sense (or proprioceptive sense) A sense of the relative position of neighboring parts of the body, whose receptors are found in muscles, tendons and joints kinetic sense, postural sense

6. Somatic sense (3/4)  How can “iron ball” be felt by human sense ? Shape ?  Skin sense or deep sense Surface pattern (texture) ?  Skin sense or deep sense Recognition of “iron” ?

7. Somatic sense (4/4)  Recognition of shape If we wear gloves…. We can feel -> deep sense  Recognition of pattern If we wear gloves…. We cannot feel -> skin sense  Recognition of “iron” Temperature sense

8. Haptic sense  “Active touch” can improve the precision of the recognition  Haptic sense Active touch sense, which is integrated sense of skin sense and deep sense with action of the body

9. Skin sense and receptors  Tactile sense Merkel disk  displacement Meissner corpuscle  velocity Ruffini corpuscle  displacement  Sense of contact force (pressure) Ruffine corpuscle  displacement Pacinian corpuscle  acceleration

10. Skin sense and receptors http://www.unmc.edu/physiology/Mann/mann5.html

11. Deep sense and receptors  Receptor in muscle Muscle spindle Stretch receptor, which sends muscle’s length information to CNS  Receptor in tendon (joint) Golgi tendon organ Another type of receptor, which provides information about changes in muscle tenstion

12. Muscle spindle http://www.ucl.ac.uk/~sjjgsca/MuscleControl.html CNS : Central nervous system

13. Golgi tendon organ

14. Display system – skin sense  Virtual sandpaper (M.Minsky, MIT, 1995) Recognition of “type of Texture” Fine uneven surface  Vibration of small amplitude, high frequency Rough uneven surface  Vibration of large amplitude, low frequency

15. Virtual Sandpaper Margaret Minsky, MIT, 1995 Experimental setup Screen image http://dspace.mit.edu/handle/1721.1/29094

16. Display system – skin sense  The principle of tactile primary colors A kind of functional electrical stimulation (FES) Each receptor is stimulated electrically and independently 3 receptors can be stimulated independently by the differences of how deep seated or singularity of feature along axone.

17. Prototype using the principle of tactile primary colors  Sense of pressure, low and high freq. vibration can be presented (2005) Prototype of the tactile display http://www.star.t.u-tokyo.ac.jp/projects/tactile-display/index-j.html

18. Display system – deep sense  Equipment type A display device is equipped on hand or fingers. Action of hand or fingers is restricted by the display device. The restriction causes a feeling of “existence of an object”

19. Display system – deep sense Cyber Grasp

20.  SPIDAR  8 wires and 8 motors detect the position and the direction of the ball  User controls the ball in 6DOF  Sense of touch or collision is transformed to tension of the wires http://www.ddd.co.jp/product/spidar/ Display system – deep sense

21. Display system – Haptic sense  Haptic display Present tactile information Stimulus of tactile organ Force feedback system Shape, hardness, softness, smoothness, friction and gravity are perceived in virtual space.

22. Display system – Haptic sense  PHANToM Desktop (SensAble Device, Inc.)  6DOF  Refresh rate  1000 [Hz]  Simple structure  Torque generated by motors is transmitted to active part by wire.

23. Display system – deep sense  Problems Always feel a display’s weight and movement Difficult to create “we don’t touch anything” Response of a motor (actuator)

24.  Deep sense Artificial muscle is used as switching device  Skin sense Uneven feeling can be realized by MR fluid Haptic interface Actuated with Magnetorheological fluid and Artificial muscle HAMA デバイス Novel Development in my Lab.

25. Power supply New haptic device Electric circuit for drive Artificial muscle switch for deep sense MR fluid for sense of cutaneous

26. New device for deep sense with Artificial muscle movable part Lock by BMX Unlock by BMX Restricted fingerReleased finger

27. New device for skin sense with MR fluid When MR fluid into electro magnetic field, the viscosity becomes higher