Visual, auditory, and haptic displays Dr. Xiangyu Wang Acknowledgment of Dr. Doug Bowman’s lecture notes

Introduction to displays Display: device which presents perceptual information Often ‘display’ used to mean ‘visual display’ Goal: display devices which accurately represent perceptions in simulated world (i.e., higher levels of immersion)

Vision Stimulus: light of wavelengths ~ nm

Visual displays for VEs Standard monitor (mono/stereo) Head-mounted/head-referenced Projected (usually stereo) –single-screen –multiple, surrounding screens Retinal display Volumetric displays

Characteristics of visual displays Field of regard (FOR), field of view (FOV) Brightness, contrast ratio Resolution Screen geometry Light transfer Refresh rate Ergonomics

VR with a monitor

Head-mounted displays (HMDs) full surround (FOR=360) simple stereo cumbersome tethering to computer often small FOV single user

Surround-screen displays less obtrusive headgear multi-user better stereo occlusion problem missing sides

Surround-screen displays Video:

Six-sided CAVE (Duke U. DiVE)

Tabletop displays direct manipulation “god’s-eye” view change orientation small FOR

Virtual retinal display (VRD) HIT lab / Microvision image scanned directly onto retina great potential

Volumetric display Pixels displayed in actual 3D space Multi-user correct viewing Size issues Can’t move/reach into display

Volumetric display Video

Which visual display to use? Consider lists of pros and cons Consider depth cues supported Consider level of visual immersion But this is a very hard question to answer empirically Instead of comparing actual displays, compare levels of immersion

Auditory displays Second most studied sense Second most common VE display Stimulus: disturbance of molecules in a medium (air) Perceptions: pitch, loudness, location

Simple VE audio Intensity fall-off (1/d2) Headphones also block out real-world noises Ambient sound (e.g. stream) Present speech instead of text

3D auditory displays Technologies: –Speaker-based –Headphone-based Uses: –Virtual objects emitting sound –Sensory substitution

Auditory displays Video clip (1) shows the actual physical environment in which the virtual reality system was installed, and shows a participant during the course of the exploratory phase of the experiment. Video clip (2) shows the virtual environment during several phases of the experimental procedure. The virtual scene is shown from both the participants subjective view and also the overhead view. The virtual audio scene is also included and is rendered for the participant's position in the scene.

Haptic Displays Touch –Actually a range of skin senses –touch / pressure –hot / cold –pain Useful for object identification & understanding

Haptic displays Exoskeleton Robot arms Phantom Tactile devices Video: haptic Prototyping – m/watch?v=- i9Wm2rTsao

SPIDAR haptic display

Passive haptics Use of props - “poor man’s” haptic Display –ARtoolkit –pen & tablet

Haptic displays Displaying to other skin senses Simple, special-purpose “displays” for temperature, air movement, etc. –Fan –Heat lamp

Olfactory displays Sense of smell not studied extensively - use in VEs? A few academic projects Smell synthesis still in the future

Olfactory displays Video: "Fragra": Entertainment System Utilizing Olfactory Display by Arito MochizukiArito Mochizuki

Vestibular/kinesthetic displays Sense of body, self-motion Virtual body representation Can “display” to these senses by: –Using motion platforms –Stimulating the proper parts of the brain

Conclusions Making VEs multi-sensory is becoming easier becoming cheaper But most VEs still use only visual display We need to study the effects of level of auditory and haptic immersion