Theoretical Foundations of Multimedia Chapter 3 Virtual Reality Devices Non interactive Slow image update rate Simple image Nonengaging content and presentation No sound Basic Screen display Low resolution image Monoscopic image Small field of view No head tracking No body motion sensing No tactile feedback Highly interactive Fast image update rate Highly complex image Highly engaging content and presentation Three-dimensional sound Head-mounted display High resolution image Stereoscopic image Full field of view Full head tracking Full body motion sensing Full tactile feedback Factors affecting the degree of immersion in virtual reality

Theoretical Foundations of Multimedia Chapter 3 NCSA’s CAVE Virtual Reality Room with stereo glasses and magnetic head/hand tracking Fully immersive using three of four walls to display the graphics Uses an SGI Power Onyx with Reality Engine 2 software

Theoretical Foundations of Multimedia Chapter 3 VR Head-Mounted Display

Theoretical Foundations of Multimedia Chapter 3 VR Head-Mounted Display Limitations: Liquid Crystal Displays (LCDs) Liquid Crystal Displays (LCDs)  pixels not as small as a CRT  pixels not as bright as a CRT  cannot change as quickly as a CRT  short focal distance makes precision, high resolution, and rapid response even more essential Muscle receptor feedback confusion Muscle receptor feedback confusion  light rays of scene indicate “distant”  eye muscles indicate “very close”

Theoretical Foundations of Multimedia Chapter 3 VR Head-Mounted Display Parallax — the apparent change in position of a stationary object when viewed from slightly different positions Parallax — the apparent change in position of a stationary object when viewed from slightly different positions A person’s eyes each see a slightly different view of an object A person’s eyes each see a slightly different view of an object As the brain receives these two images, it interprets the distance to the object in terms of the difference in position of the object in the two images As the brain receives these two images, it interprets the distance to the object in terms of the difference in position of the object in the two images Parallax can be used to fool the brain into “seeing” images as being at various distances (stereoscope and 3D movies of 50’s) Parallax can be used to fool the brain into “seeing” images as being at various distances (stereoscope and 3D movies of 50’s)

Theoretical Foundations of Multimedia Chapter 3 Demonstrating Parallax Pencil Looking at a pencil aligned What is seen using with the corner of a room both eyes What is seen with right eye covered left eye covered

Theoretical Foundations of Multimedia Chapter 3 Parallax Problems with VR Head-Mounted Displays Images may not be perfectly realistic, especially with motion images Images may not be perfectly realistic, especially with motion images When the observer’s head moves and the eyes are refocused, muscle receptor feedback data does not correlate with visual cues When the observer’s head moves and the eyes are refocused, muscle receptor feedback data does not correlate with visual cues The perspective is always that of the camera, never the viewer’s eyes The perspective is always that of the camera, never the viewer’s eyes A viewer motion feedback mechanism is needed to change the perspective A viewer motion feedback mechanism is needed to change the perspective This all contributes to “cybersickness ”: conflicting visual and muscle feedback information This all contributes to “cybersickness ”: conflicting visual and muscle feedback information

Theoretical Foundations of Multimedia Chapter 3 VR Aural Output Refer to the discussion in chapter 2 regarding the perception of sound Refer to the discussion in chapter 2 regarding the perception of sound Two key factors Two key factors  Localization  Identification The brain interprets differences in the signals it receives from the two ears in a manner analogous to binocular vision The brain interprets differences in the signals it receives from the two ears in a manner analogous to binocular vision For multimedia sound to be completely realistic, it requires head-position sensing feedback and enormous computational power — not practical for most multimedia For multimedia sound to be completely realistic, it requires head-position sensing feedback and enormous computational power — not practical for most multimedia

Theoretical Foundations of Multimedia Chapter 3 VR Input Devices The terminology of three-dimensional motion y x z Origin Roll Yaw Pitch

Theoretical Foundations of Multimedia Chapter 3 VR Position Sensing A point in space is defined in terms of distance along three mutually perpendicular axes, usually termed X, Y, and Z A point in space is defined in terms of distance along three mutually perpendicular axes, usually termed X, Y, and Z Motion is defined in terms of changes in position, which requires six parameters Motion is defined in terms of changes in position, which requires six parameters Devices that can sense and record motion are termed six-degrees-of-freedom (6-DOF) devices Devices that can sense and record motion are termed six-degrees-of-freedom (6-DOF) devices

Theoretical Foundations of Multimedia Chapter 3 VR Position Sensing Sensor output from a 6-DOF device can be Sensor output from a 6-DOF device can be  continuous  polled, or sent only upon request Parameters to consider in evaluating a tracking device: Parameters to consider in evaluating a tracking device:  Lag or Latency — the delay between the actual time of the motion and when it is available as input data; should be <50 mSec  Update rate — Rate at which measurements are made; should be as fast as possible  Precision or accuracy of the measurements  Range over which the sensors operate  Rejection of interference

Theoretical Foundations of Multimedia Chapter 3 VR Voice Input Speech Recognition Speech Recognition Complications due to variations in Complications due to variations in  Pitch  Timbre  Volume  Speed of Delivery  Inflection  Accent Natural language processing Natural language processing