Human Factors and Displays for VR Graphics David Johnson

Goals Learn about –Human vision –Methods presenting images to users

Visual Bandwidth Bandwidth of vision is probably greater than other sensory modalities –Retina bandwidth ~10 Mbits/sec article –Most important sense for VR? IMAX movie of rollercoaster can induce sensation of motion Images can induce sensations of taste/smell

Why Study Basis of Vision? If a display can match human capabilities, we are done If the display can’t, nice to know where to concentrate resources. Maybe can use tricks to suggest a higher- quality display –Optical illusionsOptical illusions

Anatomy of Vision

Eye Anatomy

Blind Spot test test Test –Brain fills in

Eye Optics

Accomodation Change in curvature of lens –Rest focus from 6m to infinity

Light Sensors Cones –Mostly in the fovea –Blue, green, red/yellow cones, also called short, medium, long Rods –Sense low levels of light

Visual Acuity Cycles per degree –Like angular resolution –Humans can resolve 0.93mm spacing at 1m –Fovea is about 1 Mpixel 1 arc minute at fovea ( 1/60 degree) –20/20 vision letters are 5 arc minute letters (strokes 1 arc minute) Around 24K x 24K over field of view –Retina is sensitive to light levels over 10^13 range Dynamic

Visual Field One Eye –120 degrees vertically and 150 degrees horizontally 60 to nose, 90 to side 50 up, 70 down Binocular 200 degrees horizontal Eye can rotate about 50 degrees

Binocular Vision Humans have 120 degree binocular overlap –2 40 degree monocular regions Depth disparity perception –0.05mm at 500mm –4mm at 5m

Eye Motion Vergence - the motion of the eye to maintain binocular vision –‘Cross-eyed’ when focus in close Version – eye movement in the same direction Duction – motion of one eye Eye makes compensatory motions when head moving –Shake hand vs. shake head Saccades – eye movement to use fovea more Microsaccades – imperceptible motions to maintain excitation of rods and cones

Frame rate 60Hz frame rate is generally considered important –flicker fusion at ~60Hz Old movies at low rates but people weren’t sensitized to it

Visual Displays How do we get computers to interact with the eye?

Overview display technologies head-mounted displays (HMD) projection-based displays

Overview Characteristics of displays field of view stereo display resolution issues brightness

HMD head-mounted displays (hmd) technology displays: LCDs, CRTs, OLEDs, other totally immersive display experience tend to have small FOV (as compared to a more natural FOV) often bulky

HMD two displays provide image for left and right eye see-through displays used for augmented reality semi-transparent overlays or video see-through What did Brooks say was an advantage of video merge vs optical merge?

HMD characteristics what are important characteristics? image resolution, brightness, contrast tied to visual acuity field of view the mechanics ergonomics, mass, moments of inertia

field of view - FOV field of view - can mean different things optical field of view - field of view as specified by the HMD optics rendered field of view - field of view as specified in software Mapping between the two of them

field of view - FOV field of regard - the total area over which a user can view afford visual integration of larger space for HMDs this is generally 360 degrees provided the head is tracked for projection systems, generally closer to 180 degrees CAVE varies

hmd display optics optics between the image plane and the user’s eye produce a virtual image farther away from the eye reduces accommodative effort ideally out a few meters to help cancel out convergence/accommodative rivalry optics magnify pixel granularity! other distortions? Leep optics – radial falloff – needs defocus eye optics screen virtual imagefocus distance

HMD displays CRT-based HMDs electron beam aimed at phosphorescent screen, resulting in emitted light generally good picture quality but often heavier and much more expensive Now rare

example of CRT-based HMD Datavisor HiRes, Datavisor 80 Datavisor HiRes: 1280x1024, 1.9’/pixel, 42° fov (100% overlap), about 4lbs, uses monochrome CRT w/ color filter shutters Datavisor 80: 1280x1024, 3’/pixel, accommodation at infinity, 80° diagonal fov, 120° with 20% overlap, about 5lbs

nvis nVisor SX liquid crystal on silicon (LCoS) LCD-type device by CRLOpto 0.9” microdisplay with 1280x1024 resolution horizontal fov: 47°, weight: 2lbs good image quality, brightness, and contrast

LCD-based hmds in general, LCD HMDs have been less expensive and lighter however, usually, much lower resolution poorer image quality lower brightness, contrast improvements in LCD technology but VR just now catching up

LCD-based HMDs Virtual Research V8 640x480 resolution approx 45° fov (100% overlap) lightweight: ~2lbs inexpensive: ~$12k

other HMDs Kaiser resolution 1024x768 about $15k uses active matrix LCD screens 2.3’/pixel 40° fov just over 2lbs OLED-based HMDs - better brightness than LCDs, 800x600 good image quality, lighter $5k, but with low fov (28°) may have short life span Kaiser ProView XL50 5DT HMD 800

Sensics First new HMD in some time Array of microdisplays OLEDs Panoramic field of view 2.9’/pixel

setting up stereo monocular: image viewed with one eye only bi-ocular: both eyes see the same image binocular: each eye sees its own image HMDs approximate stereo vision by showing a user left and right eye images what if the person can’t fuse stereo? What about vergence? what about IPD? IPD = inter-pupillary distance is this important? most HMDs do not provide enough control over the exact settings

perception and fov/stereo common explanations for problems in VR: too small fov didn’t use stereo stereo was configured improperly truth of the matter is that no one really knows in general for all situations requires specific experimentation with different setups, tasks, and applications FOV is likely important for speed of localizing stereo is likely very important for near field interaction

other pros/cons of HMDs definite advantages: fully immersive disadvantages: bulky, heavy, obtrusive poor resolution, mismatch between accommodation/convergence other comments?

projection displays setup a projector, aim it at a screen you’re doing VR research! actually a little more complex than that goal is to surround user with the virtual environment good concept potentially reduces locomotion abilities capability to increase resolution dramatically tiling the display projectors

cave CAVE Automatic Virtual Environment developed at U Illinois-Chicago did you know CAVE is trademarked?

stereo with projection vr with HMDs, stereo vision is potentially easier (minus bad artifacts) two images, one for each eye for projection vr, you have two choices active stereo passive stereo

active stereo shutter glasses are required left and right eye shutters on the glasses synchronize with images coming from projector crystal eyes transmitter synced up with graphics/projector system turns right and left shutters on/off

ia state cave

passive stereo use the polarization of light to passively send stereo two projectors per screen, each with different polarizing filters left and right eyes of glasses allow the correct polarized light through potentially less expensive people just wearing “sun glasses”

important stereo issues generally, very expensive! requires extremely bright projection systems projectors must be capable of high refresh rates (120Hz) for active stereo bright projectors are required due to issue with light efficiency first off, about half of light is lost due to left/right switching glasses passive stereo - between 12% and 59% depending on setup Screen transmission

other painful issues with projection vr alignment matters for stereo setups, especially passive stereo, alignment is crucial for tile-able displays with high resolution images must overlap and blend

other painful issues with projection vr actually, should be able to project on any surface large set of research devoted to automatic alignment image blending, color matching projection onto arbitrary surfaces not just flat walls

Desktop VR Also known as “fishtank” VR Use computer monitor –Track head –Stereo from shutter glasses

vision dome and other spherics

Workbench

Autostereoscopic displays

Volumetric displays Actuality –Rotating screen