1. Human Factors and Displays for VR Graphics David Johnson

2. Goals Learn about –Human vision –Methods presenting images to users

3. Visual Bandwidth Bandwidth of vision is probably greater than other sensory modalities –Retina bandwidth ~10 Mbits/sec article –Most important sense for VR? IMAX movie of rollercoaster can induce sensation of motion Images can induce sensations of taste/smell

4. Why Study Basis of Vision? If a display can match human capabilities, we are done If the display can’t, nice to know where to concentrate resources. Maybe can use tricks to suggest a higher- quality display –Optical illusionsOptical illusions

5. Anatomy of Vision

6. Eye Anatomy

7. Blind Spot test test Test –Brain fills in

8. Eye Optics

9. Accomodation Change in curvature of lens –Rest focus from 6m to infinity

10. Light Sensors Cones –Mostly in the fovea –Blue, green, red/yellow cones, also called short, medium, long Rods –Sense low levels of light

11. Visual Acuity Cycles per degree –Like angular resolution –Humans can resolve 0.93mm spacing at 1m –Fovea is about 1 Mpixel 1 arc minute at fovea ( 1/60 degree) –20/20 vision letters are 5 arc minute letters (strokes 1 arc minute) Around 24K x 24K over field of view –Retina is sensitive to light levels over 10^13 range Dynamic

12. Visual Field One Eye –120 degrees vertically and 150 degrees horizontally 60 to nose, 90 to side 50 up, 70 down Binocular 200 degrees horizontal Eye can rotate about 50 degrees

13. Binocular Vision Humans have 120 degree binocular overlap –2 40 degree monocular regions Depth disparity perception –0.05mm at 500mm –4mm at 5m

14. Eye Motion Vergence - the motion of the eye to maintain binocular vision –‘Cross-eyed’ when focus in close Version – eye movement in the same direction Duction – motion of one eye Eye makes compensatory motions when head moving –Shake hand vs. shake head Saccades – eye movement to use fovea more Microsaccades – imperceptible motions to maintain excitation of rods and cones

15. Frame rate 60Hz frame rate is generally considered important –flicker fusion at ~60Hz Old movies at low rates but people weren’t sensitized to it

16. Visual Displays How do we get computers to interact with the eye?

17. Overview display technologies head-mounted displays (HMD) projection-based displays

18. Overview Characteristics of displays field of view stereo display resolution issues brightness

19. HMD head-mounted displays (hmd) technology displays: LCDs, CRTs, OLEDs, other totally immersive display experience tend to have small FOV (as compared to a more natural FOV) often bulky

20. HMD two displays provide image for left and right eye see-through displays used for augmented reality semi-transparent overlays or video see-through What did Brooks say was an advantage of video merge vs optical merge?

21. HMD characteristics what are important characteristics? image resolution, brightness, contrast tied to visual acuity field of view the mechanics ergonomics, mass, moments of inertia

22. field of view - FOV field of view - can mean different things optical field of view - field of view as specified by the HMD optics rendered field of view - field of view as specified in software Mapping between the two of them

23. field of view - FOV field of regard - the total area over which a user can view afford visual integration of larger space for HMDs this is generally 360 degrees provided the head is tracked for projection systems, generally closer to 180 degrees CAVE varies

24. hmd display optics optics between the image plane and the user’s eye produce a virtual image farther away from the eye reduces accommodative effort ideally out a few meters to help cancel out convergence/accommodative rivalry optics magnify pixel granularity! other distortions? Leep optics – radial falloff – needs defocus eye optics screen virtual imagefocus distance

25. HMD displays CRT-based HMDs electron beam aimed at phosphorescent screen, resulting in emitted light generally good picture quality but often heavier and much more expensive Now rare

26. example of CRT-based HMD Datavisor HiRes, Datavisor 80 Datavisor HiRes: 1280x1024, 1.9’/pixel, 42° fov (100% overlap), about 4lbs, uses monochrome CRT w/ color filter shutters Datavisor 80: 1280x1024, 3’/pixel, accommodation at infinity, 80° diagonal fov, 120° with 20% overlap, about 5lbs

27. nvis nVisor SX liquid crystal on silicon (LCoS) LCD-type device by CRLOpto 0.9” microdisplay with 1280x1024 resolution horizontal fov: 47°, weight: 2lbs good image quality, brightness, and contrast

28. LCD-based hmds in general, LCD HMDs have been less expensive and lighter however, usually, much lower resolution poorer image quality lower brightness, contrast improvements in LCD technology but VR just now catching up

29. LCD-based HMDs Virtual Research V8 640x480 resolution approx 45° fov (100% overlap) lightweight: ~2lbs inexpensive: ~$12k

30. other HMDs Kaiser resolution 1024x768 about $15k uses active matrix LCD screens 2.3’/pixel 40° fov just over 2lbs OLED-based HMDs - better brightness than LCDs, 800x600 good image quality, lighter $5k, but with low fov (28°) may have short life span Kaiser ProView XL50 5DT HMD 800

31. Sensics First new HMD in some time Array of microdisplays OLEDs Panoramic field of view 2.9’/pixel

32. setting up stereo monocular: image viewed with one eye only bi-ocular: both eyes see the same image binocular: each eye sees its own image HMDs approximate stereo vision by showing a user left and right eye images what if the person can’t fuse stereo? What about vergence? what about IPD? IPD = inter-pupillary distance is this important? most HMDs do not provide enough control over the exact settings

33. perception and fov/stereo common explanations for problems in VR: too small fov didn’t use stereo stereo was configured improperly truth of the matter is that no one really knows in general for all situations requires specific experimentation with different setups, tasks, and applications FOV is likely important for speed of localizing stereo is likely very important for near field interaction

34. other pros/cons of HMDs definite advantages: fully immersive disadvantages: bulky, heavy, obtrusive poor resolution, mismatch between accommodation/convergence other comments?

35. projection displays setup a projector, aim it at a screen you’re doing VR research! actually a little more complex than that goal is to surround user with the virtual environment good concept potentially reduces locomotion abilities capability to increase resolution dramatically tiling the display projectors

36. cave CAVE Automatic Virtual Environment developed at U Illinois-Chicago - 1992 did you know CAVE is trademarked?

37. stereo with projection vr with HMDs, stereo vision is potentially easier (minus bad artifacts) two images, one for each eye for projection vr, you have two choices active stereo passive stereo

38. active stereo shutter glasses are required left and right eye shutters on the glasses synchronize with images coming from projector crystal eyes transmitter synced up with graphics/projector system turns right and left shutters on/off

39. ia state cave

40. passive stereo use the polarization of light to passively send stereo two projectors per screen, each with different polarizing filters left and right eyes of glasses allow the correct polarized light through potentially less expensive people just wearing “sun glasses”

41. important stereo issues generally, very expensive! requires extremely bright projection systems projectors must be capable of high refresh rates (120Hz) for active stereo bright projectors are required due to issue with light efficiency first off, about half of light is lost due to left/right switching glasses passive stereo - between 12% and 59% depending on setup Screen transmission

42. other painful issues with projection vr alignment matters for stereo setups, especially passive stereo, alignment is crucial for tile-able displays with high resolution images must overlap and blend

43. other painful issues with projection vr actually, should be able to project on any surface large set of research devoted to automatic alignment image blending, color matching projection onto arbitrary surfaces not just flat walls

44. Desktop VR Also known as “fishtank” VR Use computer monitor –Track head –Stereo from shutter glasses

45. vision dome and other spherics

46. Workbench

47. Autostereoscopic displays

48. Volumetric displays Actuality –Rotating screen