Focus Cues Kurt Akeley CS248 Lecture 20 6 December

CS248 Lecture 20Kurt Akeley, Fall 2007 Depth cues Binocular geometric cues: n Stereopsis (retinal-image disparity) n Convergence of the lines of sight Retinal-image disparityVergence Correct retinal-image disparity allows the viewer to “fuse” the scene

CS248 Lecture 20Kurt Akeley, Fall 2007 Depth cues Binocular geometric cues: n Stereopsis (retinal-image disparity) n Convergence of the lines of sight Monocular geometric cues: n Motion parallax n Object size (relative and changing) n Perspective convergence

CS248 Lecture 20Kurt Akeley, Fall 2007 Depth cues Binocular geometric cues: n Stereopsis (retinal-image disparity) n Convergence of the lines of sight Monocular geometric cues: n Motion parallax n Object size (relative and changing) n Perspective convergence Color cues: n Occlusion n Lighting and shading n Atmospheric effects (attenuation, blue shift) n Texture gradient

CS248 Lecture 20Kurt Akeley, Fall 2007 Focus matters too! This photo is of a real scene, not of a model

CS248 Lecture 20Kurt Akeley, Fall 2007 Outline Focus Focus cues Fixed-viewpoint volumetric display Experimental results Practical implications

CS248 Lecture 20Kurt Akeley, Fall 2007 Focus

CS248 Lecture 20Kurt Akeley, Fall 2007 Focus Focus defines a 1-to-1 correspondence between n Object points, which (may) radiate light, and n Image points, where the radiated light converges In practice image points capture only the object-point radiation that passes through an aperture. Aperture Object points Image points

CS248 Lecture 20Kurt Akeley, Fall 2007 A lens provides the magic Index of refraction is greater than one

CS248 Lecture 20Kurt Akeley, Fall 2007 The correspondence soso sisi f d Straight line through the center of the lens Object point Image point Thin lens equation

CS248 Lecture 20Kurt Akeley, Fall 2007 Diopters (D)

CS248 Lecture 20Kurt Akeley, Fall 2007 Diopter distances 0.67 m 1.0 m Diopter number-line assignments are relative to a reference point 2 D Diopter distances are differences between number- line assignments: 2 D = 3 D – 1 D Not computed as the reciprocal of the 0.67 m “distance” ! 0.5 m0.33 m 2 D1 D3 D

CS248 Lecture 20Kurt Akeley, Fall 2007 Thin-lens equation (using Diopters) soso sisi f Object point Image point Reference point is the center of the lens

CS248 Lecture 20Kurt Akeley, Fall 2007 Out-of-focus blur soso sisi f toto titi Image plane Object point rara rbrb E

CS248 Lecture 20Kurt Akeley, Fall 2007 Depth of field (DOF) (sometimes also called depth of focus) DOF is the amount of focus error that is inconsequential Recall that Therefore: DOF measured in Diopters is (almost) invariant with respect to focus distance ( S o ) For a given aperture radius ( r a ) and an acceptable blur radius ( r b ) n “Almost” because there is a slight dependence we are ignoring n DOF is inversely proportional to aperture radius n Once the acceptable blur radius is determined

CS248 Lecture 20Kurt Akeley, Fall 2007 Focus Cues

CS248 Lecture 20Kurt Akeley, Fall 2007 Focus cues There are two focus cues: n Accommodation (the focus response of the eye) n Retinal-image blur Neither is a quality of the light field Instead they are conditions in the human body that are stimulated by the light field

CS248 Lecture 20Kurt Akeley, Fall 2007 Accommodation Accommodation is the focus response of the eye: Resting focus is at infinity (0 D), or is corrected with fixed lenses to infinity. The ciliary muscles contract, allowing the lens to become more spherical. This increases the power of the lens, reducing the focal distance.

CS248 Lecture 20Kurt Akeley, Fall 2007 Human accommodation range 0 ∞ 1 1 2Diopters meters 348 All but one Diopter of focal range is within arm’s reach! 12 Children Young adults Me …

CS248 Lecture 20Kurt Akeley, Fall 2007 Human depth of field Human depth of field is approximately +/- 0.3 D n The optics of the eye are not perfect This corresponds to n A DOF from 2 m to infinity, or n A DOF from 10” to 12” So near-field scenes (with differing depths) are blurry, while far- field scenes are not 0 ∞ 1 1 2Diopters meters …

CS248 Lecture 20Kurt Akeley, Fall 2007 Tilt-shift miniaturization

CS248 Lecture 20Kurt Akeley, Fall 2007 Why “tilt-shift”? Recall that focus is a correspondence Before Photoshop the effect was created by tilting the image plane of the camera off the main axis:

CS248 Lecture 20Kurt Akeley, Fall 2007 Make a model appear real

CS248 Lecture 20Kurt Akeley, Fall 2007 Make a real scene appear miniaturized

CS248 Lecture 20Kurt Akeley, Fall 2007 A stereo display gets all these right … Binocular geometric cues: n Stereopsis (retinal-image disparity) n Convergence of the lines of sight Monocular geometric cues: n Motion parallax n Object size (relative and changing) n Perspective convergence Color cues: n Occlusion n Lighting and shading n Atmospheric effects (attenuation, blue shift) n Texture gradient

CS248 Lecture 20Kurt Akeley, Fall 2007 But the focus cues are all wrong No retinal-image blur cues Incorrect accommodation cue Vergence and accommodation are decoupled

CS248 Lecture 20Kurt Akeley, Fall 2007 Volumetric displays fix the focus cues … And they are autostereoscopic: n Require no tracking of the viewer’s position or orientation n Support multiple simultaneous viewers n Stereopsis is “free” References: Downing et al Favalora et al Lightspace Tech. 2003

CS248 Lecture 20Kurt Akeley, Fall 2007 But they fail in other critical ways Binocular geometric cues: n Stereopsis (retinal-image disparity) n Convergence of the lines of sight Monocular geometric cues: n Motion parallax n Object size (relative and changing) n Perspective convergence Color cues: n Occlusion n Lighting and shading n Atmospheric effects (attenuation, blue shift) n Texture gradient No view-dependent shading is possible, because viewer position is not known

CS248 Lecture 20Kurt Akeley, Fall 2007 Fixed-viewpoint Volumetric Display

CS248 Lecture 20Kurt Akeley, Fall 2007 Fixed-viewpoint volumetric display Fixed-viewpoint: n All geometric and color depth cues are correct Volumetric: n All focus cues are near-correct n No need for gaze tracking What’s the catch? n Display is head-mounted n Must track viewer position and orientation n Latency is a challenge n Must overcome ergonomic issues

CS248 Lecture 20Kurt Akeley, Fall 2007 Required volumetric resolution Autostereoscopic volumetic displays have huge pixel-count requirements in all three dimensions Fixing the viewpoint allows spatial and depth resolutions to be optimized independently: n Spatial pixel density requirements are unchanged n Foveal limit requires 2 pixels/arc min n But depth pixel density is determined by depth of field n +/- 0.3 D is more than satisfied by two pixels per diopter n A display with 4 D range has a depth pixel-count of 7 ! 1000s 7

CS248 Lecture 20Kurt Akeley, Fall 2007 Prototype display design

CS248 Lecture 20Kurt Akeley, Fall 2007 Prototype display Bite bar

CS248 Lecture 20Kurt Akeley, Fall 2007 Demo

CS248 Lecture 20Kurt Akeley, Fall 2007 Depth blending

CS248 Lecture 20Kurt Akeley, Fall 2007 Retinal image of a sine wave grating Eye image from Lower contrast

CS248 Lecture 20Kurt Akeley, Fall 2007 Modulation transfer function

CS248 Lecture 20Kurt Akeley, Fall 2007 Retinal-image contrast with summed images

CS248 Lecture 20Kurt Akeley, Fall 2007 Experimental Results

CS248 Lecture 20Kurt Akeley, Fall 2007 Work done at UC Berkeley Marty Banks Simon Watt Ahna R. Girshick David M. Hoffman …

CS248 Lecture 20Kurt Akeley, Fall 2007 Stimuli

CS248 Lecture 20Kurt Akeley, Fall 2007 Forced-choice This ?Or this ?

CS248 Lecture 20Kurt Akeley, Fall 2007 Experimental design

CS248 Lecture 20Kurt Akeley, Fall 2007 Experimental results

CS248 Lecture 20Kurt Akeley, Fall 2007 Results summary Correct focus distance results in n Shorter time to “fuse” a depth-corrugation stereogram n Ability to fuse finer depth corrugations n Better estimations of depth n People consistently underestimate depth in VR environments n Less fatigue n No forced decoupling of vergence and accommodation There is no performance penalty for depth blending For details refer to the publications:

CS248 Lecture 20Kurt Akeley, Fall 2007 Practical implications

CS248 Lecture 20Kurt Akeley, Fall 2007 Without a fixed-viewpoint volumetric display Use long viewing distances when possible n Flight simulators use either n Large done display n Collimated (infinite focus distance) optics Minimize accommodation/vergence conflict n High-quality stereo headsets have adjustable focal distance n Set it to the best average distance Is there hope of a practical fixed-viewpoint volumetric display?

CS248 Lecture 20Kurt Akeley, Fall 2007 Fixed optics f

CS248 Lecture 20Kurt Akeley, Fall 2007 Adaptive optics Images from

CS248 Lecture 20Kurt Akeley, Fall 2007 Summary Focus cues are n Accommodation (focus response) n Retinal blur Correct focus cues matter n Tilt-shift miniaturization n Experimental results Display type Correct geometric depth cues Correct shading depth cues Correct focus depth cues Stereo   Autostereoscopic volumetric    Fixed-viewpoint volumetric 

CS248 Lecture 20Kurt Akeley, Fall 2007 Project 3 and game competition Source code and write-ups due tomorrow at 5 pm Congratulations to the game competition winners: 1 st Place Balloo Belinda Gu, Edward Luong, and Joel Galenson 2 nd Place Shootout Vincent Gire and David Lissmyr

CS248 Lecture 20Kurt Akeley, Fall 2007 Final exam Location: n Gates B01 Time: n Thursday 13 December, 7 pm to 9 pm Material: n We’ll test your mastery of the content of lectures n But we’ll assume you understand and can apply the material of lectures 1-10 too Review session: n Gates B03 n Friday 7 December (tomorrow), 4:15 pm to 5 pm

CS248 Lecture 20Kurt Akeley, Fall 2007 Office hours Kurt: n Not available today after class n Next Tuesday 1:30 pm to 3 pm n Not available next Thursday (the day of the final exam) CAs: n Check the web site All: n Will monitor and respond to questions

CS248 Lecture 20Kurt Akeley, Fall 2007 Good-bye Thanks to the CAs ! n Andrew, David, and Justin My goals: n Learn a lot (accomplished) n Convey my understanding to you Final requests: n Please share your thoughts with me about how the course or slides can be improved n In person or n Please complete the on-line Axess course evaluation n The system is open now n It closes at 11:59 pm on Sunday 16 December

CS248 Lecture 20Kurt Akeley, Fall 2007 End