User Issues in 3D TV & Cinema Martin S. Banks Vision Science Program UC Berkeley.

Slides:



Advertisements
Similar presentations
Seeing 3D from 2D Images. How to make a 2D image appear as 3D! ► Output and input is typically 2D Images ► Yet we want to show a 3D world! ► How can we.
Advertisements

Cue Reliabilities and Cue Combinations Robert Jacobs Department of Brain and Cognitive Sciences University of Rochester.
A Simulator Sickness Literature Review Michael A. Mollenhauer 12/19/2003.
Chapter 10: Perceiving Depth and Size
Depth Cues Pictorial Depth Cues: aspects of 2D images that imply depth
Imaging Science FundamentalsChester F. Carlson Center for Imaging Science The Human Visual System Part 2: Perception.
Extra Credit for Participating in Experiments Go to to sign upwww.tatalab.ca We are recruiting for a variety of experiments including basic.
Cosc 6326/Psych6750X Vision and Visual Displays. Why do we have two eyes? Binocular vision has several advantages including –increased field of view –redundancy.
Fundraiser Concert for Haiti U of L Students and Faculty Performing –9pm Blues Sen-sa-shun –10pm Shawna Romolliwa –11pm Dave Renter At The Slice, Friday.
Bridget C. Hendricks, James Comerford, Frank Thorn, and Eli Peli The New England College of Optometry, Boston, MA Contrast Matching With Complex and Natural.
Slant Anisotropy and Tilt- dependent Variations in Stereo Precision Tandra Ghose Vision Science Program UC Berkeley James M. Hillis.
Three Dimensional Visual Display Systems for Virtual Environments Michael McKenna, David Zeltzer Presence, Vol. I, No. 4, 1992 Presenter: Dong Jeong.
Imaging Science FundamentalsChester F. Carlson Center for Imaging Science Binocular Vision and The Perception of Depth.
Pictorial space Lavanya Sharan March 21st, Real world vs. 2-D depiction Real world (this is obviously a photographic substitute) Image sources:
Motion Depth Cues – Motion 1. Parallax. Motion Depth Cues – Parallax.
Today: Finish Gregory Discussion, finish stereograms Tuesday: Start colour vision Thursday: Finish colour vision - Read LAND for Thursday.
Spacecraft Stereo Imaging Systems Group S3. Variables Separation of the cameras Height of the cameras – relative to the bench Angle – The direction cameras.
What’s on page 13-25? Tom Butkiewicz. Refresh Rates Flicker from shutter systems Halve refresh rates 2 eyed 120Hz != 1 eyed 60Hz Phosphors 2 Polarized.
Focus Cues Kurt Akeley CS248 Lecture 20 6 December 2007
Homework Set 5: Due Wednesday, March, 17 From Chapter 5: P2, P8, P10, P11, From Chapter 6: P1, P2, P6, PM2,
Space Perception Depth Cues Tasks Shape-from-Shading.
Reading Gregory 24 th Pinker 26 th. Seeing Depth What’s the big problem with seeing depth ?
3/23/2005 © Dr. Zachary Wartell 1 3D Displays Overview Revision 1.3 Copyright 2006 Zachary Wartell University of North Carolina Charlotte.
Ganglion cells project to the brain via the optic nerve information is projected to contralateral cortex! Visual Pathways.
Dinesh Ganotra. each of the two eyes sees a scene from a slightly different perspective.
1B50 – Percepts and Concepts Daniel J Hulme. Outline Cognitive Vision –Why do we want computers to see? –Why can’t computers see? –Introducing percepts.
Careers for Psychology and Neuroscience Majors Oct. 19th5-7pm in SU 300 Ballroom B.
Speaker: Chi-Yu Hsu Advisor: Prof. Jian-Jung Ding Leveraging Stereopsis for Saliency Analysis, CVPR 2012.
PRCTICAL DIFFICULTIES OF PROGRESSIVE LENS FITTING
3D/Multview Video. Outline Introduction 3D Perception and HVS 3D Displays 3D Video Representation Compression.
New requirement of subjective video quality assessment methodologies for 3DTV Wei Chen(*) 1,2, Jérôme Fournier 1, Marcus Barkowsky 2, Patrick Le Callet.
CAP4730: Computational Structures in Computer Graphics 3D Concepts.
Visual Angle How large an object appears, and how much detail we can see on it, depends on the size of the image it makes on the retina. This, in turns,
Spatiotemporal Information Processing No.3 3 components of Virtual Reality-2 Display System Kazuhiko HAMAMOTO Dept. of Information Media Technology, School.
Fundamental of Optical Engineering Lecture 3.  Aberration happens when the rays do not converge to a point where it should be. We may distinguish the.
With respect to STM, grouping several items together to form a single larger item is called: A.BlockingB.Lumping C.ChunkingD.Grouping Electrochemical.
Space Perception: the towards- away direction The third dimension Depth Cues Tasks Navigation Cost of Knowledge Interaction.
By Andrea Rees. Gestalt Principles 1) Closure 2) Proximity 3) Similarity 4) Figure VISUAL PERCEPTION PRINCIPLES OVERVIEW Depth Principles Binocular 1)
Chapter 5 Human Stereopsis, Fusion, and Stereoscopic Virtual Environments.
BY JESSIE PARKER VISUAL PERCEPTION PRINCIPLES. VISUAL PERCEPTION Visual perception is the ability to interpret the surrounding environment by processing.
© by Yu Hen Hu 1 Human Visual System. © by Yu Hen Hu 2 Understanding HVS, Why? l Image is to be SEEN! l Perceptual Based Image Processing.
Presentation Overview
1 Perception, Illusion and VR HNRS 299, Spring 2008 Lecture 8 Seeing Depth.
Depth Perception and Visualization Matt Williams From:
Physiological Depth Cues – Convergence. Physiological Depth Cues – Convergence – small angle of convergence = far away – large angle of convergence =
Perception By: Alyssa Beavers, Chris Gordon, Yelena Pham, Hannah Schulte.
Stereo Viewing Mel Slater Virtual Environments
CSE 185 Introduction to Computer Vision Stereo. Taken at the same time or sequential in time stereo vision structure from motion optical flow Multiple.
The Limitations of Stereoscopic 3D Display Technologies as a Path to Immersive Reality Aris Silzars, PhD Northlight Displays.
Spatiotemporal Information Processing No.3 3 components of Virtual Reality-2 Display System Kazuhiko HAMAMOTO Dept. of Information Media Technology, School.
Depth Perception and Perceptional Illusions. Depth Perception The use of visual cues to perceive the distance or three-dimensional characteristics of.
Evaluating Perceptual Cue Reliabilities Robert Jacobs Department of Brain and Cognitive Sciences University of Rochester.
Taking stereoscopic tours of astronomical scenes Stuart Levy with Robert Patterson Advanced Visualization Lab - AVL at NCSA Nat'l Center for Supercomputing.
Visual Perception Principles Visual perception principles are ‘rules’ that we apply to visual information to assist our organisation and interpretation.
Immersive Rendering. General Idea ► Head pose determines eye position  Why not track the eyes? ► Eye position determines perspective point ► Eye properties.
Stereoscopic Images Binocular vision enables us to measure depth using eye convergence and stereoscopic vision. Eye convergence is a measure of the angle.
Visual Perception There are two categories of cognitive processes that we use when we assign meaning to incoming information. What are they?
Human Visual System.
Visual Perception. What is Visual Perception? Visual perception are rules we apply to visual information to assist our organisation and interpretation.
Space Perception: the towards- away direction Cost of Knowledge Depth Cues Tasks Navigation.
Examination Techniques for Accuracy and Efficiency Astigmatism Detection and Management Options A VOSH-Florida Presentation.
Design of Visual Displays for Driving Simulator Research G. John Andersen Department of Psychology University of California, Riverside.
Careers for Psychology and Neuroscience Majors Oct. 19th5-7pm in SU 300 Ballroom B.
Journal of Vision. 2011;11(8):11. doi: / Figure Legend:
Journal of Vision. 2008;8(3):33. doi: / Figure Legend:
Journal of Vision. 2011;11(8):11. doi: / Figure Legend:
From: Slant from texture and disparity cues: Optimal cue combination
Journal of Vision. 2011;11(8):11. doi: / Figure Legend:
Common Classification Tasks
Examination Techniques for Accuracy and Efficiency
Presentation transcript:

User Issues in 3D TV & Cinema Martin S. Banks Vision Science Program UC Berkeley

Issues in 3D TV & Cinema Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals

Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals Issues in 3D TV & Cinema

Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals Issues in 3D TV & Cinema

Focal distance Vergence distance Vergence & Accommodation: Natural Viewing

Vergence Distance (diopters) 0 0 Focal Distance (diopters) Focal distance Vergence distance Vergence & Accommodation: Natural Viewing

Vergence Distance (diopters) 0 0 zone of clear single binocular vision Focal Distance (diopters) Focal distance Vergence distance Vergence & Accommodation: Natural Viewing

Vergence Distance (diopters) 0 0 Percival's zone of comfort zone of clear single binocular vision Focal Distance (diopters) Focal distance Vergence distance Vergence & Accommodation: Natural Viewing

Focal distance Vergence distance Vergence & Accommodation: Stereo Display

Vergence Distance (diopters) 0 0 Focal Distance (diopters) Focal distance Vergence distance Vergence & Accommodation: Stereo Display Percival's zone of comfort zone of clear single binocular vision

Displays with Nearly Correct Focus Cues Two multi-focal displays we’ve developed: 1.Fixed-viewpoint, volumetric display with mirror system & 3 focal planes (Akeley, Watt, Girshick, & Banks, SIGGRAPH, 2004). 2.Fixed-viewpoint, volumetric display with switchable lens & 4 focal planes (Love, Hoffman, Kirby, Hands, Gao, & Banks, Optics Express, 2009)

Multi-focal Display Akeley, Watt, Girshick & Banks (2004), SIGGRAPH.

Multi-focal Display

Akeley, Watt, Girshick & Banks (2004), SIGGRAPH. Multi-focal Display

Depth-weighted Blending Depth-weighted blending along lines of sight Weights dependent on dioptric distances to planes Akeley, Watt, Girshick, & Banks (2004), SIGGRAPH.

Do V-A Conflicts Cause Fatigue/Discomfort?

600-ms stimulus at near or far vergence-specified distance Appeared at each focal distance Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision Do V-A Conflicts Cause Fatigue/Discomfort?

** cues-inconsistent cues-consistent Severity of Symptom How tired are your eyes? How clear is your vision? How tired or sore are your neck & back? How do your eyes feel? How does your head feel? ** = p < 0.01 (Wilcoxen test) Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision Do V-A Conflicts Cause Fatigue/Discomfort?

** * Which session was more fatiguing? Which session irritated your eyes more? Which session gave you more headache? Which session did you prefer? cues-consistent much worse than inconsistent cues-inconsistent much worse than consistent no difference ** = p < 0.01 (Wilcoxen test) Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision Do V-A Conflicts Cause Fatigue/Discomfort?

Discomfort & 3D Cinema

Discomfort & 3D Cinema & TV

Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals Issues in 3D TV & Cinema

Almost never view pictures from correct position. Retinal image thus specifies different scene than depicted. Do people compensate, and if so, how? Viewing Pictures

Stimuli Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

Stimulus: simulated 3D ovoid with variable aspect ratio. Task: adjust ovoid until appears spherical. Vary monitor slant S m to assess compensation for oblique viewing positions. Spatial calibration procedure. If compensate, will set ovoid to sphere on screen (ellipse on retina). Observation Point SmSm CRT Experimental Task Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

Center of Projection Observation Point No compensation: set ovoid to make image on retina circular: retinal coordinates screen coordinates Predictions

Center of Projection Observation Point Compensation: Set ovoid to make image on screen circular: Predictions retinal coordinates screen coordinates

Aspect Ratio (screen coords) invariance predictions Viewing Angle S m (deg) SmSm Predictions

invariance predictions retinal predictions Aspect Ratio (screen coords) Viewing Angle S m (deg) SmSm

Results monoc-aperture invariance predicts retinal predicts Aspect Ratio (screen coords) Viewing Angle (deg) JLL Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

Results monoc-aperture binoc-no aperture invariance predicts retinal predicts JLL Results Aspect Ratio (screen coords) Viewing Angle (deg) Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

Compensation for Incorrect Viewing Position Pictures not useful unless percepts are robust to changes in viewing position. People compensate for oblique viewing position when viewing 2d pictures. Two theories of compensation: pictorial & surface. Data clearly favor surface compensation. Two versions of surface method: global & local. Data clearly favor local slant.

2D Pictures vs 3D Pictures Two eyes presented same image Binocular disparities specify orientation & distance of picture surface; hence useful for compensation 2D

Two eyes presented different images Binocular disparities specify orientation & distance of picture surface and layout of picture contents; hence not useful for compensation Two eyes presented same image Binocular disparities specify orientation & distance of picture surface; hence useful for compensation 3D Two eyes presented same image Binocular disparities specify orientation & distance of picture surface; hence useful for compensation 2D 2D Pictures vs 3D Pictures

Stereo (3D) Pictures For most applications, viewers will not be at correct position. Retinal disparities thus specify a different layout than depicted. Do people compensate? Is correct seating position for a 3D movie more important than for 2D movie?

Stereo Picture Geometry display surface stereo projectors

display surface stereo projectors depicted hinge Stereo Picture Geometry

display surface stereo projectors depicted hinge Stereo Picture Geometry

display surface stereo projectors depicted hinge disparity-specified hinge Stereo Picture Geometry

perceived dihedral angle? display surface stereo projectors depicted hinge disparity-specified hinge Stereo Picture Geometry

Predictions Viewing Angle (deg) 35° 17.5° 0°0° -17.5° -35° Hinge Setting (deg) Invariance: Hinge settings are 90° for all viewing angles and base slants Retinal disparity: Hinge settings vary significantly with viewing angle & base slant

Viewing Angle (deg) Hinge Setting (deg) Results non-stereo pictures

Viewing Angle (deg) stereo pictures Hinge Setting (deg) Results non-stereo pictures

Viewing Angle (deg) Hinge Setting (deg) non-stereo picturesstereo pictures Results

Summary User issues in 3D cinema & TV Vergence-accommodation conflicts cause visual fatigue & discomfort Can be handled by attending to viewer’s distance from screen & range of disparities presented relative to screen Perceptual distortions due to incorrect viewing position Compensation is good with non-stereo pictures Compensation is significantly poorer with stereo pictures suggesting that viewer position could be more important

Acknowledgments Kurt Akeley (Microsoft) Simon Watt (Univ. of Wales, Bangor) Ahna Girshick (NYU) David Hoffman (UC Berkeley) Robin Held (UC Berkeley) Funding from NIH, NSF, & Sharp Labs