Computer Graphics Psychophysics Heinrich H. Bülthoff Max-Planck-Institute for Biological Cybernetics Tübingen, Germany Heinrich H. Bülthoff Max-Planck-Institute.

Slides:



Advertisements
Similar presentations
Virtual Reality Design Virtual reality systems are designed to produce in the participant the cognitive effects of feeling immersed in the environment.
Advertisements

SPECULAR FLOW AND THE PERCEPTION OF SURFACE REFLECTANCE Stefan Roth * Fulvio Domini † Michael J. Black * * Computer Science † Cognitive and Linguistic.
2002/02/12PSYC202, Term 2, Copyright Jason Harrison Illusions tricking the processes that estimate properties of the world.
Vision For Graphics ICCV 2005 Vision for Graphics Larry Zitnick, Sing Bing Kang, Rick Szeliski Interactive Visual Media Group Microsoft Research Steve.
Image-Based Modeling, Rendering, and Lighting
Advanced Graphics, Overview Advanced Computer Graphics Overview.
Exchanging Faces in Images SIGGRAPH ’04 Blanz V., Scherbaum K., Vetter T., Seidel HP. Speaker: Alvin Date: 21 July 2004.
Visual Cognition II Object Perception. Theories of Object Recognition Template matching models Feature matching Models Recognition-by-components Configural.
Advanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light Fields Ravi Ramamoorthi
CS 561, Sessions 27 1 Towards intelligent machines Thanks to CSCI561, we now know how to… - Search (and play games) - Build a knowledge base using FOL.
Representations of Visual Appearance COMS 6160 [Spring 2007], Lecture 4 Image-Based Modeling and Rendering
Perception of illumination and shadows Lavanya Sharan February 14th, 2011.
High-Quality Video View Interpolation
Now, we want to start talking about images produced by different types of mirrors: flat, convex, and concave ones. Optical images are of two types, called.
Computing With Images: Outlook and applications
Visual Cognition I basic processes. What is perception good for? We often receive incomplete information through our senses. Information can be highly.
Perception of illumination and shadows Lavanya Sharan February 14th, 2011.
Materials II Lavanya Sharan March 2nd, Computational thinking about materials Physics-basedPseudo physics-based.
1 Lecture 11 Scene Modeling. 2 Multiple Orthographic Views The easiest way is to project the scene with parallel orthographic projections. Fast rendering.
Laurent Itti: CS599 – Computational Architectures in Biological Vision, USC Lecture 11: Visual Illusions 1 Computational Architectures in Biological.
Jochen Triesch, UC San Diego, 1 COGS Visual Modeling Jochen Triesch & Martin Sereno Dept. of Cognitive Science UC.
CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.
 Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”
View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.
Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Lighting.
Computer Vision in Graphics Production Adrian Hilton Visual Media Research Group Centre for Vision, Speech and Signal Processing University of Surrey
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean Hall 5409 T-R 10:30am – 11:50am.
1 Perception and VR MONT 104S, Spring 2008 Lecture 22 Other Graphics Considerations Review.
CAP4730: Computational Structures in Computer Graphics 3D Concepts.
MIND: The Cognitive Side of Mind and Brain  “… the mind is not the brain, but what the brain does…” (Pinker, 1997)
Technology and Historical Overview. Introduction to 3d Computer Graphics  3D computer graphics is the science, study, and method of projecting a mathematical.
Active Vision Key points: Acting to obtain information Eye movements Depth from motion parallax Extracting motion information from a spatio-temporal pattern.
Advanced Computer Graphics March 06, Grading Programming assignments Paper study and reports (flipped classroom) Final project No written exams.
Biases: An Example Non-accidental properties: Properties that appear in an image that are very unlikely to have been produced by chance, and therefore.
Image-based rendering Michael F. Cohen Microsoft Research.
IMAGE SYNTHESIS 1 Image Synthesis Image synthesis operations create images from other images or non-image data Used when a desired image is either physically.
1 Perception, Illusion and VR HNRS 299, Spring 2008 Lecture 8 Seeing Depth.
Graphical Models in Vision. Alan L. Yuille. UCLA. Dept. Statistics.
Perception Chapter 5.
Object Recognition in Images Slides originally created by Bernd Heisele.
Shadows. Shadows is important in scenes, consolidating spatial relationships “Geometric shadows”: the shape of an area in shadow Early days, just pasted.
Advanced Computer Graphics Advanced Shaders CO2409 Computer Graphics Week 16.
Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography.
1 Artificial Intelligence: Vision Stages of analysis Low level vision Surfaces and distance Object Matching.
W E L C O M E. A U G M E N T E D R E A L I T Y A SEMINAR BY JEFFREY J M EC7A ROLL NO:
Tutorial Visual Perception Towards Computer Vision
Art Technologies Unit 1 Lesson 5. In this lesson you will learn…  How optical illusions work  How optical illusions are really “errors in perception”
Evaluating Perceptual Cue Reliabilities Robert Jacobs Department of Brain and Cognitive Sciences University of Rochester.
Global Illumination. Local Illumination  the GPU pipeline is designed for local illumination  only the surface data at the visible point is needed to.
Fundamentals of Sensation and Perception RECOGNIZING VISUAL OBJECTS ERIK CHEVRIER NOVEMBER 23, 2015.
Computer Vision, CS766 Staff Instructor: Li Zhang TA: Yu-Chi Lai
IMAGE SYNTHESIS 1 Image Synthesis Image synthesis operations create images from other images or non-image data Used when a desired image is either physically.
Give examples of the way that virtual reality can be used in Psychology.
1. Introduction The presence of familiar object motion is known to facilitate visual object recognition 1,2. In addition to shape cues, observers can utilise.
Perception and VR MONT 104S, Fall 2008 Lecture 8 Seeing Depth
Sub-Surface Scattering Real-time Rendering Sub-Surface Scattering CSE 781 Prof. Roger Crawfis.
CS Chapter 11.5 – Computer GraphicsPage 145 Computer Graphics Recent advances in processor power, display technology, memory capacity, and rendering.
From local motion estimates to global ones - physiology:
Advanced Computer Graphics
Processing visual information for Computer Vision
- Introduction - Graphics Pipeline
Image-Based Rendering
The Graphics Rendering Pipeline
Image Based Modeling and Rendering (PI: Malik)
Computer Vision Computer vision attempts to construct meaningful and explicit descriptions of the world depicted in an image Using machines to Interpret!!!
A preference for global convexity in local shape perception
Announcements Review on Thurs Project 4 Extra office hour: Friday 4-5
?. ? White Fuzzy Color Oblong Texture Shape Most problems in vision are underconstrained White Color Most problems in vision are underconstrained.
Object recognition using shading
Psychophysical and Physiological Evidence for Viewer-centered Object Representations in the Primate N.K. Logothetis and J. Pauls Cerebral Cortex (1995)
Presentation transcript:

Computer Graphics Psychophysics Heinrich H. Bülthoff Max-Planck-Institute for Biological Cybernetics Tübingen, Germany Heinrich H. Bülthoff Max-Planck-Institute for Biological Cybernetics Tübingen, Germany

What is psychophysics? A research strategy to understand perception and performance by testing the relationship between the psychic (what an observer sees and reports) and the physical (patterns of light entering the eye).

Method Show and tell : –static images –video animations –vision + haptics –virtual environments Show and tell : –static images –video animations –vision + haptics –virtual environments

Basic Question: What image information does the brain use and what does it ignore ? Applied Question: How can we render images that convey only the information that the brain uses? Basic Question: What image information does the brain use and what does it ignore ? Applied Question: How can we render images that convey only the information that the brain uses? Main questions

Examples of vision problems – Recognition – Depth perception – Navigation –(Haptics) – Recognition – Depth perception – Navigation –(Haptics)

Computer Graphics Psychophysics at the MPI Tuebingen RecognitionRecognition Shape PerceptionShape Perception HapticsHaptics DrivingDriving NavigationNavigation RecognitionRecognition Shape PerceptionShape Perception HapticsHaptics DrivingDriving NavigationNavigation

Object recognition – how does it work? Naïve theory : First, the brain computes an object’s 3-D shape. Then the brain matches the 3-D shape to previously seen shapes that are stored in memory. image z-buffer 3-D shape memory image z-buffer 3-D shape memory Naïve theory : First, the brain computes an object’s 3-D shape. Then the brain matches the 3-D shape to previously seen shapes that are stored in memory. image z-buffer 3-D shape memory image z-buffer 3-D shape memory

Image-based Recognition Alternative Theory: The visual system ignores depth perception when recognizing an object. Rather, the visual system recognizes an object directly from its image. z-buffer (depth perception) z-buffer (depth perception) image image image memory (recognition) image memory (recognition) Alternative Theory: The visual system ignores depth perception when recognizing an object. Rather, the visual system recognizes an object directly from its image. z-buffer (depth perception) z-buffer (depth perception) image image image memory (recognition) image memory (recognition)

Recognition of Biological Motion Recognition of point-like walker

Biological Motion Perception Johansson ( ) Walter: insert orig. johannson movie here 30 sec max Walter: insert orig. johannson movie here 30 sec max

Depth perception from stereo is ignored Bülthoff, Bülthoff and Sinha (Nature Neuroscience 1998) Recognition is unaffected by scrambling the depth structure 2D motion pattern and not 3D structure is used for recognition Z y z x

Image-based Recognition Evidence from: Psychophysics (Buelthoff, MPI Tuebingen)Psychophysics (Buelthoff, MPI Tuebingen) –limited generalization (30°) despite full 3D information Psychophysics (Buelthoff, MPI Tuebingen)Psychophysics (Buelthoff, MPI Tuebingen) –limited generalization (30°) despite full 3D information Physiology (Logothetis, MPI Tuebingen)Physiology (Logothetis, MPI Tuebingen) –image-specific neurons in trained monkeys Physiology (Logothetis, MPI Tuebingen)Physiology (Logothetis, MPI Tuebingen) –image-specific neurons in trained monkeys Theory (Poggio, MIT)Theory (Poggio, MIT) –image-interpolation networks Theory (Poggio, MIT)Theory (Poggio, MIT) –image-interpolation networks Rendering Application (Blanz & Vetter, MPI Tuebingen)Rendering Application (Blanz & Vetter, MPI Tuebingen) –image-based face synthesis Rendering Application (Blanz & Vetter, MPI Tuebingen)Rendering Application (Blanz & Vetter, MPI Tuebingen) –image-based face synthesis

One Object - Two Interpretations Markus Raetz

Man or Hare ? Markus Raetz

Why does the brain make assumptions? Images are ambiguous. Any image can be explained by several combinations of 3-D shape, material and lighting.

e.g. The Necker Cube

view from aboveview from below

e.g. Depth-reversal ambiguity in shading A valley illuminated from the right looks the same as a hill illuminated from the left. hillvalley

Prior assumption about light source

Mould of a footprint ?

Assumption that light source is stationary

Prior assumptions about shape The Hollow Mask Illusion

3 prior assumptions 1. light from above 2. viewpoint from above 3. shape is convex 1. light from above 2. viewpoint from above 3. shape is convex

Example: failure of all 3 prior assumptions shape is concave viewpoint from below light from below shape is concave viewpoint from below light from below

“ Measuring Visual Shape using Computer Graphics Psychophysics” (see Workshop proceedings) convex concave convex concave (face) (mask) (face) (mask) convex concave convex concave (face) (mask) (face) (mask)

Procedure

Task: hill or valley ?

Assumption 1 : light direction lightfromabovelightfrombelow lightfromabovelightfrombelow

Assumption 2 : viewpoint direction (case 1 - convex shape) view from above view from above view from below view from below view from above view from above view from below view from below

Assumption 2 : viewpoint direction (case 2 – concave shape) view from below view from below view from above view from above view from below view from below view from above view from above

Data : per cent correct scores (see workshop proceedings for more details) 87 (best) 15 (worst) 64 39

Lessons for rendering Illumination and viewpoint should be from above, especially if the surface is concave. Otherwise : -Shading will look weird. -Perceived shape will be incorrect. Illumination and viewpoint should be from above, especially if the surface is concave. Otherwise : -Shading will look weird. -Perceived shape will be incorrect.

Weird shading Frankenstein monster illuminated from below.

Face Recognition Cyberware Scanner Texture Map Shape Map Cyberware Scanner Texture Map Shape Map

MPI 3-D Face Database

View-synthesis from a single image Volker Blanz and Thomas Vetter (SIGGRAPH ‘99) Input Output W 2 * + W 1 * W 3 * + W 4 * + +….. = = W 2 * + W 1 * W 3 * + W 4 * +

All views from a single image

Synthetic Actors Volker Blanz und Thomas Vetter (SIGGRAPH 1999)

Audrey Hepburn ( ) New Hat + New Illumination Original New View

Do you recognize her ? Mona Lisa New View

Virtual Tuebingen

Summary (Visual Recogntion) Visual recognition is based on images, not on 3-D geometry.Visual recognition is based on images, not on 3-D geometry. –More texture memory not polygons Better to apply a texture map from a previously seen image than to re-render an object or scene from a new viewpoint or under a new lighting condition. Visual recognition is based on images, not on 3-D geometry.Visual recognition is based on images, not on 3-D geometry. –More texture memory not polygons Better to apply a texture map from a previously seen image than to re-render an object or scene from a new viewpoint or under a new lighting condition.

Summary Does the brain use a depth buffer? – –Yes: for shape perception (shape from shading) – –No: for recognition (biological motion) Stereo goggles not necessary for recognition. Does the brain use a depth buffer? – –Yes: for shape perception (shape from shading) – –No: for recognition (biological motion) Stereo goggles not necessary for recognition.

Summary (3-D Shape Perception) Shape from shading is strongly affected by prior assumptions hard wired into the brain.Shape from shading is strongly affected by prior assumptions hard wired into the brain. Photorealistic renderings will look weird if the scene model or viewing parameters are inconsistent with these assumptions. Photorealistic renderings will look weird if the scene model or viewing parameters are inconsistent with these assumptions. Shape from shading is strongly affected by prior assumptions hard wired into the brain.Shape from shading is strongly affected by prior assumptions hard wired into the brain. Photorealistic renderings will look weird if the scene model or viewing parameters are inconsistent with these assumptions. Photorealistic renderings will look weird if the scene model or viewing parameters are inconsistent with these assumptions.

light-from-above light-from-above light-from-below light-from-belowPercent correct 50 view-from-above view-from-below view-from-below convex convex concave concave Data (see workshop proceedings) 0