Components of a computer vision system Camera Lighting Scene Interpretation Computer Scene Srinivasa Narasimhan’s slide

Computer vision vs Human Vision What we see What a computer sees Srinivasa Narasimhan’s slide

A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

A little story about Computer Vision Founder, MIT AI project In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

A little story about Computer Vision Founder, MIT AI project In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Professor of Electrical Engineering, MIT

A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Image Understanding

A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Image Understanding Image Sensing

Continue on CAPTCHA CAPTCHA stands for "Completely Automated Public Turing test to Tell Computers and Humans Apart". Picture of a CAPTCHA in use at Yahoo. http://www.cs.sfu.ca/~mori/research/gimpy/

Breaking a Visual CAPTCHA On EZ-Gimpy: a success rate of 176/191=92%! Other examples http://www.cs.sfu.ca/~mori/research/gimpy/ez/ http://www.cs.sfu.ca/~mori/research/gimpy/

Breaking a Visual CAPTCHA On more difficult Gimpy: a success rate of 33%! Other examples http://www.cs.sfu.ca/~mori/research/gimpy/hard/ http://www.cs.sfu.ca/~mori/research/gimpy/

Breaking a Visual CAPTCHA YAHOO’s current CAPTCHA format http://en.wikipedia.org/wiki/CAPTCHA

Face Detection and Recognition Applications: Security, Law Enforcement, Surveillance

Face Detection and Recognition Smart cameras: auto focus, red eye removal, auto color correction

Face Detection and Tracking

Face Detection and Tracking

Face Detection and Tracking Lexus LS600 Driver Monitor System

General Motion Tracking Hidden Dragon Crouching Tiger

General Motion Tracking Application Andy Serkis, Gollum, Lord of the Rings

Segmentation http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/

Segmentation using Graph Cuts Application Medical Image Processing

Segmentation using Graph Cuts Input Matting: Soft Segmentation Composition

Segmentation using Graph Cuts State-of-the-art Tool (videosnapcut.mp4) http://juew.org/projects/SnapCut/snapcut.htm

From 2D to 3D http://www.eecs.harvard.edu/~zickler/helmholtz.html

Projective Geometry

Single View Metrology http://research.microsoft.com/vision/cambridge/3d/default.htm

Single View Metrology http://research.microsoft.com/vision/cambridge/3d/default.htm

Stereo scene point image plane optical center

Stereo Basic Principle: Triangulation Gives reconstruction as intersection of two rays Requires Camera positions point correspondence

Using 3D structure to organize photos http://phototour.cs.washington.edu/

Using 3D structure to organize photos http://photosynth.net/

Reconstructing detailed 3D models rendered model example input image

Reconstructing detailed 3D models rendered model example input image

Reconstructing detailed 3D models http://grail.cs.washington.edu/projects/mvscpc/ rendered model example input image

Reconstructing detailed 3D models rendered model example input image

Reconstructing detailed 3D models rendered model example input image

Application: View morphing

Application: View morphing

From Static Statues to Dynamic Targets MSR Image based Reality Project http://research.microsoft.com/~larryz/videoviewinterpolation.htm …|

Spacetime Face Capture System Black & White Cameras Color Cameras Video Projectors

System in Action

Input Videos (640480, 60fps)

Spacetime Stereo Reconstruction

Entertainment: Games & Movies Applications Entertainment: Games & Movies Medical Practice: Prosthetics

Computational Photography High Dynamic Range Conventional Image High Dynamic Range Image Nayar et al 2002 When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere.

Computational Photography High Dynamic Range Modulator Sensor Optics Assorted-pixel camera High Dynamic Range Image Nayar et al 2002 When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere.

Computational Photography High Dynamic Range When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere. Handheld camera Digital Gain Adjustment

Computational Photography High Dynamic Range When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere. Handheld camera High Dynamic Range Image Zhang et al 2010

Summary Recognize things Reconstruct 3D structures Enhance Photography

If you are interested in, Major Conferences: Computer Vision and Pattern Recognition (CVPR) International Conference on Computer Vision (ICCV) European Conference on Computer Vision (ECCV) ACM SIGGRAPH Conference (SIGGRAPH) Faculty: Chuck Dyer, Vikas Singh, Li Zhang Courses: CS766 Computer Vision CS638 Special Topics Computational Photography Computational Methods in Medical Image Analysis