1. Components of a computer vision system
Camera
Lighting
Scene Interpretation
Computer
Scene
Srinivasa Narasimhan’s slide

2. Computer vision vs Human Vision
What we see
What a computer sees
Srinivasa Narasimhan’s slide

3. A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a
computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

4. A little story about Computer Vision
Founder, MIT AI project
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a
computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

5. A little story about Computer Vision
Founder, MIT AI project
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a
computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
Professor of Electrical Engineering, MIT

6. A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a
computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
Image Understanding

7. A little story about Computer Vision
In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a
computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)
Image Understanding
Image Sensing

8. Continue on CAPTCHA CAPTCHA stands for
"Completely Automated Public Turing test to Tell Computers and Humans Apart".
Picture of a CAPTCHA in use at Yahoo.

9. Breaking a Visual CAPTCHA
On EZ-Gimpy:
a success rate of 176/191=92%!
Other examples

10. Breaking a Visual CAPTCHA
On more difficult Gimpy:
a success rate of 33%!
Other examples

11. Breaking a Visual CAPTCHA
YAHOO’s current CAPTCHA format

12. Face Detection and Recognition
Applications: Security, Law Enforcement, Surveillance

13. Face Detection and Recognition
Smart cameras: auto focus, red eye removal, auto color correction

14. Face Detection and Tracking

15. Face Detection and Tracking

16. Face Detection and Tracking
Lexus LS600 Driver Monitor System

17. General Motion Tracking
Hidden Dragon Crouching Tiger

18. General Motion Tracking
Application
Andy Serkis, Gollum, Lord of the Rings

19. Segmentation

20. Segmentation using Graph Cuts
Application
Medical Image Processing

21. Segmentation using Graph Cuts
Input
Matting:
Soft Segmentation
Composition

22. Segmentation using Graph Cuts
State-of-the-art Tool (videosnapcut.mp4)

23. From 2D to 3D

24. Projective Geometry

25. Single View Metrology

26. Single View Metrology

27. Stereo
scene point
image plane
optical center

28. Stereo Basic Principle: Triangulation
Gives reconstruction as intersection of two rays
Requires
Camera positions
point correspondence

29. Using 3D structure to organize photos

30. Using 3D structure to organize photos

31. Reconstructing detailed 3D models
rendered model
example input image

32. Reconstructing detailed 3D models
rendered model
example input image

33. Reconstructing detailed 3D models
rendered model
example input image

34. Reconstructing detailed 3D models
rendered model
example input image

35. Reconstructing detailed 3D models
rendered model
example input image

36. Application: View morphing

37. Application: View morphing

38. From Static Statues to Dynamic Targets
MSR Image based Reality Project …|

39. Spacetime Face Capture System
Black & White Cameras
Color Cameras
Video Projectors

40. System in Action

41. Input Videos (640480, 60fps)

42. Spacetime Stereo Reconstruction

43. Entertainment: Games & Movies
Applications
Entertainment: Games & Movies
Medical Practice:
Prosthetics

44. Computational Photography
High Dynamic Range
Conventional Image
High Dynamic Range Image
Nayar et al 2002
When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere.

45. Computational Photography
High Dynamic Range
Modulator
Sensor
Optics
Assorted-pixel camera
High Dynamic Range Image
Nayar et al 2002
When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere.

46. Computational Photography
High Dynamic Range
When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere.
Handheld camera
Digital Gain Adjustment

47. Computational Photography
High Dynamic Range
When we take a photo, we often find that very bright or dark regions, the local contrast is lost. With digital cameras, we can much more easily capture a high dynamic range image, in which local contrast is well preserved everywhere.
Handheld camera
High Dynamic Range Image
Zhang et al 2010

48. Summary
Recognize things
Reconstruct 3D structures
Enhance Photography

49. If you are interested in,
Major Conferences:
Computer Vision and Pattern Recognition (CVPR)
International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
ACM SIGGRAPH Conference (SIGGRAPH)
Faculty:
Chuck Dyer, Vikas Singh, Li Zhang
Courses: CS766 Computer Vision
CS638 Special Topics
Computational Photography
Computational Methods in Medical Image Analysis