1. Introduction to Computer Vision CS223B, Winter 2005

2. 1/25/2005Introduction to Computer Vision2 Richard Szeliski – Guest Lecturer Ph. D., Carnegie Mellon, 1988 Researcher, Cambridge Research Lab at DEC, 1990-1995 Senior Researcher, Interactive Visual Media Group, Microsoft, 1995- Research interests: computer vision (stereo, motion), computer graphics (image-based rendering), parallel programming

3. What is Computer Vision?

4. 1/25/2005Introduction to Computer Vision4 What is Computer Vision? Image Understanding (AI, behavior) A sensor modality for robotics Computer emulation of human vision Inverse of Computer Graphics Computer vision World model Computer graphics World model

5. 1/25/2005Introduction to Computer Vision5 Intersection of Vision and Graphics modeling - shape - light - motion - optics - images IP animation rendering user-interfaces surface design Computer Graphics shape estimation motion estimation recognition 2D modeling modeling - shape - light - motion - optics - images IP Computer Vision

6. 1/25/2005Introduction to Computer Vision6 Computer Vision [Trucco&Verri’98]

7. 1/25/2005Introduction to Computer Vision7 Image-Based Modeling Images (2D) Geometry (3D) shape Photometry appearance + graphics vision image processing 2.1 Geometric image formation 2.2 Photometric image formation 3 Image processing 4 Feature extraction 5 Camera calibration 6 Structure from motion 7 Image alignment 8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 12 Photometric recovery 14 Image-based rendering

8. 1/25/2005Introduction to Computer Vision8 Related disciplines Image Processing Scientific / medical imaging Pattern Recognition Computer Graphics Learning Artificial Intelligence Visual Neuroscience Applied Mathematics

9. 1/25/2005Introduction to Computer Vision9 Mathematics What kinds of mathematics are used? Signal and image processing Euclidean and projective geometry Vector calculus Partial differential equations Optimization Probabilistic estimation

10. Syllabus What we will be covering in this course

11. 1/25/2005Introduction to Computer Vision11 Syllabus Image Transforms / Representations filters, pyramids, steerable filters warping and resampling blending image statistics, denoising/coding edge and feature detection

12. 1/25/2005Introduction to Computer Vision12 Image Pyramid Bandpass Images Lowpass Images

13. 1/25/2005Introduction to Computer Vision13 Pyramid Blending

14. 1/25/2005Introduction to Computer Vision14 Parametric (global) warping Examples of parametric warps: translation rotation aspect affine perspective cylindrical

15. 1/25/2005Introduction to Computer Vision15 Syllabus Optical Flow least squares regression iterative, coarse-to-fine parametric robust flow and mixture models layers, EM

16. 1/25/2005Introduction to Computer Vision16 Image Morphing

17. 1/25/2005Introduction to Computer Vision17 Syllabus Projective geometry points, lines, planes, transforms Camera calibration and pose point matching and tracking lens distortion Image registration mosaics

18. 1/25/2005Introduction to Computer Vision18 Panoramic Mosaics + + … + =

19. 1/25/2005Introduction to Computer Vision19 Syllabus 3D structure from motion two frame techniques factorization of shape and motion bundle adjustment

20. 1/25/2005Introduction to Computer Vision20 3D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996

21. 1/25/2005Introduction to Computer Vision21 Face Modeling

22. 1/25/2005Introduction to Computer Vision22 Syllabus Stereo correspondence local methods global optimization

23. 1/25/2005Introduction to Computer Vision23 View Morphing Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’96]

24. 1/25/2005Introduction to Computer Vision24 Z-keying: mix live and synthetic Takeo Kanade, CMU (Stereo Machine)Stereo Machine

25. 1/25/2005Introduction to Computer Vision25 Virtualized Reality TM Takeo Kanade, CMU collect video from 50+ stream reconstruct 3D model sequences http://www.cs.cmu.edu/afs/cs/project/VirtualizedR/www/VirtualizedR.html http://www.cs.cmu.edu/afs/cs/project/VirtualizedR/www/VirtualizedR.html

26. 1/25/2005Introduction to Computer Vision26 Virtualized Reality TM Takeo Kanade, CMU generate new video steerable version used for SuperBowl XXV “eye vision” system

27. 1/25/2005Introduction to Computer Vision27 Syllabus Tracking eigen-tracking on-line EM Kalman filter particle filtering appearance models

28. 1/25/2005Introduction to Computer Vision28 Syllabus Recognition subspaces and local invariance features face recognition color histograms textures Image editing segmentation curve tracking

29. 1/25/2005Introduction to Computer Vision29 Edge detection and editing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain," Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374-381, June, 1998.

30. 1/25/2005Introduction to Computer Vision30 Image Enhancement High dynamic range photography [Debevec et al.’97; Mitsunaga & Nayar’99] combine several different exposures together

31. 1/25/2005Introduction to Computer Vision31 Syllabus Image-based rendering Lightfields and Lumigraphs concentric mosaics layered models video-based rendering

32. 1/25/2005Introduction to Computer Vision32 Concentric Mosaics Interpolate between several panoramas to give a 3D depth effect [Shum & He, SIGGRAPH’99]

33. 1/25/2005Introduction to Computer Vision33 Applications Geometric reconstruction: modeling, forensics, special effects (ILM, RealVis,2D3) Image and video editing (Avid, Adobe) Webcasting and Indexing Digital Video (Virage) Scientific / medical applications (GE)

34. 1/25/2005Introduction to Computer Vision34 Applications Tracking and surveillance (Sarnoff) Fingerprint recognition (Digital Persona) Biometrics / iris scans (Iridian Technologies) Vehicle safety (MobilEye) Drowning people (VisionIQ Inc) Optical motion capture (Vicon)

35. 1/25/2005Introduction to Computer Vision35 Projects Let’s look at what students have done in previous years … Stanford http://www.stanford.edu/class/cs223b/winter01-02/projects.htmlhttp://www.stanford.edu/class/cs223b/winter01-02/projects.html CMU http://www-2.cs.cmu.edu/~ph/869/www/869.htmlhttp://www-2.cs.cmu.edu/~ph/869/www/869.html UW http://www.cs.washington.edu/education/courses/cse590ss/01wi/http://www.cs.washington.edu/education/courses/cse590ss/01wi/ GA Tech http://www.cc.gatech.edu/classes/AY2002/cs4480_spring/http://www.cc.gatech.edu/classes/AY2002/cs4480_spring/