CS576 Computer Vision Instructor: Dr. Yu-Wing Tai No official TA Tuesday and Thursday 4:00pm – 5:30pm Rm 3444 E3-1 Building 1

2 Who am I ? Course Webpage: Instructor Info: Dr. Yu-Wing Tai / Education: PhD National University of Singapore 2009 M.Phil Hong Kong University of Science and Technology 2005 B.Eng Hong Kong University of Science and Technology 2003 Research: Computer Vision, Image/Video Processing Web: Office hours: arrange by .

Grading (Absolute Grading) You start with the letter grade “C-”, you earn sub-grade from: –2 course projects (2 sub-grades) –1 term project (1-2 sub-grades) –Peer review of term project (1 sub-grade) –1 oral examination (1 sub-grade) –Bonus (See project description) –Attendant (At least 60%, 1 sub-grade) Best Grade: A+ (No limit on number of A+) Worst Grade: D+ (No exception for better grade) 3

Project 1: Feature Detection and Matching Descriptions: – Deadline: Friday mid-night (00:00) on Week 5 A project directly copy from the first project of the computer vision class in U. Washington 1 sub-grade for this project; 0.5 sub-grade for unsuccessful implementation; 0 sub-grade for late submission No bonus sub-grade for Project 1 Goal: –Fundamental project for computer vision class –Learning how to use skeleton codes from others –Learning how to find related internet resources for project 4

Project 2: Paper implementation Descriptions: – Deadline: Friday mid-night (00:00) on Week 9 1 sub-grade for this project; 0.5 sub-grade for unsuccessful implementation; 0 sub-grade for late submission Maximum 1 Bonus sub-grade for extra successful submission Goal: –Paper reading –Find your own interests area –Learning how to find related resources –Learning how to reproduce previous research projects 5

Term Project: A mini-conference submission Descriptions: – Deadline: Friday mid-night (00:00) on Week 13 1 sub-grade for submission; 1 sub-grade for acceptance; 0 sub-grade for late submission Acceptance rate will be about 30% - 50% 1 sub-grade for peer evaluation Bonus sub-grade for best presentation Bonus sub-grade for 2 outstanding reviewers Bonus sub-grade(s) for extra-acceptance project Goal: –Understand the process of research cycle and paper submission –Testing your abilities/potentials in doing research/getting PhD 6

Oral Exam minutes face-to-face question and answer session testing your knowledge and your understanding to the term project Week 15 1 sub-grade for passing; 0.5 sub-grade for failure; -1 sub-grade for absent Goal: –Testing your knowledge –Getting course feedback 7

Course Resources Books (Available for borrowing): –Computer Vision: Algorithms and Applications © 2010, Richard Szeliski –Computer Vision: A Modern Approach © 2002, David A. Forsyth and Jean Ponce Computer Vision papers: Computer Graphics papers: Microsoft Academic Search Google 8

Course Resources Computer Vision Source Codes OpenCV The Middlebury Computer Vision Pages Computer Vision Algorithm Implementations Computer Vision Datasets Columbia University Computer Vision Lab 9

Other Texts UNDERGRADUATE –A Guided Tour of Computer Vision, by V. S. Nalwa, Addison- Wesley, –Introductory Techniques for 3-D Computer Vision, by Emanuele Trucco, Alessandro Verri, Prentice-Hall, 1998 GRADUATE/ Specialized Reference –Multiple View Geometry, by Richard Hartley, Andrew Zisserman, Cambridge University Press, –Numerical Recipes in C, by William Press et al., Cambridge Univ Press, –Pattern Classification and Scene Analysis, by Richard O. Duda, Peter E. Hart, John Wiley & Sons, –Convex Optimization, by Stephen Boyd and Lieven Vandenberghe,

What is Computer Vision ? In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw” “In the 1960s, almost no one realized that machine vision was difficult.” – David Marr, years later, we are still working on this… 11

What is Computer Vision ? 12

The goal of computer vision To bridge the gap between pixels and “meaning” 13

What is Computer Vision ? 14 From Wikipedia

What is Computer Vision ? 15 Computer Vision Image Processing Computer Graphics Stereo, 3D Reconstruction, etc. Image Enhancement, Denoising, etc. Rendering, 3D Modelling, etc.

1970s 16 line labeling pictorial structures articulated body model intrinsic imagesstereo correspondenceoptical flow

1980s 17 pyramid blendingshape from shadingedge detection physically-based models regularization-based surface reconstruction range data acquisition and merging

1990s 18 factorization-based structure from motion dense stereo matching multi-view reconstruction face trackingimage segmentation face recognition

2000s 19 image-based rendering image-based modeling Interactive Techniques texture synthesis feature-based recognitionregion-based recognition

State-of-the-art ? 20

Why study computer vision ? 21

Why study computer vision ? Vision is useful Vision is interesting Vision is difficult – Half of primate cerebral cortex is devoted to visual processing – Achieving human ‐ level visual perception is probably “AI ‐ complete” 22

Challenges: viewpoint variation 23

Challenges: Illumination 24

Challenges: Scale 25

Challenges: Deformation 26

Challenges: Occlusion 27

Challenges: Background cluster 28

Challenges: Motion 29

Challenges: Object intra-class variation 30

Challenges: Local Ambiguities 31

Challenges or opportunities? Images are confusing, but they also reveal the structure of the world through numerous cues Our job is to interpret the cues! 32

Computer Vision in the Real World Special Effects in movie 33

Computer Vision in the Real World 3D Urban Modeling 34

Computer Vision in the Real World Microsoft Photosynth ( 35

Computer Vision in the Real World Face Detection 36

Computer Vision in the Real World Biometrics 37

Computer Vision in the Real World Optical Character Recognition (OCR) 38

Computer Vision in the Real World Toys and Robots 39

Computer Vision in the Real World Games 40

Computer Vision in the Real World Automotive Safety 41

Computer Vision in the Real World Vision for robotics, space exploration 42

Goal of this course Broaden your view about computer vision, but we are not going to study any specific topic in deep Teaching you how to find computer vision research resources yourself Understand the research world and teaching you how to be a good computer vision scientist 43

Question ? 44