1. Computer Science Department AI@Azusa Pacific University Artificial Intelligence -- Computer Vision

2. Why Computer Vision? Vision, communication, & action AI@Azusa Pacific University

3. Why study Computer Vision? F Images and video are everywhere F Fast-growing collection of useful applications F matching and modifying images from digital cameras F film special effects and post-processing F building representations of the 3D world from pictures F medical imaging, household robots, security, traffic control, cell phone location, face finding, video game interfaces,... F Various deep and attractive scientific mysteries F what can we know from an image? F how does object recognition work? F Greater understanding of human vision and the brain F about 25% of the human brain is devoted to vision F Images and video are everywhere F Fast-growing collection of useful applications F matching and modifying images from digital cameras F film special effects and post-processing F building representations of the 3D world from pictures F medical imaging, household robots, security, traffic control, cell phone location, face finding, video game interfaces,... F Various deep and attractive scientific mysteries F what can we know from an image? F how does object recognition work? F Greater understanding of human vision and the brain F about 25% of the human brain is devoted to vision AI@Azusa Pacific University

4. Why is Vision Interesting? F Psychology F ~ 50% of cerebral cortex is for vision. F Vision is how we experience the world. F Engineering F Want machines to interact with world. F Digital images are everywhere. F Psychology F ~ 50% of cerebral cortex is for vision. F Vision is how we experience the world. F Engineering F Want machines to interact with world. F Digital images are everywhere.

5. Vision is inferential: Illumination http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html AI@Azusa Pacific University

6. Applications of Computer Vision: Texture generation Input image Pattern Repeated New texture generated from input Simple repetition AI@Azusa Pacific University

7. Application: Football first-down line Requires (1) accurate camera registration; (2) a model for distinguishing foreground from background www.sportvision.com AI@Azusa Pacific University

8. Application areas:  Film production (the “ match move ” problem)  Heads-up display for cars  Tourism  Architecture  Training Technical challenges:  Recognition of scene  Accurate sub-pixel 3-D pose  Real-time, low latency Application areas:  Film production (the “ match move ” problem)  Heads-up display for cars  Tourism  Architecture  Training Technical challenges:  Recognition of scene  Accurate sub-pixel 3-D pose  Real-time, low latency Application: Augmented Reality AI@Azusa Pacific University

9. Application: Medical augmented Reality Visually guided surgery: recognition and registration AI@Azusa Pacific University

10. Application: Automobile navigation Lane departure warning Pedestrian detection Mobileye (see mobileye.com) Other applications: intelligent cruise control, lane change assist, collision mitigation Systems already used in trucks and high-end cars AI@Azusa Pacific University

11. Tracking (Comaniciu and Meer)

12. Understanding Action

13. Tracking and Understanding (www.brickstream.com)

14. Tracking

18. Part I: The Physics of Imaging F How images are formed F Cameras F What a camera does F How to tell where the camera was (pose) F Light F How to measure light F What light does at surfaces F How the brightness values we see in cameras are determined F How images are formed F Cameras F What a camera does F How to tell where the camera was (pose) F Light F How to measure light F What light does at surfaces F How the brightness values we see in cameras are determined AI@Azusa Pacific University

19. Part II: Early Vision in One Image F Representing local properties of the image F For three reasons F Sharp changes are important in practice -- find “edges” F We wish to establish correspondence between points in different images, so we need to describe the neighborhood of the points F Representing texture by giving some statistics of the different kinds of small patch present in the texture. F Tigers have lots of bars, few spots F Leopards are the other way F Representing local properties of the image F For three reasons F Sharp changes are important in practice -- find “edges” F We wish to establish correspondence between points in different images, so we need to describe the neighborhood of the points F Representing texture by giving some statistics of the different kinds of small patch present in the texture. F Tigers have lots of bars, few spots F Leopards are the other way AI@Azusa Pacific University

20. Part III: Vision in Multiple Images F The geometry of multiple views F Where could it appear in camera 2 (3, etc.) given it was here in 1? F Stereopsis F What we know about the world from having 2 eyes F Structure from motion F What we know about the world from having many eyes F or, more commonly, our eyes moving. F Correspondence F Which points in the images are projections of the same 3D point? F Solve for positions of all cameras and points. F The geometry of multiple views F Where could it appear in camera 2 (3, etc.) given it was here in 1? F Stereopsis F What we know about the world from having 2 eyes F Structure from motion F What we know about the world from having many eyes F or, more commonly, our eyes moving. F Correspondence F Which points in the images are projections of the same 3D point? F Solve for positions of all cameras and points. AI@Azusa Pacific University

21. Part IV: High Level Vision F Model based vision F find the position and orientation of known objects F Using classifiers and probability to recognize objects F Templates and classifiers F how to find objects that look the same from view to view with a classifier F Relations F break up objects into big, simple parts, find the parts with a classifier, and then reason about the relationships between the parts to find the object F Model based vision F find the position and orientation of known objects F Using classifiers and probability to recognize objects F Templates and classifiers F how to find objects that look the same from view to view with a classifier F Relations F break up objects into big, simple parts, find the parts with a classifier, and then reason about the relationships between the parts to find the object AI@Azusa Pacific University

22. http://www.ri.cmu.edu/projects/project_271.html AI@Azusa Pacific University

23. http://www.ri.cmu.edu/projects/project_320.html AI@Azusa Pacific University

24. Object and Scene Recognition F Definition: Identify objects or scenes and determine their pose and model parameters F Applications F Industrial automation and inspection F Mobile robots, toys, user interfaces F Location recognition F Digital camera panoramas F 3D scene modeling F Definition: Identify objects or scenes and determine their pose and model parameters F Applications F Industrial automation and inspection F Mobile robots, toys, user interfaces F Location recognition F Digital camera panoramas F 3D scene modeling AI@Azusa Pacific University

25. Invariant Local Features F Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters SIFT Features AI@Azusa Pacific University

26. Examples of view interpolation AI@Azusa Pacific University

27. Recognition using View Interpolation AI@Azusa Pacific University