1. A Brief Overview of Computer Vision Jinxiang Chai

2. What is Computer Vision? Computer vision is the science and technology of machines that see. Concerned with the theory for building artificial systems that obtain information from images. The image data can take many forms, such as a video sequence, views from multiple cameras, or multi-dimensional data from a medical scanner

3. Applications Robot perception (e.g. an industrial robot or an autonomous vehicle, autonomous helicopter, humanoid robots).

4. Honda ASIMO Humanoid Robot Face detection Face recognition Posture/gesture recognition (e.g., hand waving) Environment recognition (e.g., obstacles)

5. Applications Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). Detecting events (e.g. for visual surveillance or people counting).

6. Detecting Events Customer tracking and activity analysis

7. Applications Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). Detecting events (e.g. for visual surveillance or people counting). Modeling objects or environments

8. Modeling buildings, plants, faces, cars etc.

9. Applications Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). Detecting events (e.g. for visual surveillance or people counting). Modeling objects or environments Interaction (e.g. as the input to a device for computer- human interaction).

10. Interactions Interactions with computers and video games, etc. Face recognition for automatic login Computer vision for game interfaces (Sony eyetoy, Microsoft Kinect)

11. Applications Robot perception (e.g. an industrial robot or an autonomous vehicle, humanoid robots). Detecting events (e.g. for visual surveillance or people counting). Modeling objects or environments Interaction (e.g. as the input to a device for computer- human interaction).computer- human interaction Organizing information (e.g. for indexing databases of images and image sequences).

12. Organizing information Flickr (www. Flickr.com) has 3 billion images Youtube has tons of videos. Need new ways to search, analyze, summarize a large collection of internet images and videos

13. Image Representation An image is a 2D rectilinear array of Pixels - A width X height array where each entry of the array stores a single pixel

14. Image Representation A pixel stores color information Luminance pixels - gray-scale images (intensity images) - 0-255 - 8 bits per pixel Red, green, blue pixels (RGB) - Color images - Each channel: 0-255 - 24 bits per pixel

15. Image Representation An image is a 2D rectilinear array of Pixels - A width X height array where each entry of the array stores a single pixel - Each pixel stores color information (255,255,255)

16. Images Which kind of information you can obtain from images

17. Images Which kind of information you can obtain from images Edge detection

18. Images Which kind of information you can obtain from images Edge detectionCorner& feature detection

19. Images Which kind of information you can obtain from images Edge detectionCorner& feature detection Geometric primitive detection

20. Images Which kind of information you can obtain from images Edge detectionCorner& feature detection Geometric primitive detection Object detection

21. Images Which kind of information you can obtain from images Edge detectionCorner& feature detection Geometric primitive detection Object detection Face alignment and recognition ……

22. How about multiple images? What can we obtain if we have multiple images?

23. How about multiple images? What can we obtain if we have multiple images? Two images of the same scene

24. Structure and motion analysis Given two or more images of the same scene or object, estimate camera motion and 3D object structure (e.g., depth) unknowncameraviewpoints

25. Structure and motion analysis Given two or more images of the same scene or object, estimate camera motion and 3D object structure (e.g., depth) unknowncameraviewpoints How to estimate camera parameters? - where is the camera? - where is it pointing? - what are internal parameters, e.g. focal length?

26. Structure and motion analysis Given two or more images of the same scene or object, estimate camera motion and 3D object structure (e.g., depth) unknowncameraviewpoints How to estimate camera parameters? - where is the camera? - where is it pointing? - what are internal parameters, e.g. focal length? Camera calibration!

27. Structure and motion analysis Reconstruct the depth information. Input images How to find the depth information of this point?

28. Structure and motion analysis Reconstruct the depth information. Input images How to find the depth information of this point? - find the corresponding point in the right image.

29. Structure and motion analysis Reconstruct the depth information. Input images

30. Structure and motion analysis Reconstruct the depth information. Input images Depth images

31. Structure and motion analysis Reconstruct 3D models from multiple images Reconstruction results from 23 images

32. All together video Click herehere - feature detection - feature matching (epipolar geometry) - structure from motion - stereo reconstruction - triangulation - texture mapping

33. How about video sequences? What can we obtain from video?

34. How about video sequences? What can we obtain from video? Optical flow: where are pixels moving to?

35. How about multiple video sequences Modeling dynamic objects (video click here) here

36. Modeling human motion from video Single-view camera Interactively construct human motion form video