1. CS 523 (CS 423/EE 533) Computer Vision
Lecture 1
INTRODUCTION TO COMPUTER VISION

2. About the Course

3. Syllabus http://vvgl.ozyegin.edu.tr Objective
Introduction to the theory, tools, and algorithms of 3D computer vision
Instructor
Assist. Prof. M. Furkan Kıraç
Room: 219
Hours
Wednesdays, 10:40-13:30, Room: 241
Grading
Projects: 6x10%
Final Exam: 40%

4. Grading
Short Projects: Late submissions are not accepted. Copying answers from others’ work is not permitted.
Final Exam: At least 3 of the 6 Short Projects must be turned in by the due date in order to qualify for the Final Exam. No make-up will be given for the Final Exam. Students can take the Bütünleme exam if they miss the Final Exam.

5. Recommended Books
Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010.
Computer Vision: A Modern Approach, David A. Forsyth and Jean Ponce, Prentice-Hall, 2002.
Introductory Techniques for 3D Computer Vision, Emanuele Trucco and Alessandro Verri, Prentice-Hall 1998.

6. OpenCV Resources
Learning OpenCV, Gary Bradski and Adrian Kaehler, O'Reilly, 2008.
OpenCV 2 Computer Vision Application Programming Cookbook, Robert Laganière, Packt Publishing, 2011.
Mastering OpenCV with Practical Computer Vision Projects, Daniel Lélis Baggio, et al., Packt Publishing, 2012.

7. Week
Lectures
24 September 2014
Lecture 1
1 October 2014
Lecture 2
8 October 2014
Lecture 3
15 October 2014
Lecture 4
22 October 2014
Lecture 5
29 October 2014
Lecture 6
5 November 2014
Lecture 7
12 November 2014
Lecture 8
19 November 2014
Lecture 9
26 November 2014
Lecture 10
3 December 2014
Lecture 11
10 December 2014
Lecture 12
17 December 2014
Lecture 13
24 December 2014
Lecture 14
31 December 2014
Lecture 15 ?

8. Applications of Computer Vision

9. Image Stitching

10. Image Matching

11. Object Recognition

12. 3D Reconstruction

13. Interior Modeling

14. 3D Augmented Reality

15. 3D Camera Tracking

16. Stereo Conversion for 3DTV

17. Depth Estimation and View Interpolation for 3DTV

18. Human Tracking

19. License Plate Recognition

20. Human Pose Estimation

21. Course Outline

22. Topics to be covered 3D geometry fundamentals
Transformations and projections
Camera calibration
Feature detection and matching
Image stitching
Single view geometry
Two view geometry
Multiple view geometry
Stereo vision and depth estimation
3D structure from motion
3D camera tracking

23. Relation to Other Fields

24. Computer Vision
Figure from "Computer Vision: Algorithms and Applications,” Richard Szeliski, Springer, 2010.

25. Computer Graphics Lights and materials Shading Texture mapping
Environment effects
Animation
3D scene modeling
3D character modeling
(OpenGL)

26. Computer Graphics

27. Image Processing Topics
Resampling
Enhancement
Noise filtering
Restoration
Reconstruction
Segmentation
Image compression
(MATLAB and OpenCV)

28. Image Processing

29. Video Processing Topics
Spatio-temporal sampling
Motion estimation
Frame-rate conversion
Multi-frame noise filtering
Multi-frame restoration
Super-resolution
Video compression
(MATLAB & OpenCV)

30. Video acquisition-display chain
Capture
Representation
Coding
Transmission
Decoding
Rendering

31. Human vs. Computer

32. Optical illusions

33. Actual vs. Perceived Intensity (Mach band effect)

34. Brightness Adaptation of the Eye

35. Optical illusions

37. Optical illusions

38. Why is Computer Vision Difficult?

39. Human perception

40. Human perception

41. Human Visual System

42. Human Eye

45. Photoreceptors: Rods & Cones

47. Rods vs. Cones Rods Cones Perceive brightness only Night vision
Perceive color
Day vision
Red, green, and blue cones

48. Cone Distribution
Blue is less-focused
64%
32% 2%

49. Visual Threshold drop during Dark Adaptation

50. Spatial Resolution of the Human Eye
Photopic (bright-light) vision:
Approximately 7 million cones
Concentrated around fovea
Scotopic (dim-light) vision
Approximately million rods
Distributed over retina
(HDTV: 1920x1080 = 2 million pixels)

51. Frequency Responses of Cones
Same amount of energy produces different sensations of brightness at different wavelengths
Green wavelength contributes most to the perceived brightness.

52. Trichromatic Color Mixing
Any color can be obtained by mixing three primary colors Red, Green, Blue (RGB) with the right proportion

54. Image Formation

55. Human Eye vs. Camera Camera components Eye components Lens
Lens, cornea
Shutter
Iris, pupil
Film
Retina
Cable to transfer images
Optic nerve to send the incident light information to the brain

56. Human Vision

57. Image formation

58. Pin-Hole Camera Model
Image always sharp (ideally)

59. Point Spread Effect
Point spread increases with the increase in the hole size and distance of the image plane from the hole

60. Out-of-Focus Blur

61. Shrinking the Aperture

62. Converging Lens

63. Correction with a Converging Lens
Focus blur takes place if the image plane is moved in either direction

65. Perfectly In-Focus for a Certain Distance Only
“circle of
confusion”

66. Depth-of-Field

67. Depth-of-Field

68. “Sharp Image” within Depth-of-Field due to Finite Sensor Size
Focus blur takes place if the image plane is moved in either direction

69. Focal Length (F) and Depth (Z)
Image always sharp (ideally)

70. Aperture Size Affects Depth-Of-Field

71. Aperture
f-number for “full-stop”: area changes by a factor of 2 per stop

72. Camera f-number
f-number for “full-stop”: area changes by a factor of 2 per stop

73. Exposure Time

74. Motion Blur Effect due to Finite Exposure Time
Image always sharp (ideally)

75. Decrease in aperture implies…
Increase in depth-of-field
Decrease in motion blur
Decrease in exposure

76. 2D Image Representation

77. Image Capture
(Courtesy Gonzalez & Woods)

78. Digital Image Capture

79. Digital Image Capture
Light sensitive diodes convert photons to electrons

80. Color Image Capture: Single vs. Three CCD Arrays
Bayer filter
(cheaper but introduces spatial resolution loss)
RGB splitter
(three separate imaging sensors, higher resolution)

81. Digital Camera Issues Noise Color Blooming In-camera processing
caused by low light
Color
color fringing (chromatic aberration) artifacts from Bayer patterns
Blooming
charge overflowing into neighboring pixels
In-camera processing
over-sharpening can produce halos
Compression
creates blocking artefacts

82. Digitization: Sampling and Quantization

83. Over Sampling

84. Over Quantization

85. Images as Matrices of Integers
(0,0) m
126
127
126
128
127
124
158
125
126
127
123
120
144
163
123
126
125
121
128
155
160
126
123
127
122
142
162
164
120
122
124
130
157
161
166
119
121
123
145
162
164
165
0 → black, 255 → white n
0 ≤ s(m,n) ≤ 255 } quantization
0 ≤ m ≤ M-1
0 ≤ n ≤ N-1
sampling
MxN 8-bit gray-scale (intensity, luminance) image

86. Images as Functions
We can think of an image as a function, f, from R2 to R:
f( x, y ) gives the intensity at position ( x, y )
Realistically, we expect the image only to be defined over a rectangle, with a finite range:
f: [a,b]x[c,d]  [0,1]
A color image is just three functions pasted together. We can write this as a “vector-valued” function:

87. RGB Color Bands (Channels)

88. YUV Bands Also called Y Cb Cr
Color
Also called Y Cb Cr
Y : Luma Cb : Chrominance_blue Cr : Chrominance_red Y
U (Cb) V
(Cr)

89. YUV-RGB Conversion

90. Summary

91. Summary
Human visual system
Pin-hole camera model
Image representation

92. Problems to be Addressed
How to find camera parameters?
Where is the camera, where is it directed at?
What is the movement of the camera?
Where are the objects located in 3D?
What are the dimensions of objects in 3D?
What is the 3D structure of a scene?
How to process stereo video?
How to detect and match image features?
How to stitch images?