User Interface Engineering Exercise Class 2 Camera Calibration, Undistortion, and Image Dilation. Instructor: Otmar Hilliges TA’s: Tobias Nageli, Liu Zhiyong, Karthik Sheshadri Broken down table process into 4 hws, this deals with image formation. Allows obtain clean signal.

Overview Perspective projection based on the pinhole model Camera Calibration Lens Distortion and Correction Morphological Operations (Erosion & Dilation)

Image Formation Basic image model, pinhole camera, simplify the problem into two rays at extremes.

Perspective Projection: The Pinhole Model The image plane π in an actual camera is at a distance of f behind the center of projection and the projected image is inverted. However we can assume that the image plane is in front of the center of projection to avoid this inversion he line through the center of projection (i.e., optical center) c o and perpendicular to the image plane π is the optical axis. The 3D point c x is projected onto n x through the optical center c (Perspective Division)

Image Formation: Optics can cause issues

Image Formation 1 𝑓 = 1 𝑧 + 1 𝑒

Image Formation

Lens Distortion Assume Staright lines in real world project to straight lines in image. Most lenses radial distortion, manifests as a curvature in the projection of straight lines.

Perspective Projection: The Pinhole Model The Pinhole Model equations in homogeneous co-ordinates Say that this eqn is a projection from the 3D world cords to image plane and uses the focal length of the camera .

Perspective Projection: The Pinhole Model The displacements from the optical center need to be put in pixel co-ordinates.

Perspective Projection: The Pinhole Model Xp -> normalized 0 to 1 space

Perspective Projection: The Pinhole Model (Camera Intrinsics)

Perspective Projection: The Pinhole Model (Camera Extrinsics)

Perspective Projection: The Pinhole Model

Perspective Projection: The Pinhole Model

Lens Distortion

Lens Distortion 𝑢 𝑑 𝑣 𝑑 = 1+ 𝑘 1 𝑟 2 𝑢− 𝑢 0 𝑣− 𝑣 0 + 𝑢 0 𝑣 0 , 𝑢 𝑑 𝑣 𝑑 = 1+ 𝑘 1 𝑟 2 𝑢− 𝑢 0 𝑣− 𝑣 0 + 𝑢 0 𝑣 0 , With 𝑟 2 = 𝑢− 𝑢 0 2 + 𝑣− 𝑣 0 2

OpenCV Tutorial Online tutorial: http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/ca mera_calibration.html

Calibration procedure Practical tips and tricks Place your camera such that most of the image is covered by the checkerboard Use varying orientations and distances for the views Screenshots

Deliverables Send your calibration intrinsics Send in the estimated distortion coefficients Record a video of your undistorted images http://ait.ethz.ch/teaching/courses/2013-FS-User-Interface- Engineering/

Morphological Operations Size of s determines extent of ero/dil . Ero -> keep

Morphological Operations Absolute Image Differences for Motion Segmentation

Morphological Operations

Morphological Operations