ELE 488 Fall 2006 Image Processing and Transmission (10-17-06) 10/17/06 ELE 488 Fall 2006 Image Processing and Transmission (10-17-06) Geometric Transformation translation rotation scaling Affine Image Registration Goodness of fit There are many other examples not touched here. But with these examples, we can say something about why image processing. We saw some examples of enhancement, which makes the image looks better, or show more details. We saw some examples of extracting information from the images, both still images and video. We saw the same image have vastly different file sizes. This has to do with image coding and data compression. We also saw video encoding can be used to make it less vulnerable to transmission error. Image is information. How to store this kind of information, distribute it are important issued. A related topic is the digital library, which presents many challenges. Book, index, search, etc.

Mosaicking and Registration Reference Image Floating Image

Identify Features and Match to Determine Geometric Transformation Image I (reference) h(x,y) Image II (floating) h–1(x’,y’) Two cartesian coordinate systems: (x,y) and (x’,y’) Forward: (x’,y’) = h(x,y) Reverse: (x,y) = h-1(x’,y’)

Four Basic Transformations translation rotation (1, 1) y x scaling shearing (1, 0) y’ x’ (2, 1) (3, 1)

Combining Transformations Rotation and translation Successive application of rotation, translation and scaling. Involves matrix multiplications and additions Homogeneous Coordinates Matrix multiplication, no addition

Translation – Rotation – Scaling Note: c=0

Combining Transformations 1) Translate 2) Rotate 3) Translate

Homogeneous Transforms Rigid-body: Rotation and Translation 2x2 rotation submatrix is orthonormal Translation/Rotation/Scaling are determined by rotation and scaling parameters

Affine and Projective Transformations Affine: 6 free parameters Projective: 8 free parameters More general geometric transformations between two planes Widely used in modeling cameras s scaling factor.

Inverse Transform Examples:

Reflections Reflect in x-axis Reflect in y-axis Reflect in line y=x

Reflect about a general line y=ax+b

Pinhole Camera Model

Pinhole Camera Model (shifted image plane)

Image Formation: pin hole model Projection of point (X,Y,Z) in 3D space to (x,y) in image plane

Mosaicking and Registration Reference Image Floating Image Two cameras Or one camera at two positions

Two Cameras Point in 3-D has two sets of coordinates

Rotation of Camera About One Axis

General Rotation of Camera Two 3-D coordinates of a point are related by a rotation matrix: rotation matrix R Decompose R into 3 rotations about the first camera axes: Counter clockwise rotation about Z axis Counter clockwise rotation about X axis Counter clockwise rotation about Y axis

Translation location of 2nd camera in 1st camera coordinates

3-D Coordinate Transformation The two 3-D coordinates of a point are related by a 3-D rotation and a translation: Counter clockwise rotation about Z axis Counter clockwise rotation about X axis Counter clockwise rotation about Y axis from pinhole camera

3-D Coordinate Transformation

3-D Coordinate Transformation Independently scale each side of the equation

No Translation Homogeneous transformation Knowing establishes the correspondence between the images.

Examples

Flat Scene Knowing establishes the correspondence between the images. See slide 9 Homogeneous transformation Knowing establishes the correspondence between the images.

Planar Scene Homogeneous transformation Knowing establishes the correspondence between the images.

Mosaic: Firestone Library Photos: P. Ramadge, Mosaic: Y.P. Tan/R. Radke The zero translation approximation was used to register these images

Affine and Projective Transformations Projective Transformations are closely connected to image formation models and camera rotation Affine is an approximation:

Small Rotations Can be approximated by affine

Three Questions of Image Registration How is the spatial transformation (= warping function) applied and what family of warping functions are allowed? How do we measure how well the warped floating image aligns with the reference image? How to we find the best warp in the allowed class to best match the images (optimization)

Warping Function Examples Translation Rotation Scaling Affine Projective

Goodness of Fit: How well does warped image align with reference 1) Use all pixel values: 2) Only use pixel values around selected “features”: x y What are features?

Features and Feature Selection Sharp  corners v u GOOD BAD v u

Registration As An Optimization Problem Consider transformations parameterized by a vector of parameters θ Find the transformation that minimizes the registration metric (e.g., above) In some special cases this can be solved analytically (in closed form) In other cases it can be approximated and solved analytically In the most complex cases it has to be solved iteratively by a gradient-like descent method.

Example: Translation + SSD Feature Metric 1 feature - can do this by brute force search by finding the best match More features?

Simplification Find the best translation for each feature individually by a brute force search (can be refined) This gives (called a correspondence) Then select the translation for the entire image to minimize : This yields

Image Registration 756x504 pixels 16 bit color Reference Image Floating Image 756x504 pixels 16 bit color Allowed warpings is translations Matching metric is how well a selected subset of “features” match using SAD best translation match. average pixels where images overlap

Registration Using Affine Transformation translation, rotation, scaling, shearing

Registration Using Projective Transformation Add: tilting