1. Static Image Mosaicing
Amin Charaniya
EE 264: Image Processing and Reconstruction

2. Presentation Overview
Problem definition
Background
Literature Survey
Image transformations
Image Registration
Coarse Image registration
Transformation Optimization
Image Blending
Implementation and Results
Conclusions (limitations and enhancements)

3. The Problem Q: “Static” ? Image 1 Image 2 +
Mosaiced image
Q: “Static” ?
Ans.: No moving objects in the scene.

4. The Solution Original images Image Registration / Alignment / Warping
Image Blending

5. Constraints Scene Camera Motion Other Constraints Static / Dynamic
Planar / Non planar (perspective distortion)
Camera Motion
Translation (sideways motion)
Panning and Tilting (rotation about the Y and X axes)
Scaling (zooming, forward / backward motion)
General motion
Other Constraints
Automated / User input

6. Background and Literature survey
Barnea & Silverman, 1972 (L1 Norm)
Kuglin & Hines, 1975 (Phase Correlation)
Mann & Picard, 1994 (Cylindrical projection)
Irani & Anandan, 1995 (Static and Dynamic mosaics)
Szeliski, 1996 (Transformation optimization)
Badra, 1998 (Rotation and Zooming)
Peleg and Rousso, 2000 (Adaptive Manifolds, Mosaicing using strips)

7. Image transformations
Input
image
Output
Rigid transformation
Original
shape
Affine transformation
Projective transformation
Homogeneous coordinates
Polynomial transformations

8. Presentation Overview
Problem definition
Background
Literature Survey
Image transformations
Image Registration
Coarse Image registration
Transformation Optimization
Image Blending
Implementation and Results
Conclusions (limitations and enhancements)

9. { Image Registration Coarse Image Transformation Registration
Initial transformation
Transformation
Optimization
Error
Improved ? {
Phase Correlation
L1 Norm
User input

10. Phase Correlation d(x,y) (x0, y0) Kuglin & Hines, 1975
Translation property of Fourier Transform
Inverse
transform
d(x,y)
maximum
(x0, y0)

11. Spatial Correlation, L1 Norm
Barnea and Silverman f2 f2
E(x0,y0) = |f1(x,y) – f2(x- x0, y- y0)| f1
Spatial correlation techniques
User input

12. Transformation Optimization
Richard Szeliski, “Video Mosaics for Virtual Environments”, 1996.
Optimization of initial transformation matrix M, to minimize error.
Levenberg-Marquardt non-linear minimization algorithm.
minimize
Compute partial derivatives

13. Transformation Optimization
Advantages
Faster convergence
Statistically optimal solution
Limitations
Local minimization (need a good initial guess)

14. Presentation Overview
Problem definition
Background
Literature Survey
Image transformations
Image Registration
Coarse Image registration
Transformation Optimization
Image Blending
Implementation and Results
Conclusions (limitations and enhancements)

15. Image Blending Smooth transition (edges, illumination artifacts)
Simple averaging
Weighted averaging
Sample weight function – “hat filter”
xmax
More weight at the center of the image, less at the edges

16. Image blending
Simple averaging
Weighted averaging

17. Presentation Overview
Problem definition
Background
Literature Survey
Image transformations
Image Registration
Coarse Image registration
Transformation Optimization
Image Blending
Implementation and Results
Conclusions (limitations and enhancements)

18. Implementation Implemented using Matlab Source Images
BE 230 lab images (fixed tripod)
College 8 images (free hand motion, perpective distortion)
East Field House images (free hand motion)
Equipment: Sony DCR-TRV 900 3CCD digital camcorder

19. Sample results

20. Sample results

21. Conclusions/Enhancements
Better automatic coarse registration techniques needed.
Need to handle more general camera motion.

22. Thanks for listening !!
Questions ?