Download presentation
Presentation is loading. Please wait.
1
Image Stitching Computer Vision CS 678
Some slides from Richard Szeliski
2
Panoramic Image Mosaics
Full screen panoramas (cubic): Mars: 2003 New Years Eve: Richard Szeliski Image Stitching
3
Gigapixel panoramas & images
Mapping / Tourism / WWT Medical Imaging Richard Szeliski Image Stitching
4
Image Mosaics … + = Goal: Stitch together several images into a seamless composite Richard Szeliski Image Stitching
5
Today’s lecture Image alignment and stitching motion models
image warping point-based alignment complete mosaics (global alignment) Richard Szeliski Image Stitching
6
Motion models
7
Motion models What happens when we take two images with a camera and try to align them? translation? rotation? scale? affine? Perspective? Richard Szeliski Image Stitching
8
Image Warping
9
Image Warping image filtering: change range of image g(x) = h(f(x))
image warping: change domain of image g(x) = f(h(x)) f x f x h f x f x h Richard Szeliski Image Stitching
10
Image Warping image filtering: change range of image g(x) = h(f(x))
image warping: change domain of image g(x) = f(h(x)) f g h f g h Richard Szeliski Image Stitching
11
Parametric (global) warping
Examples of parametric warps: aspect translation rotation perspective affine cylindrical Richard Szeliski Image Stitching
12
2D coordinate transformations
translation: x’ = x + t x = (x,y) rotation: x’ = R x + t similarity: x’ = s R x + t affine: x’ = A x + t perspective: x’ H x x = (x,y,1) (x is a homogeneous coordinate) These all form a nested group (closed w/ inv.) Richard Szeliski Image Stitching
13
Image Warping Given a coordinate transform x’ = h(x) and a source image f(x), how do we compute a transformed image g(x’) = f(h(x))? h(x) x x’ f(x) g(x’) Richard Szeliski Image Stitching
14
Forward Warping Send each pixel f(x) to its corresponding location x’ = h(x) in g(x’) What if pixel lands “between” two pixels? h(x) x x’ f(x) g(x’) Richard Szeliski Image Stitching
15
Forward Warping Send each pixel f(x) to its corresponding location x’ = h(x) in g(x’) What if pixel lands “between” two pixels? Answer: add “contribution” to several pixels, normalize later (splatting) h(x) x x’ f(x) g(x’) Richard Szeliski Image Stitching
16
Inverse Warping Get each pixel g(x’) from its corresponding location x’ = h(x) in f(x) What if pixel comes from “between” two pixels? h(x) x x’ f(x) g(x’) Richard Szeliski Image Stitching
17
Inverse Warping Get each pixel g(x’) from its corresponding location x’ = h(x) in f(x) What if pixel comes from “between” two pixels? Answer: resample color value from interpolated (prefiltered) source image x x’ f(x) g(x’) Richard Szeliski Image Stitching
18
Interpolation Possible interpolation filters:
nearest neighbor bilinear bicubic (interpolating) sinc / FIR Needed to prevent “jaggies” and “texture crawl” Richard Szeliski Image Stitching
19
Motion models (reprise)
20
Motion models Translation 2 unknowns Affine 6 unknowns Perspective
3D rotation 3 unknowns Richard Szeliski Image Stitching
21
Plane perspective mosaics
8-parameter generalization of affine motion works for pure rotation or planar surfaces Limitations: local minima slow convergence difficult to control interactively Richard Szeliski Image Stitching
22
Image warping with homographies
image plane in front image plane below Richard Szeliski Image Stitching
23
Image Mosaics (Stitching)
[Szeliski & Shum, SIGGRAPH’97] [Szeliski, FnT CVCG, 2006]
24
Image Mosaics (stitching)
Blend together several overlapping images into one seamless mosaic (composite) … + = Richard Szeliski Image Stitching
25
Mosaics for Video Coding
Convert masked images into a background sprite for content-based coding = Richard Szeliski Image Stitching
26
Establishing correspondences
Direct method: Use generalization of affine motion model [Szeliski & Shum ’97] Feature-based method Extract features, match, find consisten inliers [Lowe ICCV’99; Schmid ICCV’98, Brown&Lowe ICCV’2003] Compute R from correspondences (absolute orientation) Richard Szeliski Image Stitching
27
Panoramas What if you want a 360 field of view?
mosaic Projection Cylinder Richard Szeliski Image Stitching
28
Cylindrical panoramas
Steps Reproject each image onto a cylinder Blend Output the resulting mosaic Richard Szeliski Image Stitching
29
Cylindrical Panoramas
Map image to cylindrical or spherical coordinates need known focal length Image 384x300 f = 180 (pixels) f = 280 f = 380 Therefore, it is desirable if the global motion model we need to recover is translation, which has only two parameters. It turns out that for a pure panning motion, if we convert two images to their cylindrical maps with known focal length, the relationship between them can be represented by a translation. Here is an example of how cylindrical maps are constructed with differently with f’s. Note how straight lines are curved. Similarly, we can map an image to its longitude/latitude spherical coordinates as well if f is given. Richard Szeliski Image Stitching
30
Image Stitching Align the images over each other
camera pan ↔ translation on cylinder Blend the images together Richard Szeliski Image Stitching
31
image stitching Take pictures on a tripod (or handheld)
Warp images to spherical coordinates Extract features Align neighboring pairs using RANSAC Write out list of neighboring translations Correct for drift Read in warped images and blend them Crop the result and import into a viewer Richard Szeliski Image Stitching
32
What do we do about the “bad” matches?
Matching features What do we do about the “bad” matches? Richard Szeliski Image Stitching
33
RAndom SAmple Consensus
Select one match, count inliers Richard Szeliski Image Stitching
34
RAndom SAmple Consensus
Select one match, count inliers Richard Szeliski Image Stitching
35
Find “average” translation vector
Least squares fit Find “average” translation vector Richard Szeliski Image Stitching
36
Assembling the panorama
Stitch pairs together, blend, then crop Richard Szeliski Image Stitching
37
Problem: Drift Error accumulation
small (vertical) errors accumulate over time apply correction so that sum = 0 (for 360° pan.) Richard Szeliski Image Stitching
38
Full-view (360° spherical) panoramas
39
Full-view Panorama + + + + Richard Szeliski Image Stitching
40
Texture Mapped Model Richard Szeliski Image Stitching
41
Global alignment Register all pairwise overlapping images
Use a 3D rotation model (one R per image) Use direct alignment (patch centers) or feature based Infer overlaps based on previous matches (incremental) Optionally discover which images overlap other images using feature selection (RANSAC) Richard Szeliski Image Stitching
42
Image Blending
43
Image feathering Weight each image proportional to its distance from the edge (distance map [Danielsson, CVGIP 1980] 1. Generate weight map for each image 2. Sum up all of the weights and divide by sum: weights sum up to 1: wi’ = wi / ( ∑i wi) Richard Szeliski Image Stitching
44
Image Feathering Richard Szeliski Image Stitching
45
Feathering 1 + = Richard Szeliski Image Stitching
46
Effect of window size 1 left 1 right Richard Szeliski Image Stitching
47
Effect of window size 1 1 Richard Szeliski Image Stitching
48
Good window size “Optimal” window: smooth but not ghosted
1 “Optimal” window: smooth but not ghosted Doesn’t always work... Richard Szeliski Image Stitching
49
Pyramid Blending Burt, P. J. and Adelson, E. H., A multiresolution spline with applications to image mosaics, ACM Transactions on Graphics, 42(4), October 1983, Richard Szeliski Image Stitching
50
Laplacian level 4 Laplacian level 2 Laplacian level left pyramid
Richard Szeliski Image Stitching left pyramid right pyramid blended pyramid
51
Laplacian image blend Compute Laplacian pyramid
Compute Gaussian pyramid on weight image (can put this in A channel) Blend Laplacians using Gaussian blurred weights Reconstruct the final image Q: How do we compute the original weights? A: For horizontal panorama, use mid-lines Q: How about for a general “3D” panorama? Richard Szeliski Image Stitching
52
Poisson Image Editing Blend the gradients of the two images, then integrate For more info: Perez et al, SIGGRAPH 2003 Richard Szeliski Image Stitching
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.