Download presentation
Presentation is loading. Please wait.
1
Computing motion between images
The basics
2
Matching images We will deal with two similar tasks:
A. Given a point in one image, find its matching point in the other image - feature matching/optical flow. B. Find the transformation mapping one image to the other - Image alignment/ Image Registration. Feature matching/ optical flow Alignment: Scale by 3/4
3
Why is it useful Camera stabilization Tracking moving objects
Finding the camera motion and 3D reconstruction. Image processing by combining several images (mosaicing, super-resolution…) Compression. …..
4
Matching by sliding window
A basic method to find matching points between images. (correspondence) a neighborhood of the point (Window) in image I1 is compared to several Windows in I2. The window with the best score wins. Various scores can be used.
5
Matching by sliding window
The winning window The window in image 1 A sliding window in I2 I2 I2 I2
6
Matching Criteria - Difference
Common matching criteria: SSD - Sum of Squared Differences k - the size of the window. p1 = (x1, y1) is the center of the window in I1. p2 = (x2, y2) is the center of the window in I2. SAD - Sum of Absolute Differences
7
Matching Criteria - Correlation
Cross Correlation is similar to SSD, but can be implemented more efficiently: The window to search The image to search (I2) The maximum cross correlation score The cross correlation score
8
Handling illumination changes
SSD, SAD and Cross Correlation assume constant brightness. On order to handle illumination changes, Normalized cross-correlation can be used. I1 I2
9
Normalized Cross Correlation
When ordering the pixels in the windows in vectors v1, v2: The Normalized Cross Correlation is: Squared Vector norm Inner product
10
The aperture problem ? ? ?
11
The aperture problem (cont’)
Easy to track in both directions. Hard to track vertically.
12
The aperture problem (cont’)
T junction normal flow real motion
13
And what about smooth areas ?
Use bigger windows: Less noise Smaller aperture problem Might include different motions Smooth the image ! ? ?
14
Using a Pyramid for optical flow
Smoothing the images: we get an estimation of the motion in uniform regions. Handling large motions: Even if the motion in the original image is big, the motion in the small level is small. Once the solution was find in level k, there is only small motion to fix in the level k-1: The optical flow calculations become much faster ! Pyramid[1] Pyramid[2]
15
Auto-Correlation (and how does the SSD handle a combination of motions)
The AC meaning: How similar does the image looks to it selves in different shifts. It has a strong connection with the aperture problem. The SSD should find the best motion – depending on the auto-correlation function.
16
Deformations The window matching assumes a pure image translation in small regions. Possible solutions for deformations: Invariant features Iterations of motion computations + warping
17
Limitations of Window Matching
Accuracy: A pixel is always matched to integer location on the grid. The image motion is usually not integer. Neighborhood/Scene constraints: High level knowledge about the scene/camera may help in limiting the search, and reducing errors. (for example, the scene is planar)
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.