Computer vision

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids DFT - Discrete Fourier transform

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids DFT - Discrete Fourier transform

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Why? Objects in the world appear in different ways depending on the scale of observation Introduction “real-world objects are composed of different structures at different scales”

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Introduction How to best describe a tree? scaleMost informative description ~10 -9 [m]Molecules comprising the tree ~10 1 [m]Leaves, branches ~10 3 [m]Surrounding forest

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation The need for multi-scale representation arises when designing methods for automatically analyzing and deriving information from real-world measurements The form of description may be strongly dependent upon the scales at which the world is modelled (in clear contrast to mathematical concepts, such as 'point' and 'line', which are independent of the scale of observation) Introduction

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Applications Applications Multi scale representation is used for… Compression De-noising Multi-scale pattern matching Image stitching Segmentation etc…

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Approaches Multi scale approaches Pyramids Scale space Wavelets Multi-grid Approaches

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Scale space Scale space Handling image structures at different scales, by representing an image as a one-parameter family of smoothed images Scale-space representation is parameterized by the size of the smoothing kernel used for suppressing fine-scale structures

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Scale space Scale space Scale space representation can be generally described as: A parameterized transformation T Is the original signal/image

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Scale space Scale space The main type of scale space is the linear (Gaussian) scale space Convolution kernel (Gaussian In this case) Is the original signal/image

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Scale space Scale space

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Scale space Scale space Question: could any lowpass filter g with a parameter t can be used to generate a scale space? Answer: No! the smoothing filter must not introduce new spurious structures at coarse scales that do not correspond to simplifications of corresponding structures at finer scales “New structures must not be created when going from a fine scale to any coarser scale”

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Scale space Scale space Question: How to choose a particular type of scale-space representation? Answer: Establish a set of scale-space axioms. The axioms narrow the possible candidates class

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Scale space Scale space Scale space axioms for the linear scale-space representation: linearity shift invariance rotational symmetry positivity etc... Gaussian scale space is considered the canonical way to generate a linear scale space since it fulfills all scale-space axioms

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Pyramids Multi scale approaches Pyramids Approaches “A signal or an image is subject to repeated smoothing and subsampling” Hierarchical representation of an image

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Lowpass Pyramids Lowpass pyramids vs. Bandpass pyramids Lowpass pyramid:: generated by repeatedly smoothing the image with an appropriate smoothing filter and then subsampling the smoothed image. As this process proceeds, the result will be a set of gradually more smoothed images, where in addition the spatial sampling density decreases level by level. If illustrated graphically, this multi- scale representation will look like a pyramid, from which the name has been obtained Filter Down- sample Filter Down- sample Pyramids

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Lowpass pyramid:: generated by repeatedly smoothing the image with an appropriate smoothing filter and then subsampling the smoothed image. As this process proceeds, the result will be a set of gradually more smoothed images, where in addition the spatial sampling density decreases level by level. If illustrated graphically, this multi- scale representation will look like a pyramid, from which the name has been obtained Pyramids When the filter assigns Gaussian weights we call this a Gaussian pyramid Lowpass pyramids vs. Bandpass pyramids Multi scale signal representation:: Lowpass Pyramids

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Bandpass pyramid:: obtained by forming the difference between adjacent levels in a pyramid, where in addition some kind of interpolation is performed between representations at adjacent levels of resolution, to enable the computation of pixelwise differences Pyramids Lowpass pyramids vs. Bandpass pyramids Multi scale signal representation:: Bandpass Pyramids

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Laplacian pyramid:: Should be called the ‘difference of Gaussians pyramid’. Is given roughly by smoothing with two Gaussians of different sizes, then subtracting and subsampling Pyramids The Laplacian Pyramid is computed from the Gaussian Pyramid Gaussian mask Multi scale signal representation:: Bandpass Pyramids

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids Us- sample Gaussian pyramid Laplacian pyramid Multi scale signal representation:: Bandpass Pyramids Down- sample W W Us- sample

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids Multi scale signal representation:: Pyramid frequencies frequency 7 Gauusian frequencies

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids Multi scale signal representation:: Pyramid frequencies frequency Gauusian frequencies

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids Multi scale signal representation:: Pyramid frequencies frequency Gauusian frequencies

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Pyramids Multi scale signal representation:: Pyramid frequencies frequency Laplacian frequencies

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Applications Applications Example: multi-resolution splining A common technical problem in combining images is the visible edge between them, even slight differences in image gray level across the boundary can make that boundary quite noticeable. Thus, a technique is required which will modify image color values near the boundary to obtain a smooth transition between the combined images

Laplacian Pyramid

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Applications Applications Example: multi-resolution splining Multi-resolution spline technique is used for combining two or more images, where such images should first be decomposed into a set of bandpass component using Gaussian and Laplacian pyramid Suppose we wish to spline the left half of image A with the right half of image B

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Applications Applications Example: multi-resolution splining Algorithm 1.Create Laplacian pyramids, LA and LB, for images A and B 2.A third Laplacian pyramid LS is constructed from LA and LB 3.To obtain the splined image expand and sum the levels of LS (d) P. J. Burt and E. H. Adelson

Applications and Algorithms in CV Tutorial 3: Multi scale signal representation Multi scale signal representation:: Applications Applications Example: more complex splining