- photometric aspects of image formation gray level images point-wise operations linear filtering MSRI workshop

Image Brightness values I(x,y) MSRI workshop

Analog intensity function Temporal/spatial sampled function Image model Mathematical tools Analog intensity function Temporal/spatial sampled function Quantization of the gray levels Point sets Random fields List of image features, regions Analysis Linear algebra Numerical methods Set theory, morphology Stochastic methods Geometry, AI, logic MSRI workshop

What gives rise to images photometric properties of the environment geometric properties of the environment MSRI workshop

Basic ingredients Radiance – amount of energy emitted along certain direction Iradiance – amount of energy received along certain direction BRDF – bidirectional reflectance distribution Lambertian surfaces – the appearance depends only on radiance, not on the viewing direction Image intensity for a Lambertian surface MSRI workshop

Challenges MSRI workshop

Computer Vision Visual Sensing Images I(x,y) – brightness patterns image appearance depends on structure of the scene material and reflectance properties of the objects position and strength of light sources MSRI workshop

Recovery of the properties of the environment from single or multiple views Vision problems Segmentation Recognition Reconstruction Vision Based Control - Action Visual Cues Stereo, motion, shading, texture, contour, brightness MSRI workshop

Segmentation – partition image into separate objects Clustering and search algorithms in the space of visual cues MSRI workshop

Recognition – given an image classify what object it represents Face recognition Digit recognition Activity recognition Patter Recognition and Machine Learning techniques MSRI workshop

Computing Stages Low-level vision (pixels, raw data) - Image processing Mid-level vision (transformations, measurements, features) High-level vision (higher level semantic entities) MSRI workshop

Computing Stages Low-level vision (pixels, raw data) - Image processing Applied operator is linear -> theory of (discrete) linear systems MSRI workshop

Discrete time system maps 1 discrete time signal to another f[x] g[x] h Special class of systems – linear , time-invariant systems Superposition principle Shift (time) invariant – shift in input causes shift in output Examples MSRI workshop

Convolution sum: unit impulse – if x = 0 it’s 1 and zero everywhere else Every discrete time signal can be written as a sum of scaled and shifted impulses The output the linear system is related to the input and the transfer function via convolution h f g filter f g Convolution sum: Notation: In matrix form: MSRI workshop

Fourier Transform and and FT phase and magnitude definition FT example gallery Central theme – approximate a function given a family of Basis functions MSRI workshop

Fourier Transform Examples FT phase and frequency information Role of the phase information MSRI workshop

Filtering - Fourier Transform Original Image – Band Pass – its FFT Original - Low Pass – its FFT Original - High Pass – its FFT MSRI workshop

Computing Derivatives MSRI workshop

Pointwise Image Operations Lookup table – match image intensity to the displayed brightness values Manipulation of the lookup table – different Visual effects – mapping is often non-linear MSRI workshop

Contrast gamma Contrast MSRI workshop Brightness

Quantization Thresholding Histogram Histogram – frequency gray-level -> empirical distribution h[i] – number of pixels of intensity i Histogram equalization – making histogram flat MSRI workshop

Representation of images at multiple levels of resolution Image Pyramids Representation of images at multiple levels of resolution Importance – at different resolutions different features look differently Over-complete representations Used for localization properties, motion computation and matching, biological motivation MSRI workshop

Pyramid construction MSRI workshop

Pyramid construction/reconstruction Take an original image – convolve with a blurring Filter and subsample to get an image at lower resolution Reduction – how is the signal at level l+1 related to level l MSRI workshop

Pyramid construction/reconstruction Expansion – how to reconstruct the signal at level l given related to level l-1 – notation Signal at level l, expanded k – times Idea – take the smaller signal fill every second entry With zero and convolve with the blurring filter MSRI workshop

“Drop” vs “Smooth and Drop” Drop every second pixel Smooth and Drop every second pixel Aliasing problems MSRI workshop

Gaussian Pyramid Consecutive smoothing and sub-sampling MSRI workshop

Laplacian Pyramid Idea when we convolve and down-sample some information will get lost i.e. we reverse the process we cannot get the original image back. Example: Blurred image of Lower resolution (original image - upsampled blurred image) they are not the same – fine details are lost MSRI workshop

Laplacian pyramid Store the fine differences which get lost Each level of the Laplacian pyramid difference between two consecutive levels of gaussian pyramids MSRI workshop

Laplacian Pyramid blurred1 image – upsampled blurred2 image original image – original image smoothed by gaussian MSRI workshop

Schematic for construction/reconstruction Image blurr/down2 Blurred1 up2/blurr add Recon up2/blurr subtract Fine1 MSRI workshop

Laplacian pyramids Laplacian pyramids – each level holds the additional information which is needed for better resolution We can obtain same image by convolving the image with difference of Gaussians filter or appropriate width Reflects the similarity between difference of Gaussians DoG and Laplacian operator introduced in the edge detection stage MSRI workshop

Application Texture Synthesis MSRI workshop