Sampling and Reconstruction The sampling and reconstruction process Real world: continuous Digital world: discrete Basic signal processing Fourier transforms The convolution theorem The sampling theorem Aliasing and antialiasing Uniform supersampling Nonuniform supersampling University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Camera Simulation Sensor response Lens Shutter Scene radiance University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Imagers = Signal Sampling All imagers convert a continuous image to a discrete sampled image by integrating over the active “area” of a sensor. Examples: Retina: photoreceptors CCD array Virtual CG cameras do not integrate, they simply sample radiance along rays … University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Displays = Signal Reconstruction All physical displays recreate a continuous image from a discrete sampled image by using a finite sized source of light for each pixel. Examples: DACs: sample and hold Cathode ray tube: phosphor spot and grid DAC CRT University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Sampling in Computer Graphics Artifacts due to sampling - Aliasing Jaggies Moire Flickering small objects Sparkling highlights Temporal strobing Preventing these artifacts - Antialiasing University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Jaggies Retort sequence by Don Mitchell Staircase pattern or jaggies University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Basic Signal Processing

Fourier Transforms Spectral representation treats the function as a weighted sum of sines and cosines Each function has two representations Spatial domain - normal representation Frequency domain - spectral representation The Fourier transform converts between the spatial and frequency domain Spatial Domain Frequency Domain University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Spatial and Frequency Domain Spatial Domain Frequency Domain Pictures of Fourier transforms Square Rotated square Smaller and larger squares Get coordinates of frequency space right Check out optical transforms book University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Convolution Definition Convolution Theorem: Multiplication in the frequency domain is equivalent to convolution in the space domain. Symmetric Theorem: Multiplication in the space domain is equivalent to convolution in the frequency domain. University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

The Sampling Theorem

Sampling: Spatial Domain What is the spacing? Is it unit spacing or is it T University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Sampling: Frequency Domain Magic fact 1: Fourier transform of a sequence of pulses is a sequence of pulses University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Reconstruction: Frequency Domain Overlay box University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Reconstruction: Spatial Domain Magic fact 2: The Fourier transform of a box is a sinc Step by step Long discussion about interpolating functions and properties of the sinc University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Sampling and Reconstruction If the reconstruction is not perfect, can get artifacts. Thus, even though no aliasing, this is often referred to as post-aliasing. University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Sampling Theorem This result if known as the Sampling Theorem and is due to Claude Shannon who first discovered it in 1949 A signal can be reconstructed from its samples without loss of information, if the original signal has no frequencies above 1/2 the Sampling frequency For a given bandlimited function, the rate at which it must be sampled is called the Nyquist Frequency University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Aliasing

Undersampling: Aliasing University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Sampling a “Zone Plate” y Zone plate: Sampled at 128x128 Reconstructed to 512x512 Using a 30-wide Kaiser windowed sinc x Interesting question that confused me. Does the zone pattern plate repeat? I was confused and thought I was in the Fourier domain. And hence I thought the zone plate would repeat in the spatial domain. Does it In fact repeat? Can you prove it? Someone answered that the reason why was that the frequency was coupled to position. Left rings: part of signal Right rings: prealiasing University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Ideal Reconstruction Ideally, use a perfect low-pass filter - the sinc function - to bandlimit the sampled signal and thus remove all copies of the spectra introduced by sampling Unfortunately, The sinc has infinite extent and we must use simpler filters with finite extents. Physical processes in particular do not reconstruct with sincs The sinc may introduce ringing which are perceptually objectionable Properties of the display and visual system ClearType example of designing the filter assuming properties of the display University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Sampling a “Zone Plate” y Zone plate: Sampled at 128x128 Reconstructed to 512x512 Using optimal cubic x Reconstructed using Mitchell’s cubic filter (1/3,1/3) Left rings: part of signal Right rings: prealiasing Middle rings: postaliasing University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Mitchell Cubic Filter Properties: From Mitchell and Netravali (1,0) cubic b-spline (0,C) cardinal cubics; (0,.5) catmull-rom (B,0) duff’s tensioned b-splines From Mitchell and Netravali University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Antialiasing

Antialiasing by Prefiltering Need to explain why this is a good idea. Aliases are annoying: They move around depending on the sampling rate The are very noticable because they show up at all frequencies The beat against themselves to form patterns like Moires Boxes in frequency space Boxes in the spatial domain And vice versa Asymmetry is confusing. Color code diagrams showing whether they are in the spatial domain or frequency space Frequency Space University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Antialiasing Antialiasing = Preventing aliasing Analytically prefilter the signal Solvable for points, lines and polygons Not solvable in general e.g. procedurally defined images Uniform supersampling and resample Nonuniform or stochastic sampling Only for polygons with constant or intensity varying as a polynomial Not solvable for perspective-correct polygons, or texture-mapped polygons, … Complex when geometry intersects within a pixel University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Uniform Supersampling Increasing the sampling rate moves each copy of the spectra further apart, potentially reducing the overlap and thus aliasing Resulting samples must be resampled (filtered) to image sampling rate NVIDIA shows in 2D frequency space Samples Pixel University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Point vs. Supersampled Point 4x4 Supersampled Checkerboard sequence by Tom Duff University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Analytic vs. Supersampled How to compute the exact area. Compute the intersection of the polygon with the area corresponding. Exact Area 4x4 Supersampled University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Distribution of Extrafoveal Cones Two properties The energy should be evenly distributed and not concentrated into spikes The spectrum should be deficient in low energt This is called blue noise. The maximal response to noise is about 4.5 cycles per degree [Mitchell 87] Monkey eye cone distribution Fourier transform Yellot theory Aliases replaced by noise Visual system less sensitive to high freq noise University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Non-uniform Sampling Intuition Uniform sampling Non-uniform sampling The spectrum of uniformly spaced samples is also a set of uniformly spaced spikes Multiplying the signal by the sampling pattern corresponds to placing a copy of the spectrum at each spike (in freq. space) Aliases are coherent, and very noticable Non-uniform sampling Samples at non-uniform locations have a different spectrum; a single spike plus noise Sampling a signal in this way converts aliases into broadband noise Noise is incoherent, and much less objectionable University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Jittered Sampling Add uniform random jitter to each sample University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Jittered vs. Uniform Supersampling 4x4 Jittered Sampling 4x4 Uniform University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Analysis of Jitter Non-uniform sampling Jittered sampling University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell

Poisson Disk Sampling Dart throwing algorithm Very expensive Mitchell87 and lrt describe a better algorithm. Poisson disk version of the checkerboard Also, the reason this gets pushed out is that there is more white space in the jittered pattern. This means that the effective sampling rate is actually less because of this extra empty space. The Fourier transform of an irregularly space set of pulses is not necessarily a set of pulses. Can efficiently compute Fourier transform of a set of pulses. Have not described explicitly. That we are multiplying the samples times the image and this corresponds To convolving the FT of the image with the spectral. Dart throwing algorithm University of Texas at Austin CS395T - Advanced Image Synthesis Fall 2007 Don Fussell