Image 2013, Fall

Image Processing Quantization  Uniform Quantization  Halftoning & Dithering Pixel Operation  Add random noise  Add luminance  Add contrast  Add saturation Filtering  Blur  Detected edges Warping  Scale  Rotate  Warp Combining  Composite  Morph Ref] Thomas Funkhouser, Princeton Univ.

What is an Image? An image is a 2D rectilinear array of samples A pixel is a sample, not a little square

Image Resolution Intensity resolution  Each pixel has only “Depth” bits for colors/intensities Spatial resolution  Image has only “Width” x “Height” pixels Temporal resolution  Monitor refreshes images at only “Rate” Hz Width x HeightDepthRate NTSC640x Workstation1280x Film3000 x Laser Printer6600x Typical Resolutions Height Width Depth

Source of Error Intensity quantization  Not enough intensity resolution Spatial aliasing  Not enough spatial resolution Temporal aliasing  Not enough temporal resolution Real valueSample value

Quantization Artifact due to limited intensity resolution  Frame buffers have limited number of bits per pixel  Physical devices have limited dynamic range

Uniform Quantization 반올림 (Round)

Uniform Quantization Image with decreasing bits per pixel

Reducing Effects of Quantization Our eyes tend to integrate or average the fine detail into an overall intensity Halftoning  A technique used in newspaper printing  The process of converting a continuous tone image into an image that can be printed with one color ink (grayscale) or four color inks (color) Dithering  Dithering is used to create a wide variety of patterns for use as backgrounds, fills and shadings, as well as for creating halftones, and for correcting aliasing. Halftones are created through a process called dithering  Dithering  Methods( 방법론 ), Halfton  result( 결과물 )

Classical Halftoning Use dots of varying size to represent intensities  Area of dots proportional to intensity in image

Halftoning A technique used in newspaper printing Only two intensities are possible, blob of ink and no blob of ink But, the size of the blob can be varied Also, the dither patterns of small dots can be used

Classical Halftoning Original ImageHalfton Image

Halfton patterns Use cluster of pixels to represent intensity  Trade spatial resolution for intensity resolution 2 by 2 pixel grid patterns 3 by 3 pixel grid patterns mask

Halfton patterns

Dithering Distribute errors among pixels  Exploit spatial integration in our eye  Display greater range of perceptible intensities

Halftoning – Moire Patterns Moire[more-ray] patterns  Repeated use of same dot pattern for particular shade results in repeated pattern (Perceived as a moire pattern)  result from overlapping repetitive elements  Instead, randomize halftone pattern

Halftoning – Moire Patterns

Signal Processing (1/2) Signal  Real World : Continuous Signal  Digital World: Discrete Signal Image  spatial domain not temporal domain intensity variation over space

Signal Processing (2/2) Sampling and reconstruction process  Sampling The process of selecting a finite set of values from a signal Samples (the selected values)  Reconstruction recreate the original continuous signal from the samples

Sampling and Reconstruction

Aliasing In general  Artifact due to under-sampling or poor reconstruction Specifically, in graphics  Spatial aliasing  Temporal aliasing Sampling below the Nyquist rate

Nyquist rate and aliasing Nyquist rate  lower bound on the sampling rate  Sampling at the Nyquist rate sampling rate > 2* max frequency 주파수 (frequency): 초당 사이클의 횟수 (Hz)

Spatial Aliasing Artifact due to limited spatial resolution

Spatial Aliasing Artifact due to limited spatial resolution

Temporal Aliasing Artifact due to limited temporal resolution  Spinning Backward The wheel is spinning at 5 revolution per second –3 rd row  appear to be spinning backward at revolution per second

Temporal Aliasing Artifact due to limited temporal resolution  Flickering

Anti-Aliasing Sample at higher rate  Not always possible  Doesn’t always solve problem Pre-filter to form bandlimited signal  Form bandlimited function(low-pass filter)  Trades aliasing for blurring

Pixel Operations Grayscale  Compute luminance L  L=0.30R+0.59G+0.11B (old)  R G B (CRT, HDTV) Original ImageGrayscale Image

Pixel Operations Negative Image  Simply inverse pixel components  Ex) RGB(5, 157, 178)  RGB(250, 98, 77) Original ImageNegative Image (R, G, B)  (255-R, 255-G, 255-B)

Pixel Operations Adjusting Brightness  Simply scale pixel components Must clamp to range (e.g., 0 to 255) (R, G, B)  (R, G, B) * scale

Pixel Operations Adjusting Contrast  Compute mean luminance L for all pixels Luminance = 0.30* r *g *b  Scale deviation from L for each pixel component Must clamp to range (e.g., 0 to 255) R  L + (R-L)*scale G  L + (G-L)*scale B  L + (B-L)*scale

Pixel Operations Changing Brightness Changing Contrast

Pixel Operations Adjusting Saturation  Color Convert RGB  HSV  Scale from S for each pixel component  Color Convert HSV  RGB Must clamp to range (e.g., 0 to 255)

Filtering Convolution  Spatial domain Output pixel is weighted sum of pixels in neighborhood of input image Patten of weights is the “filter”

Filtering Convolution

Filtering Adjust Blurriness  Convolve with a filter whose entries sum to one Each pixel becomes a weighted average of its neighbors Filter Sum = 1

Filtering Edge Detection  Convolve with a filter that finds differences between neighbor pixels

Filtering Sharpening  Convolve with a Sharpening filter

Filtering Embossing  Convert the image into grayscale  Convolve with a Embossing filter  Plus 0.5 for each pixel Must clamp to range (e.g., 0 to 255)

Summary Image representation  A pixel is a sample, not a little square  Images have limited resolution Halftoning and dithering  Reduce visual artifacts due to quantization  Distribute errors among pixels Exploit spatial integration in our eye Sampling and reconstruction  Reduce visual artifacts due to aliasing  Filter to avoid undersampling Blurring is better than aliasing

Image Processing Quantization  Uniform Quantization  Halftoning & Dithering Pixel Operation  Add random noise  Add luminance  Add contrast  Add saturation Filtering  Blur  Detected edges Warping  Scale  Rotate  Warp Combining  Composite  Morph Ref] Thomas Funkhouser, Princeton Univ.

Image Warping Move pixels of image  Mapping Forward Reverse  Resampling Point sampling Triangle Filter Gaussian filter

Mapping Define transformation  Describe the destination (x,y) for every location (u,v) in the source (or vice-verse, if invertible) Source (u, v)Destination (x, y)

Example Mappings Scale by factor  x = factor * u  y = factor * v Source (u, v) Destination (x, y)

Example Mappings Rotate by  degrees  x = ucos  vsin   y = usin  vcos  Source (u, v)Destination (x, y)

Example Mappings Shear in X by factor  x = u + factor * v  y = v Shear in Y by factor  x = u  y = v + factor * u

Other Mappings Any function of u and v  X = f x (u, v)  Y = f y (u, v)

Image Warping Implementation I Forward mapping for (int u =0; u < umax; u++) { for (int v = 0; v < vmax; v++) { float x = f x (u, v); float y = f y (u, v); dst (x, y) = src(u, v) ; }

Forward Mapping Iterate over source image  Many source pixels can map to same destination pixel  Some destination pixels may not be covered

Image Warping Implementation II Reverse mapping for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = f x -1 (x, y); float v = f y -1 (x, y); dst (x, y) = src(u, v); }

Reverse Mapping Iterate over destination image  Must resample source  May oversample, but much simpler

Resampling Evaluate source image at arbitrary (u,v)  (u, v) does not usually have integer coordinates

Resampling  Point sampling  Triangle filter  Gaussain filter

Point Sampling Take value at closest pixel  int iu = trunc(u + 0.5)  int iv = trunc(v + 0.5)  dst(x, y) = src(iu, iv) This method is simple, but it cause aliasing

Filtering Compute weighted sum of pixel neighborhood  Weights are normalized values of kernel function  Equivalent to convolution at samples

Triangle Filtering Kernel is triangle function

Triangle Filtering (with width = 1) Bilinearly interpolate four closest pixels  a = linear interpolation of src(u1, v2) and src(u2, v2)  b = linear interpolation of src(u1, v1) and src(u2, v1)  dst = linear interpolation of “a” and “b”

Gaussian Filtering Kernel is Gaussian function Filter width =2

Filtering Methods Comparison Trade-offs  Aliasing versus blurring  Computation speed PointBilinearGaussian

Image Warping Implementation Reverse mapping for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = f x -1 (x, y); float v = f y -1 (x, y); dst (x, y) = resample_src(u, v, w); }

Example: Scale scale (src, dst, sx, sy) : Float w = max (1.0/sx, 1.0/sy) for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = x / sx; float v = y / sy; dst (x, y) = resample_src(u, v, w); }

Example: Rotate Rotate (src, dst, theta) : for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = x*cos(-theta)  y*sin(-theta); float v = x*sin(-theta)  y*cos(-theta); dst (x, y) = resample_src(u, v, w); }

Example: Fun Swirl (src, dst, theta) : for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = rot(dist(x, xcenter)*theta); float v = rot(dist(y, yecnter)*theta); dst (x, y) = resample_src(u, v, w); }

Combining images Image compositing  Blue-screen mattes  Alpha channel Image morphing  Specifying correspondence  Warping  Blending

Image Compositing Separate an image into “elements”  Render independently  Composite together Application  Cel animation  Chroma-keying  Blue-screen matting

Blue Screen Matting Composite foreground and background images  Create background image  Create foreground image with blue background  Insert non-blue foreground pixels into background Problem : no partial coverage

Alpha Channel Encodes pixel coverage informations   = 0  no coverage (or transparent)   = 1  full coverage (or opaque)  0 <  < 1  partial coverage (or semi-transparent) Example:  = 0.3

Composite with Alpha  Control the linear interpolation of foreground and background pixels when elements are composited

Pixels with Alpha Alpha channel convention  (r, g, b, a) represents a pixel that is a covered by the color C = (r/a, g/a, b/a) Color components are premultiplied by a Can display (r, g, b) value directly Closure in composition algebra

Semi-Transparent Objects Suppose we put A over B background G  How much of B is blocked by A ?  A  How much of B show through A ?  A )  How much of G shows through both A and B ?  A )  B )  α A A+ (1-α A )α B B+ (1-α A )(1-α B )G = A over [ B over G ]

Opaque Objects How do we combine 2 partially covered pixels?  3 possible colors (0, A, B)  4 regions (0, A, B, AB)

Composition Algebra 12 reasonable combinations

Image Morphing Animate transition between two images

Cross-Dissolving Blend images with “over” operator  alpha of bottom image is 1.0  alpha of top image varies from 0.0 to 1.0 blend(i, j) = (1-t) src(i, j) + t dst(i, j) (0 <= t <= 1)

Image Morphing Combines warping and cross-disolving

Feature Based Morphing Beier & Neeley use pairs of lines to specify warp  Given p in dst image, where is p’ in source image ?

Feature Based Morphing Feature Matching Image Warping Morphed Image

Feature Based Morphing Morphing Warping Black Or White-Michael Jackson

Image Processing Quantization  Uniform Quantization  Halftoning & Dithering Pixel Operation  Add random noise  Add luminance  Add contrast  Add saturation Filtering  Blur  Detected edges Warping  Scale  Rotate  Warp Combining  Composite  Morph Ref] Thomas Funkhouser, Princeton Univ.