Image 2016, Fall

Image Processing Quantization Pixel Operation Filtering Warping Uniform Quantization Halftoning & Dithering Pixel Operation Add random noise Add luminance Add contrast Add saturation Filtering Blur Detected edges Warping Scale Rotate Warp Combining Composite Morph Ref] Thomas Funkhouser, Princeton Univ.

What is an Image? An image is a 2D rectilinear array of samples A pixel is a sample, not a little square

Image Resolution Intensity resolution Spatial resolution Each pixel has only “Depth” bits for colors/intensities Spatial resolution Image has only “Width” x “Height” pixels Temporal resolution Monitor refreshes images at only “Rate” Hz Width Width x Height Depth Rate NTSC 640x480 8 30 Workstation 1280x1024 24 75 Film 3000 x 2000 12 Laser Printer 6600x5100 1 - Depth Height Typical Resolutions

Source of Error Intensity quantization Spatial aliasing Not enough intensity resolution Spatial aliasing Not enough spatial resolution Temporal aliasing Not enough temporal resolution Real value Sample value

Quantization Artifact due to limited intensity resolution Frame buffers have limited number of bits per pixel Physical devices have limited dynamic range

Uniform Quantization 반올림(Round)

Uniform Quantization Image with decreasing bits per pixel

Reducing Effects of Quantization Our eyes tend to integrate or average the fine detail into an overall intensity Halftoning A technique used in newspaper printing The process of converting a continuous tone image into an image that can be printed with one color ink (grayscale) or four color inks (color) Dithering Dithering is used to create a wide variety of patterns for use as backgrounds, fills and shadings, as well as for creating halftones, and for correcting aliasing. Halftones are created through a process called dithering Dithering  Methods(방법론) , Halfton  result(결과물)

Classical Halftoning Use dots of varying size to represent intensities Area of dots proportional to intensity in image

Halftoning A technique used in newspaper printing Only two intensities are possible, blob of ink and no blob of ink But, the size of the blob can be varied Also, the dither patterns of small dots can be used

Classical Halftoning Original Image Halfton Image

Halfton patterns Use cluster of pixels to represent intensity Trade spatial resolution for intensity resolution 2 by 2 pixel grid patterns mask 3 by 3 pixel grid patterns

Halfton patterns

Dithering Distribute errors among pixels Exploit spatial integration in our eye Display greater range of perceptible intensities

Halftoning – Moire Patterns Moire[more-ray] patterns Repeated use of same dot pattern for particular shade results in repeated pattern (Perceived as a moire pattern) result from overlapping repetitive elements Instead, randomize halftone pattern

Halftoning – Moire Patterns

Signal Processing (1/2) Signal Image Real World : Continuous Signal Digital World: Discrete Signal Image spatial domain not temporal domain intensity variation over space

Signal Processing (2/2) Sampling and reconstruction process Sampling The process of selecting a finite set of values from a signal Samples (the selected values) Reconstruction recreate the original continuous signal from the samples

Sampling and Reconstruction

Aliasing In general Specifically, in graphics Artifact due to under-sampling or poor reconstruction Specifically, in graphics Spatial aliasing Temporal aliasing Sampling below the Nyquist rate

Nyquist rate and aliasing lower bound on the sampling rate Sampling at the Nyquist rate sampling rate > 2* max frequency 주파수(frequency): 초당 사이클의 횟수(Hz)

Spatial Aliasing Artifact due to limited spatial resolution

Spatial Aliasing Artifact due to limited spatial resolution

Temporal Aliasing Artifact due to limited temporal resolution Spinning Backward The wheel is spinning at 5 revolution per second 3rd row  appear to be spinning backward at revolution per second

Temporal Aliasing Artifact due to limited temporal resolution Flickering

Anti-Aliasing Sample at higher rate Not always possible Doesn’t always solve problem Pre-filter to form bandlimited signal Form bandlimited function(low-pass filter) Trades aliasing for blurring

Pixel Operations Grayscale Compute luminance L L=0.30R+0.59G+0.11B (old)  0.2125R+0.7154G+0.0721B (CRT, HDTV) Original Image Grayscale Image

Pixel Operations Negative Image Simply inverse pixel components Ex) RGB(5, 157, 178)  RGB(250, 98, 77) Original Image Negative Image (R, G, B)  (255-R, 255-G, 255-B)

Pixel Operations Adjusting Brightness Simply scale pixel components Must clamp to range (e.g., 0 to 255) (R, G, B)  (R, G , B) * scale

Pixel Operations Adjusting Contrast Compute mean luminance L for all pixels Luminance = 0.30* r + 0.59*g + 0.11*b Scale deviation from L for each pixel component Must clamp to range (e.g., 0 to 255) R  L + (R-L)*scale G  L + (G-L)*scale B  L + (B-L)*scale

Pixel Operations Changing Brightness Changing Contrast

Pixel Operations Adjusting Saturation Color Convert RGB  HSV Scale from S for each pixel component Color Convert HSV  RGB Must clamp to range (e.g., 0 to 255)

Filtering Convolution Spatial domain Output pixel is weighted sum of pixels in neighborhood of input image Patten of weights is the “filter”

Filtering Convolution

Filtering Adjust Blurriness Convolve with a filter whose entries sum to one Each pixel becomes a weighted average of its neighbors Filter Sum = 1

Filtering Edge Detection Convolve with a filter that finds differences between neighbor pixels

Filtering Sharpening Convolve with a Sharpening filter

Filtering Embossing Convert the image into grayscale Convolve with a Embossing filter Plus 0.5 (128) for each pixel Must clamp to range (e.g., 0 to 255)

Summary Image representation Halftoning and dithering A pixel is a sample, not a little square Images have limited resolution Halftoning and dithering Reduce visual artifacts due to quantization Distribute errors among pixels Exploit spatial integration in our eye Sampling and reconstruction Reduce visual artifacts due to aliasing Filter to avoid undersampling Blurring is better than aliasing

Image Processing Quantization Pixel Operation Filtering Warping Uniform Quantization Halftoning & Dithering Pixel Operation Add random noise Add luminance Add contrast Add saturation Filtering Blur Detected edges Warping Scale Rotate Warp Combining Composite Morph Ref] Thomas Funkhouser, Princeton Univ.

Image Warping Move pixels of image Mapping Resampling Forward Reverse Point sampling Triangle Filter Gaussian filter

Mapping Define transformation Describe the destination (x,y) for every location (u,v) in the source (or vice-verse, if invertible) Source (u, v) Destination (x, y)

Example Mappings Scale by factor x = factor * u y = factor * v Destination (x, y) Source (u, v)

Example Mappings Rotate by q degrees x = ucosq - vsinq y = usinq + vcosq Source (u, v) Destination (x, y)

Example Mappings Shear in X by factor Shear in Y by factor x = u + factor * v y = v Shear in Y by factor x = u y = v + factor * u

Other Mappings Any function of u and v X = fx (u, v) Y = fy (u, v)

Image Warping Implementation I Forward mapping for (int u =0; u < umax; u++) { for (int v = 0; v < vmax; v++) { float x = fx (u, v); float y = fy (u, v); dst (x, y) = src(u, v) ; }

Forward Mapping Iterate over source image Many source pixels can map to same destination pixel Some destination pixels may not be covered

Image Warping Implementation II Reverse mapping for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = fx-1 (x, y); float v = fy-1 (x, y); dst (x, y) = src(u, v); }

Reverse Mapping Iterate over destination image Must resample source May oversample, but much simpler

Resampling Evaluate source image at arbitrary (u,v) (u, v) does not usually have integer coordinates

Resampling Resampling Point sampling Triangle filter Gaussain filter

Point Sampling Take value at closest pixel int iu = trunc(u + 0.5) int iv = trunc(v + 0.5) dst(x, y) = src(iu, iv) This method is simple, but it cause aliasing

Filtering Compute weighted sum of pixel neighborhood Weights are normalized values of kernel function Equivalent to convolution at samples

Triangle Filtering Kernel is triangle function

Triangle Filtering (with width = 1) Bilinearly interpolate four closest pixels a = linear interpolation of src(u1, v2) and src(u2, v2) b = linear interpolation of src(u1, v1) and src(u2, v1) dst = linear interpolation of “a” and “b”

Gaussian Filtering Kernel is Gaussian function Filter width =2

Filtering Methods Comparison Trade-offs Aliasing versus blurring Computation speed Point Bilinear Gaussian

Image Warping Implementation Reverse mapping for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = fx-1 (x, y); float v = fy-1 (x, y); dst (x, y) = resample_src(u, v, w); }

Example: Scale scale (src, dst, sx, sy) : Float w = max (1.0/sx, 1.0/sy) for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = x / sx; float v = y / sy; dst (x, y) = resample_src(u, v, w); }

Example: Rotate Rotate (src, dst, theta) : for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = x*cos(-theta) - y*sin(-theta); float v = x*sin(-theta) + y*cos(-theta); dst (x, y) = resample_src(u, v, w); }

Example: Fun Swirl (src, dst, theta) : for (int x =0; x < xmax; x++) { for (int y = 0; y < ymax; y++) { float u = rot(dist(x, xcenter)*theta); float v = rot(dist(y, yecnter)*theta); dst (x, y) = resample_src(u, v, w); }

Combining images Image compositing Image morphing Blue-screen mattes Alpha channel Image morphing Specifying correspondence Warping Blending

Image Compositing Separate an image into “elements” Application Render independently Composite together Application Cel animation Chroma-keying Blue-screen matting

Blue Screen Matting Composite foreground and background images Create background image Create foreground image with blue background Insert non-blue foreground pixels into background Problem : no partial coverage

Alpha Channel Encodes pixel coverage informations Example: a = 0.3 a = 0  no coverage (or transparent) a = 1  full coverage (or opaque) 0 < a < 1  partial coverage (or semi-transparent) Example: a = 0.3

Composite with Alpha Composite with Alpha Control the linear interpolation of foreground and background pixels when elements are composited

Pixels with Alpha Alpha channel convention (r, g, b, a) represents a pixel that is a covered by the color C = (r/a, g/a, b/a) Color components are premultiplied by a Can display (r, g, b) value directly Closure in composition algebra

Semi-Transparent Objects Suppose we put A over B background G How much of B is blocked by A ? aA How much of B show through A ? (1-aA) How much of G shows through both A and B ? (1-aA) (1-aB) αAA+ (1-αA )αBB+ (1-αA )(1-αB )G = A over[B over G]

Opaque Objects How do we combine 2 partially covered pixels? 3 possible colors (0, A, B) 4 regions (0, A, B, AB)

Composition Algebra 12 reasonable combinations

Image Morphing Animate transition between two images

Cross-Dissolving Blend images with “over” operator alpha of bottom image is 1.0 alpha of top image varies from 0.0 to 1.0 blend(i, j) = (1-t) src(i, j) + t dst(i, j) (0 <= t <= 1)

Image Morphing Combines warping and cross-disolving

Feature Based Morphing Beier & Neeley use pairs of lines to specify warp Given p in dst image, where is p’ in source image ?

Feature Based Morphing Feature Matching Image Warping Image Warping Morphed Image

Feature Based Morphing Warping Black Or White-Michael Jackson

Image Processing Quantization Pixel Operation Filtering Warping Uniform Quantization Halftoning & Dithering Pixel Operation Add random noise Add luminance Add contrast Add saturation Filtering Blur Detected edges Warping Scale Rotate Warp Combining Composite Morph Ref] Thomas Funkhouser, Princeton Univ.