02/14/02(c) University of Wisconsin 2002, CS 559 Last Time Filtering Image size reduction –Take the pixel you need in the output –Map it to the input –Place a filter at that location, and do the weighted sum of neighboring pixels –Place the filter value in the output image

02/14/02(c) University of Wisconsin 2002, CS 559 Today Enlarging Images Morphing Compositing Project 1: Image Editing Software

02/14/02(c) University of Wisconsin 2002, CS 559 Enlarging Images Warp function: f(x)=kx, k < 1 Problem: Have to create pixel data –More pixels in output than in input Solution: Filter up to larger size –Apply the filter at intermediate pixel locations –eg: To get double image size, apply filter at every pixel and every half pixel New pixels are interpolated from old ones –Filter encodes interpolation function –May want to edge-enhance images after enlarging

02/14/02(c) University of Wisconsin 2002, CS 559 Enlargement

02/14/02(c) University of Wisconsin 2002, CS 559 Ideal Enlargement Reconstruct original function Resample at higher frequency Original function was band-limited, so re-sampling does not add any extra frequency information

02/14/02(c) University of Wisconsin 2002, CS 559 Image Morphing Process to turn one image into another –Also the basis for video effects like cross-dissolve Define path from each point in the original image to its destination in the output image –Color may change along path –Many ways to define paths Animate points along paths –At each time increment, step pixels along their paths and interpolate their colors –Resample the result to get a regular image

02/14/02(c) University of Wisconsin 2002, CS 559 Filtering and Color To filter a color image, simply filter each of R,G and B separately Re-scaling and truncating are more difficult to implement: –Adjusting each channel separately may change color significantly –Adjusting intensity while keeping hue and saturation may be best, although some loss of saturation is probably OK

02/14/02(c) University of Wisconsin 2002, CS 559 Compositing Compositing combines components from two or more images to make a new image The basis for film special effects (even before computers) –Create digital imagery and composite it into live action Important part of animation – even hand animation –Background change more slowly than foregrounds, so composite foreground elements onto constant background

02/14/02(c) University of Wisconsin 2002, CS 559 Very Simple Example over =

02/14/02(c) University of Wisconsin 2002, CS 559 Mattes A matte is an image that shows which parts of another image are foreground objects Term dates from film editing and cartoon production How would I use a matte to insert an object into a background? How are mattes usually generated for television?

02/14/02(c) University of Wisconsin 2002, CS 559 Working with Mattes To insert an object into a background –Call the image of the object the source –Put the background into the destination –For all the source pixels, if the matte is white, copy the pixel, otherwise leave it unchanged To generate mattes: –Use smart selection tools in Photoshop or similar They outline the object and convert the outline to a matte –Blue Screen: Photograph/film the object in front of a blue background, then consider all the blue pixels in the image to be the background

02/14/02(c) University of Wisconsin 2002, CS 559 Alpha Basic idea: Encode opacity information in the image Add an extra channel, the alpha channel, to each image –For each pixel, store R, G, B and Alpha –alpha = 1 implies full opacity at a pixel –alpha = 0 implies completely clear pixels Images are now in RGBA format, and typically 32 bits per pixel (8 bits for alpha)

02/14/02(c) University of Wisconsin 2002, CS 559 Smoothing Edges Reduce alpha gradually at edges to smooth them

02/14/02(c) University of Wisconsin 2002, CS 559 Pre-Multiplied Alpha Instead of storing (R,G,B,  ), store (  R,  G,  B,  ) The compositing operations in the next several slides are easier with pre-multiplied alpha To display and do color conversions, must extract RGB by dividing out  –  =0 is always black –Some loss of precision as  gets small, but generally not a problem

02/14/02(c) University of Wisconsin 2002, CS 559 Combining Images and Alpha If the image is of a translucent object, then  represents the amount of the background that is blocked If combining two translucent objects: –(1-  a )(1-  b ) of the background shows through both –  a (1-  b ) passes through B but is blocked by A –  b (1-  a ) passes through A but is blocked by B –  a  b of the background is blocked by both Assume a pixel represents the color of a small area –Typically a square, but not necessarily Interpret  to represent the fraction of the pixel area covered by an object –We’ll see ways of deciding how much background shows through when images are combined

02/14/02(c) University of Wisconsin 2002, CS 559 Compositing Assumptions We will combine two images, f and g, to get a third composite image –Not necessary that one be foreground and background –Background can remain unspecified Both images are the same size and use the same color representation Multiple images can be combined in stages, operating on two at a time

02/14/02(c) University of Wisconsin 2002, CS 559 Image Decomposition The composite image can be broken into regions –Parts covered by f only –Parts covered by g only –Parts covered by f and g –Parts covered by neither f nor g Same goes for sub-pixels in places where  1

02/14/02(c) University of Wisconsin 2002, CS 559 Basic Compositing Operation The different compositing operations define which image “wins” in each sub-region of the composite At each pixel, combine the pixel data from f and the pixel data from g with the equation: F and G describe how much of each input image survives, and c f and c g are pre-multiplied pixels, and all four channels are calculated To define a compositing operation, define F and G

02/14/02(c) University of Wisconsin 2002, CS 559 Sample Images Image Alpha

02/14/02(c) University of Wisconsin 2002, CS 559 “Over” Operator

02/14/02(c) University of Wisconsin 2002, CS 559 “Over” Operator Computes composite with the rule that f covers g

02/14/02(c) University of Wisconsin 2002, CS 559 “Inside” Operator

02/14/02(c) University of Wisconsin 2002, CS 559 “Inside” Operator Computes composite with the rule that only parts of f that are inside g contribute

02/14/02(c) University of Wisconsin 2002, CS 559 “Outside” Operator

02/14/02(c) University of Wisconsin 2002, CS 559 “Outside” Operator Computes composite with the rule that only parts of f that are outside g contribute

02/14/02(c) University of Wisconsin 2002, CS 559 “Atop” Operator

02/14/02(c) University of Wisconsin 2002, CS 559 “Atop” Operator Computes composite with the over rule but restricted to places where there is some g

02/14/02(c) University of Wisconsin 2002, CS 559 “Xor” Operator Computes composite with the rule that f contributes where there is no g, and g contributes where there is no f

02/14/02(c) University of Wisconsin 2002, CS 559 “Xor” Operator

02/14/02(c) University of Wisconsin 2002, CS 559 “Clear” Operator Computes a clear composite Note that (0,0,0,  >0) is a partially opaque black pixel, whereas (0,0,0,0) is fully transparent, and hence has no color

02/14/02(c) University of Wisconsin 2002, CS 559 “Set” Operator Computes composite by setting it to equal f Copies f into the composite

02/14/02(c) University of Wisconsin 2002, CS 559 Unary Operators Darken: Makes an image darker (or lighter) without affecting its opacity Dissolve: Makes an image transparent without affecting its color

02/14/02(c) University of Wisconsin 2002, CS 559 “PLUS” Operator Computes composite by simply adding f and g, with no overlap rules Useful for defining cross-dissolve in terms of compositing:

02/14/02(c) University of Wisconsin 2002, CS 559 Obtaining  Values Hand generate (paint a grayscale image) Automatically create by segmenting an image into foreground background: –Blue-screening is the analog method Remarkably complex to get right –“Lasso” is the Photoshop operation With synthetic imagery, use a special background color that does not occur in the foreground –Brightest blue is common –Why blue?

02/14/02(c) University of Wisconsin 2002, CS 559 Compositing With Depth Can store pixel “depth” instead of alpha Then, compositing can truly take into account foreground and background Generally only possible with synthetic imagery –Image Based Rendering is an area of graphics that, in part, tries to composite photographs taking into account depth