Filtering and Color To filter a color image, simply filter each of R,G and B separately Re-scaling and truncating are more difficult to implement: –Adjusting each channel separately may change color significantly –Adjusting intensity while keeping hue and saturation may be best, although some loss of saturation is probably OK

Compositing Compositing combines components from two or more images to make a new image The basis for film special effects (even before computers) –Create digital imagery and composite it into live action Important part of animation – even hand animation –Background change more slowly than foregrounds, so composite foreground elements onto constant background

Very Simple Example over =

Mattes A matte is an image that shows which parts of another image are foreground objects Term dates from film editing and cartoon production To composite with a matte: –Take foreground pixels over white parts of the matte and copy them into the background image

Alpha Basic idea: Encode opacity information in the image Add an extra channel, the alpha channel, to each image –alpha = 1 implies full opacity at a pixel –alpha = 0 implies completely clear pixels Images are now in RGBA format, and typically 32 bits per pixel (8 bits for alpha)

Smoothing Edges Reduce alpha gradually at edges to smooth them

Pre-Multiplied Alpha Instead of storing (R,G,B,  ), store (  R,  G,  B,  ) The compositing operations in the next several slides are easier with pre-multiplied alpha To display and do color conversions, must extract RGB by dividing out  –  =0 is always black –Some loss of precision as  gets small, but generally not a problem

Alpha and Translucent Objects If the image is of a translucent object, then  represents the amount of the background that is blocked When combining two translucent objects: –(1-  a )(1-  b ) of the background shows through both –  a (1-  b ) passes through B but is blocked by A –  b (1-  a ) passes through A but is blocked by B –  a  b of the background is blocked by both

Alpha and Opaque Objects Assume a pixel represents the color of a small area –Typically a square, but not necessarily Interpret  to represent the fraction of the pixel area covered by an object Question: When we combine two images, how much of the pixel is covered? –What should the new  be?

Sub-Pixel Configurations We will assume partial overlap, implying that we have no specific knowledge of the sub-pixel structure No overlap  o =  a +  b Full overlap  o =  b Partial overlap  o =  a + (1-  a )  b

Compositing Assumptions We will combine two images, f and g, to get a third composite image –Not necessary that one be foreground and background –Background can remain unspecified Both images are the same size and use the same color representation Multiple images can be combined in stages, operating on two at a time

Sample Images

Image Decomposition The composite image can be broken into regions –Parts covered by f only –Parts covered by g only –Parts covered by f and g –Parts covered by neither f nor g Same goes for sub-pixels in places where  1

Sample Decomposition

Basic Compositing Operation The different compositing operations define which image “wins” in each sub-region of the composite At each pixel, combine the pixel data from f and the pixel data from g with the equation: F and G describe how much of each input image survives, and c f and c g are pre-multiplied pixels, and all four channels are calculated

“Over” Operator Computes composite with the rule that f covers g

“Over” Operator

“Inside” Operator Computes composite with the rule that only parts of f that are inside g contribute

“Inside” Operator

“Outside” Operator Computes composite with the rule that only parts of f that are outside g contribute

“Outside” Operator

“Atop” Operator Computes composite with the over rule but restricted to places where there is some g

“Atop” Operator

“Xor” Operator Computes composite with the rule that f contributes where there is no g, and g contributes where there is no f

“Xor” Operator

“Clear” Operator Computes a clear composite Note that (0,0,0,  >0) is a partially opaque black pixel, whereas (0,0,0,0) is fully transparent, and hence has no color

“Set” Operator Computes composite by setting it to equal f Copies f into the composite

Unary Operators Darken: Makes an image darker (or lighter) without affecting its opacity Dissolve: Makes an image transparent without affecting its color

“PLUS” Operator Computes composite by simply adding f and g, with no overlap rules Useful for defining cross dissolve in terms of compositing:

Obtaining  Values Hand generate (paint a grayscale image) Automatically create by segmenting an image into foreground background: –Blue-screening is the analog method Remarkably complex to get right –“Lasso” is the Photoshop operation With synthetic imagery, use a special background color that does not occur in the foreground –Brightest blue is common

Compositing With Depth Can store pixel “depth” instead of alpha Then, compositing can truly take into account foreground and background Generally only possible with synthetic imagery –Image Based Rendering is an area of graphics that, in part, tries to composite photographs taking into account depth