Announcements Class web site – Handouts –class info –lab access/accounts –survey Readings for Monday (via web site) –Paul Heckbert, Survey of Texture Mapping, IEEE Computer Graphics and Applications, 6(11), November 1986, –Beier, T. and Neely, S., Feature-Based Image Metamorphosis, ACM Computer Graphics (SIGGRAPH'92), 26(2), July 1992, CSE 590 “Vision for Graphics”

Today Intro Admin Survey Introductions Course overview 2D image processing Blending Filtering Pyramids on Monday (1/8) image warping, morphing image enhancement

Vision for Graphics—Why? Vision and Graphics are inverse problems Computer vision World model Computer graphics World model

Intersection of Vision and Graphics modeling - shape - light - motion - optics - images IP animation rendering user-interfaces surface design Computer Graphics shape estimation motion estimation recognition 2D modeling modeling - shape - light - motion - optics - images IP Computer Vision

Cross Fertilization Vision impacts graphics image-based rendering model acquisition motion capture perceptual user interfaces special effects image editing Graphics impacts vision reflectance transparency shape modeling

Course Objectives What to expect Knowledge of vision that is relevant to graphics How to apply your expertise in image analysis to synthesis Fundamentals Explore new avenues for research What not to expect Not a graphics course Not a complete vision course

Administrative Stuff Web Site Grading 2 programming projects 1 final research project Class presentation Class participation Software and Hardware Programming projects in C/C++ Support code for Windows and Linux Lab: 228 Sieg Hall (Win2K PC’s) –Fill out forms to get key access, CSE class account –You’re welcome to use your own machines instead Digital still and video cameras, tripods, etc.

Prerequisites Prior course on vision OR graphics Assume Familiarity with image representations Basic image processing (linear filtering, transforms, etc.) Differential equations, linear algebra Camera modeling and projection Ability to read research articles, fill in gaps Questions? See Steve or Rick

Areas Image analysis/synthesis Creating graphical models Image editing Image-based rendering Perceptual user interfaces Motion capture Capturing light and reflectance

Image Processing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain," Proc. IEEE: Computer Vision and Pattern Recognition, pp , June,

Motion Estimation Interview with a Vampire, Courtesy Doug Roble, Digital Domain mosaic demo

Pose Estimation Ascending Stairs,Eadweard Muybridge,

3D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996

Image-Based Rendering View Morphing, Seitz and Dyer, SIGGRAPH 96

Modeling light "Interface", courtesy of Lance Williams, 1985 Environment Matting and Compositing, Zongker, Werner, Curless, and Salesin. SIGGRAPH 99

Image Blending

Feathering = Encoding transparency I(x,y) = (  R,  G,  B,  ) I blend = I left + I right See Blinn reading (CGA, 1994) for details

Affect of Window Size 0 1 left right 0 1

Affect of Window Size

Good Window Size 0 1 “Optimal” Window: smooth but not ghosted

What is the Optimal Window? To avoid seams window = size of largest prominent feature To avoid ghosting window <= 2*size of smallest prominent feature Natural to cast this in the Fourier domain largest frequency <= 2*size of smallest frequency image frequency content should occupy one “octave” (power of two) FFT

What if the Frequency Spread is Wide Idea (Burt and Adelson) Compute F left = FFT(I left ), F right = FFT(I right ) Decompose Fourier image into octaves (bands) –F left = F left 1 + F left 2 + … Feather corresponding octaves F left i with F right i –Can compute inverse FFT and feather in spatial domain Sum feathered octave images in frequency domain Better implemented in spatial domain FFT

Octaves in the Spatial Domain Bandpass Images Lowpass Images

Image Pyramids

Pyramid Creation “Laplacian” Pyramid Created from Gaussian pyramid by subtraction L l = G l – expand(G l+1 ) filter mask “Gaussian” Pyramid

Pyramids Advantages of pyramids Faster than Fourier transform Avoids “ringing” artifacts Many applications small images faster to process good for multiresolution processing compression progressive transmission Known as “mip-maps” in graphics community Precursor to wavelets Wavelets also have these advantages

Pyramid Blending

laplacian level 4 laplacian level 2 laplacian level 0 left pyramidright pyramidblended pyramid

Blending Regions Other applications Removing block artifacts in compressed images

Limitations?

Related Topics Matting Given image and background(s), estimate foreground What if foreground object is refractive? –Environment matting Hole filling Remove scratches, holes in an image Texture synthesis Environment Matting and Compositing, Zongker, Werner, Curless, and Salesin. SIGGRAPH 99