Pre-Filter Antialiasing Kurt Akeley CS248 Lecture 4 4 October

CS248 Lecture 4Kurt Akeley, Fall 2007 Review

CS248 Lecture 4Kurt Akeley, Fall 2007 Aliasing Aliases are low frequencies in a rendered image that are due to higher frequencies in the original image.

CS248 Lecture 4Kurt Akeley, Fall 2007 Jaggies Original: Rendered:

CS248 Lecture 4Kurt Akeley, Fall 2007 Convolution theorem Let f and g be the transforms of f and g. Then Something difficult to do in one domain (e.g., convolution) may be easy to do in the other (e.g., multiplication)

CS248 Lecture 4Kurt Akeley, Fall 2007 Aliased sampling of a point (or line) x = * = F(s)f(x)

CS248 Lecture 4Kurt Akeley, Fall 2007 Aliased reconstruction of a point (or line) x = * = F(s)f(x) sinc(x)

CS248 Lecture 4Kurt Akeley, Fall 2007 Reconstruction error (actually none!) Original Signal Aliased Reconstruction Phase matters!

CS248 Lecture 4Kurt Akeley, Fall 2007 Sampling theory Fourier theory explains jaggies as aliasing. For correct reconstruction: n Signal must be band-limited n Sampling must be at or above Nyquist rate n Reconstruction must be done with a sinc function All of these are difficult or impossible in the general case. Let’s see why …

CS248 Lecture 4Kurt Akeley, Fall 2007 Why band-limiting is difficult Band-limiting primitives prior to rendering compromises image assembly in the framebuffer n Band-limiting changes (spreads) geometry n Finite spectrum  infinite spatial extent n Interferes with occlusion calculations n Leaves visible seams between adjacent triangles Can’t band-limit the final (framebuffer) image n There is no final image, there are only samples n If the sampling is aliased, there is no recovery Nyquist-rate sampling requires band-limiting

CS248 Lecture 4Kurt Akeley, Fall 2007 Why ideal reconstruction is difficult In theory: n Required sinc function has n Negative lobes (displays can’t produce negative light) n Infinite extent (cannot be implemented) In practice: n Reconstruction is done by a combination of n Physical display characteristics (CRT, LCD, …) n The optics of the human eye n Mathematical reconstruction (as is done, for example, in high-end audio equipment) is not practical at video rates.

CS248 Lecture 4Kurt Akeley, Fall 2007 Two antialiasing approaches are practiced Pre-filtering (this lecture) n Band-limit primitives prior to sampling n OpenGL ‘smooth’ antialiasing Increased sample rate (future lecture) n Multisampling, super-sampling n OpenGL ‘multisample’ antialiasing

CS248 Lecture 4Kurt Akeley, Fall 2007 Pre-Filter Antialiasing

CS248 Lecture 4Kurt Akeley, Fall 2007 Ideal point (or line cross section) x = * = F(s)f(x) sinc(x)

CS248 Lecture 4Kurt Akeley, Fall 2007 Band-limited unit disk x = * = F(s)f(x) sinc(x)

CS248 Lecture 4Kurt Akeley, Fall 2007 Almost-band-limited unit disk x = * = F(s)f(x) sinc(x) sinc 2 (x) Finite extent! sinc 3 (x)

CS248 Lecture 4Kurt Akeley, Fall 2007 Equivalences These are equivalent: n Low-pass filtering n Band-limiting n (Ideal) reconstruction These are equivalent: n Point sampling of band-limited geometry n Weighted area sampling (FvD ) of full-spectrum geometry

CS248 Lecture 4Kurt Akeley, Fall 2007 Pre-filter pros and cons Pros n Almost eliminates aliasing due to undersampling n Allows pre-computation of expensive filters n Great image quality (with important caveats!) Cons n Can’t handle interactions of primitives with area n Reconstruction is still a problem

CS248 Lecture 4Kurt Akeley, Fall 2007 Pre-filtering in the vertex pipeline Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations struct { float x,y,z,w; float r,g,b,a; } vertex; struct { vertex v0,v1,v2 } triangle; struct { short int x,y; float depth; float r,g,b,a; } fragment; struct { int depth; byte r,g,b,a; } pixel; Framebuffer Point sampling of pre- filtered primitives sets fragment alpha value Fragments are blended into the frame buffer

CS248 Lecture 4Kurt Akeley, Fall 2007 Pre-filter point rasterization alpha point being sampled at the center of this pixel

CS248 Lecture 4Kurt Akeley, Fall 2007 Pre-filter line rasterization line being sampled at the center of this pixel alpha

CS248 Lecture 4Kurt Akeley, Fall 2007 Pre-filtered primitive stored in table Supports pre-computation Simplifies distance calculation (square root in table) Can compensate for reconstruction errors n Sampling and reconstruction go hand-in-hand 3 Point size 4 X screen frac bits 4 Y screen frac bits 4 Pixel index 8 Fragment Alpha 32K x 8

CS248 Lecture 4Kurt Akeley, Fall 2007 Line tables get big! 5 Line slope 4 X screen frac bits 4 Y screen frac bits 4 Pixel index 8 1M x 8 3 Line width Fragment Alpha

CS248 Lecture 4Kurt Akeley, Fall 2007 Triangle pre-filtering is difficult Three arbitrary vertex locations define a huge parameter space n Resulting table is too large to implement Edges can be treated independently n But this is a poor approximation near vertexes, where two or three edges affect the filtering n And triangles can be arbitrarily small

CS248 Lecture 4Kurt Akeley, Fall 2007 Pre-filtering in the vertex pipeline Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations struct { float x,y,z,w; float r,g,b,a; } vertex; struct { vertex v0,v1,v2 } triangle; struct { short int x,y; float depth; float r,g,b,a; } fragment; struct { int depth; byte r,g,b,a; } pixel; Framebuffer Point sampling of pre- filtered primitives sets fragment alpha value Fragments are blended into the frame buffer

CS248 Lecture 4Kurt Akeley, Fall 2007 Ideal pre-filtered image assembly Impulse-defined points and lines have no area So they don’t occlude each other So fragments should be summed into pixels C’ pixel = C pixel + A frag C frag

CS248 Lecture 4Kurt Akeley, Fall 2007 Practical pre-filtered image assembly Frame buffers have limited numeric range So summation causes overflow or clamping ‘Uncorrelated’ clamped blend yields good results n But clamping R, G, and B independently causes color shift ‘Anti-correlated’ clamped blend is an alternative n Best for rendering pre-filtered triangles n sorted front-to-back n But requires frame buffer to store alpha values C’ pixel = (1-A frag )C pixel + A frag C frag

CS248 Lecture 4Kurt Akeley, Fall 2007 Anti-correlated blend function i = min(A frag, (1-A pixel )) A’ pixel = A pixel + I C’ pixel = C pixel + i C frag

CS248 Lecture 4Kurt Akeley, Fall 2007 Demo

CS248 Lecture 4Kurt Akeley, Fall 2007 OpenGL API glEnable(GL_POINT_SMOOTH); glEnable(GL_LINE_SMOOTH); glEnable(GL_POLYGON_SMOOTH); glHint(GL_POINT_SMOOTH_HINT, GL_NICEST); glHint(GL_LINE_SMOOTH_HINT, GL_NICEST); glHint(GL_POLYGON_SMOOTH_HINT, GL_NICEST); glEnable(GL_BLEND); glBlendFunc(GL_SRC_ALPHA, GL_ONE);// sum glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);// blend glBlendFunc(GL_SRC_ALPHA_SATURATE, GL_ONE);// saturate

CS248 Lecture 4Kurt Akeley, Fall 2007 Filtering and Human Perception

CS248 Lecture 4Kurt Akeley, Fall 2007 Resolution of the human eye Eye’s resolution is not evenly distributed n Foveal resolution is ~20x peripheral n Systems can track direction of view, draw high- resolution inset n Flicker sensitivity is higher in periphery One eye can compensate for the other n Research at NASA suggests high-resolution dominant display Human visual system is well engineered …

CS248 Lecture 4Kurt Akeley, Fall 2007 Imperfect optics - linespread IdealActual

CS248 Lecture 4Kurt Akeley, Fall 2007 Linespread function

CS248 Lecture 4Kurt Akeley, Fall 2007 Filtering x = * = F(s)f(x)

CS248 Lecture 4Kurt Akeley, Fall 2007 Frequencies are selectively attenuated = * = f(x) *

CS248 Lecture 4Kurt Akeley, Fall 2007 Selective attenuation (cont.) = * = f(x) *

CS248 Lecture 4Kurt Akeley, Fall 2007 Modulation transfer function (MTF)

CS248 Lecture 4Kurt Akeley, Fall 2007 Optical “imperfections” improve acuity Image is pre-filtered n Cutoff is approximately 60 cpd n Cutoff is gradual – no ringing Aliasing is avoided n Foveal cone density is 120 / degree n Cutoff matches retinal Nyquist limit Vernier acuity is improved n Foveal resolution is 30 arcsec n Vernier acuity is 5-10 arcsec n Linespread blur includes more sensors

CS248 Lecture 4Kurt Akeley, Fall 2007 Summary Two approaches to antialiasing are practiced n Pre-filtering (this lecture) n Increased sample rate Pre-filter antialiasing n Works well for non-area primitives (points, lines) n Works poorly for area primitives (triangles, especially in 3-D) The human eye is well adapted n Sampling theory helps us here too

CS248 Lecture 4Kurt Akeley, Fall 2007 Assignments Before Thursday’s class, read n FvD 3.1 through 3.14, Rasterization Project 1: n Continue work n Demos Wednesday 10 October

CS248 Lecture 4Kurt Akeley, Fall 2007 End