Rasterization Kurt Akeley CS248 Lecture 5 9 October

CS248 Lecture 5Kurt Akeley, Fall 2007 The vertex pipeline Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations struct { float x,y,z,w; float r,g,b,a; } vertex; struct { vertex v0,v1,v2 } triangle; struct { short int x,y; float depth; float r,g,b,a; } fragment; struct { int depth; byte r,g,b,a; } pixel; Frame buffer Screen coordinates

CS248 Lecture 5Kurt Akeley, Fall 2007 S creen coordinates Ideal screen coordinates are continuous n Not integer pixel locations! Implementations always use discrete math n Fixed-point or floating-point n Always with substantial sub-pixel precision n Fixed-point illustrated in the pre-filter antialiasing lecture A pixel is a ‘big’ thing n Spatial resolution can approach # of pixels on screen n Data resolution can be large too Multiple copies of pixel data structure n SGI RealityEngine frame buffer was deeper than wide or tall struct { float x,y,z; float r,g,b,a; } vertex;

CS248 Lecture 5Kurt Akeley, Fall 2007 Key facts about perspective projection Straight lines project to straight lines (on a plane) n Only vertexes need to be transformed n That’s why we’re interested in lines and polygons Parameterizations (e.g., distance) are warped: More on projection in later lectures …

CS248 Lecture 5Kurt Akeley, Fall 2007 Two fundamental operations Fragment selection n Identify pixels for which fragments are to be generated n ‘attributes’ are special n Must be n Exact, for aliased rendering n Superset, for antialiased rendering n Should be efficient, for performance Attribute assignment n Assign attribute values to each fragment n E.g., color, depth, … struct { short int x,y; float depth; float r,g,b,a; } fragment;

CS248 Lecture 5Kurt Akeley, Fall 2007 Fragment selection Generate one fragment for each pixel that is intersected (or covered) by the primitive Intersected could mean that the primitive’s area intersects the pixel’s: n Center point, or n Square region, or n Filter-function (in area-sampling terms) Some examples …

CS248 Lecture 5Kurt Akeley, Fall 2007 Point-sampled fragment selection Generate fragment if pixel center is inside triangle Implements point-sampled aliased rasterization

CS248 Lecture 5Kurt Akeley, Fall 2007 Point-sampled fragment selection Pixels along shared edges should have exactly one fragment selected for them n Must handle on-edge/on-vertex sample points consistently

CS248 Lecture 5Kurt Akeley, Fall 2007 Tiled fragment selection Generate fragment if unit square intersects triangle Implements multisample and tiled rasterizations

CS248 Lecture 5Kurt Akeley, Fall 2007 Tiled fragment selection Multisample rasterization n 4x4 samples per pixel

CS248 Lecture 5Kurt Akeley, Fall 2007 Tiled fragment selection Tiled rasterization n 4x4 pixels per tile

CS248 Lecture 5Kurt Akeley, Fall 2007 Antialiased fragment selection Generate fragment if filter function intersects triangle Implements pre-filtered antialiasing

CS248 Lecture 5Kurt Akeley, Fall 2007 Fragment selection (continued) What if the primitive doesn’t have a geometric area? n Delta-function points and lines don’t Three choices: n Rule-based approach n x n pixel point n Bresenham line (details later in this lecture) n Pre-filter n Bandlimited  infinite spatial extent n Assign a screen-space geometry n Circle for point n Rectangle for line

CS248 Lecture 5Kurt Akeley, Fall 2007 Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps n Parameterize the attribute n Fit a function (surface) to the vertex values n Point-sample this parameterization Which parameterization? n Constant (aka flat shading) n No continuity at shared edges

CS248 Lecture 5Kurt Akeley, Fall 2007 Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps n Parameterize the attribute n Fit a function (surface) to the vertex values n Point-sample this parameterization Which parameterization? n Constant (aka flat shading) n No continuity at shared edges n Bilinear (planar surface) n Value continuity at shared edges

CS248 Lecture 5Kurt Akeley, Fall 2007 Bilinear (planar) parameterization x y red

CS248 Lecture 5Kurt Akeley, Fall 2007 Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps n Parameterize the attribute n Fit a function (surface) to the vertex values n Point-sample this parameterization Which parameterization? n Constant (aka flat shading) n No continuity at shared edges n Bilinear (planar surface) n Value continuity at shared edges n Cubic (non-planar surface) n Slope continuity at shared edges

CS248 Lecture 5Kurt Akeley, Fall 2007 Mach banding – value discontinuities Flat shaded, but appeared ‘scalloped’

CS248 Lecture 5Kurt Akeley, Fall 2007 Mach banding – slope discontinuities Same peak intensities

CS248 Lecture 5Kurt Akeley, Fall 2007 Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps n Parameterize the attribute n Fit a function (surface) to the vertex values n Point-sample this parameterization Which parameterization? n Constant (aka flat shading) n No continuity at shared edges n Bilinear (planar surface) n Value continuity at shared edges n Cubic (non-planar surface) n Slope continuity at shared edges n Gouraud (hybrid)

CS248 Lecture 5Kurt Akeley, Fall 2007 Gouraud shaded quad Fragment selection n Walk (iterate along) edges n Change edges at vertexes Attribute assignment n Loop in a loop algorithm: n Iterate linearly along edges n Iterate linearly edge-to-edge n Outer loop is complex n E.g., either 2 or 3 regions n Parameterization is a function of n Screen orientation n Choice of spans

CS248 Lecture 5Kurt Akeley, Fall 2007 Problems with quads / polygons “All” projected quadrilaterals are non-planar n Due to discrete coordinate precision What if quadrilateral is concave? n Concave is complex (split spans -- see example) n Non-planar  concave for some view What if quadrilateral intersects itself? n A real mess (no vertex to signal change –- see example) n Non-planar  “bowtie” for some view

CS248 Lecture 5Kurt Akeley, Fall 2007 All polygons are triangles (or should be) Triangle is always convex n Regardless of arithmetic precision n Simplifies rasterization—no special cases Three points define a plane n All triangles are planar n All parameterizations are (or can be) planar Modern GPUs decompose polygons to triangles n SGI switched in 1990 with the VGX product n OpenGL is designed to allow triangulation n Optimized quadrilateral decomposition developed

CS248 Lecture 5Kurt Akeley, Fall 2007 Complex polygons There are algorithms to rasterize n Self-intersecting polygons n Polygons with holes n … These polygons have applications in 2-D rendering But they are not useful for 3-D rendering n Too slow to render n Don’t have meaningful attribute parameterizations So we will ignore them

CS248 Lecture 5Kurt Akeley, Fall 2007 Normal-based quad decomposition Compute A C and B D Connect vertex pair with the greater dot product Avoid connecting the ‘stirrups’ Bottom line: decomposition matters! A B C D

CS248 Lecture 5Kurt Akeley, Fall 2007 Iteration vs. direct evaluation Along edges Between adjacent pixels Iteration: Direct evaluation:

CS248 Lecture 5Kurt Akeley, Fall 2007 Iteration vs. direct evaluation Iteration n Is less numerically intensive (no multiplication) Direct evaluation n Is more precise (no accumulated error) n Parallelizes better (no sequence presumption)

CS248 Lecture 5Kurt Akeley, Fall 2007 DDA iteration Digital Differential Analyzer (DDA) Implements iteration in fixed-point representation E.g., iiiiiiii.ffff (8.4) or siiiiiii.ffff (s7.4) n Repeatedly adds delta value to accumulated value Loses ½ LSB precision per iteration step Require log2(n) fraction bits for n steps n To reach the correct extreme values n Dimensions of rendering space determine maximum number of steps n May differ from size of frame buffer n 2-D iteration requires an extra bit

CS248 Lecture 5Kurt Akeley, Fall 2007 Triangle rasterization examples Gouraud shaded (GTX) Edge walk, planar parameterization (VGX) Barycentric direct evaluation (InfiniteReality) Small tiles (Bali – proposed) Per-pixel evaluation (Pixel Planes 4)

CS248 Lecture 5Kurt Akeley, Fall 2007 Algorithm properties Setup and execution costs n Setup: constant per triangle n Execution: relative to triangle’s projected area Ability to parallelize Ability to cull to a rectangular screen region n To support tiling n To support “scissoring” Scissor region Triangle to be rasterized

CS248 Lecture 5Kurt Akeley, Fall 2007 Gouraud shaded (GTX) Two-stage algorithm n DDA edge walk n fragment selection n attribute assignment n DDA scan-line walk n attribute assignment only Requires expensive scan-line setup n Location of first sample is non- unit distance from edge Parallelizes in two stages (e.g., GTX) Cannot scissor efficiently Works on quadrilaterals dadx

CS248 Lecture 5Kurt Akeley, Fall 2007 Edge walk, planar evaluation (VGX) dadx dady

CS248 Lecture 5Kurt Akeley, Fall 2007 Edge walk, planar evaluation (VGX) Hybrid algorithm n Edge DDA walk for fragment selection n Efficient generation of conservative fragment set n Pixel-center DDA walk for attribute assignment n Never step off sample grid, so n Sub-pixel adjustment is made just once, –Rather than for each scan-line Scissor cull possible n Adds complexity to edge walk n Easy for attribute evaluation Parallelizes similarly to Gouraud

CS248 Lecture 5Kurt Akeley, Fall 2007 Interpolation outside the triangle

CS248 Lecture 5Kurt Akeley, Fall 2007 DDA can operate out-of-range MSBs beyond desired range don’t influence result n Carry chain flows up, not down n Can handle arbitrarily large slopes n Can iterate outside the triangle’s area Must not clamp (range limit) intermediate results! Doesn’t work for floating point! + Accum Delta

CS248 Lecture 5Kurt Akeley, Fall 2007 Wrapping ValueBinary Overflow Underflow Problem: overflow or underflow of iterated value n Integer arithmetic “wraps” n Maximum value overflows to zero n Zero underflows to maximum value n Minor iteration error  huge value error

CS248 Lecture 5Kurt Akeley, Fall 2007 Guard bits Solution: extend range to detect “wrapped” values n Add one or two “guard” MSBs n Non-zero guard bit(s)  out-of-range value n ‘Clamp’ out-of-range values to the nearer of zero or max

CS248 Lecture 5Kurt Akeley, Fall 2007 Guard-bit example ValueBinaryClamped ValueBinary (7) (6) Overflow Underflow 2-bit value, 1 guard bit

CS248 Lecture 5Kurt Akeley, Fall 2007 Guard-bit implementation ( n -bit) Out 0 In 0 Out 1 In 1 Out n -1 In n -1 In guard

CS248 Lecture 5Kurt Akeley, Fall 2007 DDA bit-assignment examples Pixel (12)Subpixel (10) Iteration (12) Guard (1) Depth (24) Iteration (13) Guard (2) Edge walk in 4k x 4k rendering space (35 bits) Depth walk in 4k x 4k rendering space (39 bits) (one extra for diagonal)

CS248 Lecture 5Kurt Akeley, Fall 2007 Barycentric (InfiniteReality) Hybrid algorithm n Approximate edge walk for fragment selection n Pineda edge functions used to generate AA masks n Direct barycentric evaluation for attribute assignment n Minimizes setup cost n Additional computational complexity accepted n Handles small triangles well Scissor cull implemented

CS248 Lecture 5Kurt Akeley, Fall 2007 Barycentric attribute evaluation (x 0, y 0, v 0 ) (x 1, y 1, v 1 ) (x 2, y 2, v 2 ) (x, y, v) a2a2 a1a1 a0a0

CS248 Lecture 5Kurt Akeley, Fall 2007 Small tiles (Bali – proposed) Frame buffer tiled into nxn (16x16) regions n Each tile is owned by one of k separate engines Two-level rasterization: n Tile selection (avoid broadcast, conservative) n Fragment selection and attribute assignment Parallelizes well Handles small triangles well Scissors well n At tile selection stage

CS248 Lecture 5Kurt Akeley, Fall 2007 Tiled fragment selection Tiled rasterization n 4x4 pixels per tile

CS248 Lecture 5Kurt Akeley, Fall 2007 Engine per pixel (Pixel Planes 4) Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

CS248 Lecture 5Kurt Akeley, Fall 2007 Engine-per-pixel (Pixel Planes 4) Individual direct-evaluation engine at every pixel ! n Solves edge equations to determine inclusion n Solves attribute equations to determine values Setup involves computation of plane and edge slopes Execution is in constant-time n Clever evaluation tree makes this possible n Extremely fast for large triangles, but n Extremely inefficient for small triangles n Effectively generates a fragment for every pixel n Scissor culling is a non-issue

CS248 Lecture 5Kurt Akeley, Fall 2007 Pixel Planes 4 fragment selection Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

CS248 Lecture 5Kurt Akeley, Fall 2007 Pixel Planes 4 fragment selection Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

CS248 Lecture 5Kurt Akeley, Fall 2007 Pixel Planes 4 fragment selection Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

CS248 Lecture 5Kurt Akeley, Fall 2007 Pixel Planes 4 attribute evaluation Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

CS248 Lecture 5Kurt Akeley, Fall 2007 Other approaches Homogeneous recursive descent n Rasterizes unprojected, unclipped geometry n Used by NVIDIA GPUs n Read Olano and Greer, Graphics Hardware 1997 Scan-line rasterization n Keep sorted list of primitives per scanline n Generate image directly (no frame buffer) Ray tracing …

CS248 Lecture 5Kurt Akeley, Fall 2007 Bresenham lines Developed by Jack Bresenham at IBM for pen plotters n Evolved over time, however Like DDA, but with no division required for setup n In a sense the division is done bit-by-bit as the line is generated. X- or Y-major iteration Limitations: n Original version does not handle subpixel vertexes n Error term cannot be used for pre-filter AA Defining property: one pixel per iteration step n Diagonal lines are ‘less bright’ n DDA can be used to adjust this

CS248 Lecture 5Kurt Akeley, Fall 2007 Bresenham lines Y-majorX-major Symmetric (or nearly so) Always one pixel per iteration along the major axis

CS248 Lecture 5Kurt Akeley, Fall 2007 Bresenham line pseudo-code // first octant (assumes 0 >1; // division by 2 while (x <= x1) { DrawFragment(x,y); x += 1; error -= dy; if (error < 0) { y += 1; error += dx; } }

CS248 Lecture 5Kurt Akeley, Fall 2007 Revisit pre-filtered antialiased lines 6 Line slope 4 X screen frac bits 4 Y screen frac bits 3 Pixel index 8 1M x 8 3 Line width Fragment Alpha For x-major line n Line slope from DDA delta fraction bits n X screen frac bits needed for end-points only n Y screen frac bits from DDA accumulation fraction bits

CS248 Lecture 5Kurt Akeley, Fall 2007 Summary Screen coordinates are continuous, not pixel addresses Rasterization converts primitives to fragments n Fragment selection: identify ‘covered’ pixels n Attribute evaluation: determine color, depth, … Modern 3-D graphics systems … n Point sample for fragment selection and attribute evaluation n Decompose polygons and quads to triangles n Prefer direct evaluation over iteration n Prefer floating-point to fixed-point representations

CS248 Lecture 5Kurt Akeley, Fall 2007 Assignments Before next Tuesday’s class, read n Paul Haeberli and Kurt Akeley, The Accumulation Buffer: Hardware Support for High-Quality Rendering, Proceedings of SIGGRAPH, pp , n Kurt Akeley, RealityEngine Graphics, Proceedings of SIGGRAPH, pp , n Optional: Marc Olano and Trey Greer, Triangle Scan Conversion Using 2D Homogeneous Coordinates, Proceedings of Graphics Hardware, pp , Project 1: n Demos tomorrow! n Sign up for a slot today

CS248 Lecture 5Kurt Akeley, Fall 2007 End