Under the Hood: 3D Pipeline

Motherboard & Chipset PCI Express x16

Discrete 3D Graphics Architecture CPU North Bridge North Bridge 3D GFX Local Memory Local Memory System Memory System Memory South Bridge South Bridge Display Devices Graphics Sub-System AGP / PCI-E Frame Buffer Z-Buffer Texture Vertex…

Discrete Graphics Cards NVIDIA ATI

Integrated 3D Graphics (Unified Memory Architecture) CPU NB + 3D GFX NB + 3D GFX Local Memory Local Memory System Memory System Memory South Bridge South Bridge Display Devices Graphics Sub-System AGP / PCI-E Display Devices Optional Frame Buffer Z-Buffer Texture Vertex… Local Memory Local Memory

Example: Intel Core i7 (1 st Generation)

Example: Intel Core i7 (2 nd Generation)

Keep in mind that… 3D graphics processor works in parallel to the CPU

Many Views of Graphics Pipeline Simple “Front-End/Back-End” view. Textbook version in [Foley/van Dam]. David Kirk’s (nVidia CTO) version presented in EG Hardware Workshop 1998: (slide 05) ons/kirk/index.htm ons/kirk/index.htm

Simplified View The Data Flow: 3D Polygons (+Colors, Lights, Normals, Texture Coordinates…etc.)  2D Polygons  2D Pixels (I.e., Output Images) Transform (& Lighting) Rasterization

Some different versions…

14 A Detailed 3D Graphics Pipeline

A Quick Review By default, graphic pipeline will do the following: 1)Take as input various per-vertex quantities (color, light source, eye point, texture coordinates, etc.) 2)Calculate a final color for each vertex using a basic lighting model (OpenGL uses Phong lighting) 3)For each pixel, linearly interpolate the three surrounding vertex colors to shade the pixel (OpenGL uses Gouraud shading) 4)Write the pixel color value to the frame buffer

Why Need Hardware All parts of graphics pipeline can be done in software. –But very slowly. –Example: mesaGL For some applications, speed is beauty –Games –Walkthrough –Visualization

Evolutions of Graphics Hardware 1.Gouraud-shaded polygons. 2.Then came antialiasing. 3.Then came texture mapping. 4.Now comes programmable shading.

Fixed vs. Programmable Starting in 1999 some graphics cards used the standard lighting model and Gouraud shading to draw polygon fragments entirely in hardware Implementing the pipeline in hardware made processing polygons much faster, but the developer could not modify the pipeline (hence “ fixed function pipeline ” ) New programmable hardware allows programmers to write vertex and pixel programs to change the pipeline

OpenGL Fixed Function Vertex Transform [MVP],[MV],[MV] -T Lighting [0,1] [0,1] Texgen Texture Matrixn Color SecondaryColor TexCoordn EdgeFlag Normal Vertex (object) TexCoordn EdgeFlag Vertex (eye) Vertex (clip) Front&Back Color Front&Back SecondaryColor

21 FrontFacing Color Coord Depth OpenGL Fixed Function Fragment Tex n TE n SumFog [0,1] Coord FrontFacing Color SecondaryColor TexCoord[n] zz e z (|z e |,f ) Depth

Programmable Shaders A concept made popular by Pixar’s RenderMan. First appeared in hardware: UNC PixelFlow –See SIGGRAPH papers by Molnar 1995 and Olano Made affordable by nVidia GeForce3 and XBox.

GL2 Vertex Processor Temporaries Vertex Shader Uniform Color SecondaryColor TexCoordn EdgeFlag Normal Vertex (object) TexCoordn EdgeFlag Vertex (eye) Vertex (clip) Front&Back Color Front&Back SecondaryColor

24 GL2 Fragment Processor TexCoord[n] FrontFacing zz e z (|z e |,f ) Coord FrontFacing Color Coord Color SecondaryColor Depth Temporaries Fragment Shader Uniform Texture

Rasterization

Triangles Only We will discuss the rasterization of triangles only. Why? –Polygon can be decomposed into triangles. –A triangle is always convex. –Results in algorithms that are more hardware friendly.

Being Hardware Friendly [Angel 5e] Section : –Intersect scan lines with polygon edges. –Sort the intersections, first by scan lines, then by order of x on each scan line. –It works for polygons in general, not just in triangles. –O(n log n) complexity  feasible in software implementation only (i.e., not hardware friendly)

Using Edge Equations Find the edge equations for P1-P2, P2-P3, P1-P3 For each pixel, test which sides of the edges it is at. Q: Do we need to test every pixel on the screen? P1 P2 P3 ege defined by Ax+By+c=0

Example P1 P2 P3 ege defined by Ax+By+C=0

Example Now we have P1-P3 defined by 2x+y-4=0 To test if a point (u,v) is inside the triangle, we calculate 2(u)+(v)-4. For example, (0,0) leads to 2(0)+(0)-4=-4 So the negative side is inside. Use P2 to decide which side is inside. Negate A,B,C if you prefer the positive side P1 P2 P3 Ax+By+C=0

Color and Z Now we know which pixels must be drawn. The next step is to find their colors and Z’s. Gouraud shading: linear interpolation of the vertex colors. Isn’t it straightforward? –Interpolate along the edges. (Y direction) –Then interpolate along the span. (X direction)

32 Attribute Derivation Color Interpolation (255, 0, 0) (0, 255, 0)(0, 0, 255) (255, 0, 0) (0, 0, 0) Red (0, 0, 0) (0, 255, 0) Green (0, 0, 255) (0, 0, 0) Blue

Interpolation in World Space vs Screen Space P1=(x1, y1, z1, c1); P2=(x2, y2, z2, c2); P3=(x3, y3, z3, c3) in world space If (x3, y3) = (1-t)(x1, y1) + t(x2, y2) then z3=(1-t)z1+t z2; c3=(1-t)c1+t c2 P1=(x1,y1,z1) P3=(x3,y3,z3) P2=(x2,y2,z2)

Interpolation in World Space vs Screen Space But, remember that we are interpolating on screen coordinates (x’, y’): P’2 P’1 P2 P1 P3 x’ y’

Let P’ 1 =(x’ 1, y’ 1 ); P’ 2 =(x’ 2, y’ 2 ) and P’ 3 =(x’ 3, y’ 3 )= (1-s)(x’ 1, y’ 1 ) + s(x’ 2, y’ 2 ) Does s=t? If not, should we compute z3 and c3 by s or t? Express s in t (or vice versa), we get something like: So, if we interpolate z on screen space, we get the z of some other point on the line This is OK for Z’s, but may be a problem for texture coordinates (topic of another lecture)

Interpolation OpenGL uses interpolation to find proper texels from specified texture coordinates

Perspectively Correct Interpolation The previous slide is not “perspectively correct” Consider the interpolation along the vertical (Y) direction. Should the checkers spaced equally? Use the s to t conversion shown earlier, then you get the perspectively correct interpolation.

Appendix

Derivation of s and t Two end points P 1 =(x 1, y 1, z 1 ) and P 2 =(x 2, y 2, z 2 ). Let P 3 =(1-t)P 1 +(t)P 2 After projection, P 1, P 2, P 3 are projected to (x’ 1, y’ 1 ), (x’ 2, y’ 2 ), (x’ 3, y’ 3 ) in screen coordinates. Let (x’ 3, y’ 3 )=(1-s)(x’ 1, y’ 1 ) + s(x’ 2, y’ 2 ).

(x’ 1, y’ 1 ), (x’ 2, y’ 2 ), (x’ 3, y’ 3 ) are obtained from P 1, P 2, P 3 by:

Since We have:

When P 3 is projected to the screen, we get (x’ 3, y’ 3 ) by dividing by w, so: But remember that (x’ 3, y’ 3 )=(1-s)(x’ 1, y’ 1 ) + s(x’ 2, y’ 2 ) Looking at x coordinate, we have

We may rewrite s in terms of t, w 1, w 2, x’ 1, and x’ 2. In fact, or conversely Surprisingly, x’ 1 and x’ 2 disappear.