1. The Rendering Pipeline
CS 490: Computer Graphics
Chapter 7:
The
Rendering
Pipeline

2. Interactive Computer Graphics
Overview
Geometric processing: normalization, clipping, hidden surface removal, lighting, projection (front end)
Rasterization or scan conversion, including texture mapping (back end)
Fragment processing and display

3. Interactive Computer Graphics
Geometric Processing
Front-end processing steps (3D floating point; may be done on the CPU)
Evaluators (converting curved surfaces to polygons)
Normalization (modeling transform, convert to world coordinates)
Projection (convert to screen coordinates)
Hidden-surface removal (object space)
Computing texture coordinates
Computing vertex normals
Lighting (assign vertex colors)
Clipping
Perspective division (convert to 3 Diamension)
Backface culling

4. Interactive Computer Graphics
Rasterization
Back-end processing works on 2D objects in screen coordinates
Processing includes
Scan conversion of primitives including shading
Texture mapping
Fog
Scissors test
Alpha test
Stencil test
Depth-buffer test
Other fragment operations: blending, dithering, logical operations

5. Display Convert frame buffer to video signal Other considerations:
Interactive Computer Graphics
Display
Convert frame buffer to video signal
Other considerations:
Color correction
Antialiasing

6. Geometric Transformations
Interactive Computer Graphics
Geometric Transformations
Five coordinate systems of interest:
Object coordinates
Eye (world) coordinates [after modeling transform, viewer at the origin]
Clip coordinates [after projection]
Normalized device coordinates [after ÷w]
Window (screen) coordinates [scale to screensize]

7. Line-Segment Clipping
Interactive Computer Graphics
Line-Segment Clipping
Clipping may happen in multiple places in the pipeline (e.g. early trivial accept/reject)
After projection, have lines in plane, with rectangle to clip against

8. Clipping a Line Against xmin
Interactive Computer Graphics
Clipping a Line Against xmin
Given a line segment from (x1,y1) to (x2,y2), Compute m=(y2–y1)/(x2–x1)
Line equation: y = mx + h
h = y1 – m x1 (y intercept)
Plug in xmin to get y
Check if y is between y1 and y2.
This take a lot of floating-point math. How to minimize?

9. Cohen-Sutherland Clipping
Interactive Computer Graphics
Cohen-Sutherland Clipping
For both endpoints compute a 4-bit outcode (o1, o2) depending on whether coordinate is outside cliprect side
Some situations can be handled easily

10. Cohen-Sutherland Conditions
Interactive Computer Graphics
Cohen-Sutherland Conditions
Cases.
1. If o1=o2=0, accept
2. If one is zero, one nonzero, compute an intercept. If necessary compute another intercept. Accept.
3. If o1 & o2  0. If both outcodes are nonzero and the bitwise AND is nonzero, two endpoints lie on same outside side. Reject.
3. If o1 & o2  0. If both outcodes are nonzero and the bitwise AND is zero, may or may not have to draw the line. Intersect with one of the window sides and check the result.

11. Cohen-Sutherland Results
Interactive Computer Graphics
Cohen-Sutherland Results
In many cases, a few integer comparisons and Boolean operations suffice.
This algorithm works best when there are many line segments, and most are clipped
But note that the y=mx+h form of equation for a line doesn’t work for vertical lines (need a special case in your code)

12. Polygon Clipping Clipping a polygon can result in lots of pieces
Interactive Computer Graphics
Polygon Clipping
Clipping a polygon can result in lots of pieces
Replacing one polygon with many may be a problem in the rendering pipeline
Could treat result as one polygon: but this kind of polygon can cause other difficulties
Some systems allow only convex polygons, which don’t have such problems (OpenGL has tessellate function in glu library)

13. Sutherland-Hodgeman Polygon Clipping
Interactive Computer Graphics
Sutherland-Hodgeman Polygon Clipping
Could clip each edge of polygon individually
Pipelined approach: clip polygon against each side of rectangle in turn
Treat clipper as “black box” pipeline stage

14. Clipping Pipeline Clip each bound in turn
Interactive Computer Graphics
Clipping Pipeline
Clip each bound in turn

15. Interactive Computer Graphics
Clipping in Hardware
Build the pipeline stages in hardware so you can perform four clipping stages at once
[A partial answer to the question of what to do with all that chip area, 1 billion + transistors…]

16. Clipping complicated objects
Interactive Computer Graphics
Clipping complicated objects
Suppose you have many complicated objects, such as models of parts of a person with thousands of polygons each
When and how to clip for maximum efficiency?
How to clip text? Curves?

17. Clipping Other Primitives
Interactive Computer Graphics
Clipping Other Primitives
It may help to clip more complex shape early in the pipeline
This may be simpler and less accurate
One approach: bounding boxes (sometimes called trivial accept-reject)
This is so useful that modeling systems often store bounding box

18. Interactive Computer Graphics
Clipping Curves, Text
Some shapes are so complex that they are difficult to clip analytically
Can approximate with line segments
Can allow the clipping to occur in the frame buffer (pixels outside the screen rectangle aren’t drawn)
Called “scissoring”
How does performance compare?

19. Hidden surface removal
Interactive Computer Graphics
Hidden surface removal
Object space vs. Image space
The main image-space algorithm: depth buffer
Drawbacks
Aliasing
Rendering invisible objects
Doesn’t handle transparency correctly
How would object-space hidden surface removal work?

20. Depth sorting The painter’s algorithm: draw from back to front
Interactive Computer Graphics
Depth sorting
The painter’s algorithm: draw from back to front
Depth-sort hidden surface removal:
sort display list by z-coordinate from back to front
render
Drawbacks
it takes some time (especially with bubble sort!)
it doesn’t work

21. Depth-sort difficulties
Interactive Computer Graphics
Depth-sort difficulties
Polygons with overlapping projections.
Cyclic overlap
Interpenetrating polygons
What to do?

22. Interactive Computer Graphics
Scan Conversion
At this point in the pipeline, we have only polygons and line segments. Render!
To render, convert to pixels (“fragments”) with integer screen coordinates (ix, iy), depth, and color
Send fragments into fragment-processing pipeline

23. Rendering Line Segments
Interactive Computer Graphics
Rendering Line Segments
How to render a line segment from (x1, y1) to (x2, y2)?
Use the equation y = mx + h
What about horizontal vs. vertical lines?

24. DDA Algorithm DDA: Digital Differential Analyzer
Interactive Computer Graphics
DDA Algorithm
DDA: Digital Differential Analyzer
for (x=x1; x<=x2; x++)
y += m;
draw_pixel(x, round(y,) color)
Handle slopes 0 <= m <= 1; handle others symmetrically
Does this need floating point math?

25. Bresenham’s Algorithm
Interactive Computer Graphics
Bresenham’s Algorithm
The DDA algorithm requires a floating point add and round for each pixel: eliminate?
Note that at each step we will go E or NE. How to decide which? −

26. Bresenham Decision Variable
Interactive Computer Graphics
Bresenham Decision Variable
Note that at each step we will go E or NE. How to decide which?
Hint: consider decision variable d=a-b, where a and b are distances to NE and E pixels −

27. Bresenham Decision Variable
Interactive Computer Graphics
Bresenham Decision Variable
Bresenham algorithm uses decision variable d=a-b, where a and b are distances to NE and E pixels
If d<=0, go NE; lower if d>0, go E upper
Let dx = x2-x1, dy=y2-y1
Use decision variable d = dx (a-b) [only sign matters] −

28. Bresenham Decision Variable
Interactive Computer Graphics
Bresenham Decision Variable
d =(a-b) dx
Let dk be the value of d at x = k + ½
Move E:
dk = dx(a-b) = dx((j+3/2–yk) – (yk–(j+1/2)))
dk+1 = dx(a-b) = dx((j+3/2–yk–m) – (yk+m–(j+1/2)))
dk+1 – dk = dx (-2m) = - 2 dY
Algorithm:
dk+1 = dk – 2 dy (if dk > 0) (last move was E)
dk+1 = dk – 2 (dy-dx) (if dk <= 0) (last move was NE) −

29. Bresenham’s Algorithm
Interactive Computer Graphics
Bresenham’s Algorithm
Set up loop computing d at x1, y1
for (x=x1; x<=x2; )
x++;
d += 2dy;
if (d >= 0) {
y++;
d –= 2dx; }
drawpoint(x,y);
Pure integer math, and not much of it
So easy that it’s usually implemented in one graphics instruction for several points in parallel

30. Interactive Computer Graphics
Rasterizing Polygons
Polygons may be or may not be simple, convex, flat. How to render?
Amounts to inside-outside testing: how to tell if a point is in a polygon?

31. Interactive Computer Graphics
Winding Test
Most common way to tell if a point is in a polygon: the winding test.
Define “winding number” w for a point: signed number of revolutions around the point when traversing boundary of polygon once
When is a point “inside” the polygon?

32. OpenGL and Complex Polygons
Interactive Computer Graphics
OpenGL and Complex Polygons
OpenGL guarantees correct rendering only for simple, convex, planar polygons
OpenGL tessellates concave polygons
Tessellation depends on winding rule you tell OpenGL to use: Odd, Nonzero, Pos, Neg, ABS_GEQ_TWO

33. Interactive Computer Graphics
Winding Rules

34. Scan-Converting a Polygon
Interactive Computer Graphics
Scan-Converting a Polygon
General approach: ideas?
One idea: flood fill
Draw polygon edges
Pick a point (x,y) inside and flood fill with DFS  Depth First Seach 
flood_fill(x,y) {
if (read_pixel(x,y)==white) {
write_pixel(x,y,black);
flood_fill(x-1,y);
flood_fill(x+1,y);
flood_fill(x,y-1);
flood_fill(x,y+1);
} }

35. Interactive Computer Graphics
Scan-Line Approach
More efficient: use a scan-line rasterization algorithm
For each y value, compute x intersections. Fill according to winding rule
How to compute intersection points?
How to handle shading?
Some hardware can handle multiple scanlines in parallel

36. Interactive Computer Graphics
Singularities
If a vertex lies on a scanline, does that count as 0, 1, or 2 crossings?
How to handle singularities?
One approach: don’t allow. Perturb vertex coordinates
OpenGL’s approach: place pixel centers half way between integers (e.g. 3.5, 7.5), so scanlines never hit vertices

37. Aliasing How to render the line with reduced aliasing?
Interactive Computer Graphics
Aliasing
How to render the line with reduced aliasing?
What to do when polygons share a pixel?

38. Anti-Aliasing Simplest approach: area-based weighting
Interactive Computer Graphics
Anti-Aliasing
Simplest approach: area-based weighting
Fastest approach: averaging nearby pixels
Most common approach: supersampling (compute four values per pixel and avg, e.g.)
Best approach: weighting based on distance of pixel from center of line; Gaussian fall-off

39. Temporal Aliasing Need motion blur for motion that doesn’t flicker
Interactive Computer Graphics
Temporal Aliasing
Need motion blur for motion that doesn’t flicker
Common approach: temporal supersampling
render images at several times within frame time interval
average results

40. Display Considerations
Interactive Computer Graphics
Display Considerations
Color systems
Color quantization
Gamma correction
Dithering and Halftoning

41. Additive and Subtractive Color
Interactive Computer Graphics
Additive and Subtractive Color

42. Interactive Computer Graphics
Common Color Models

43. Color Systems RGB YIQ CMYK HSV, HLS Chromaticity Color gamut
Interactive Computer Graphics
Color Systems
RGB
YIQ
CMYK
HSV, HLS
Chromaticity
Color gamut

44. HLS Hue: “direction” of color: red, green, purple, etc.
Interactive Computer Graphics
HLS
Hue: “direction” of color: red, green, purple, etc.
Saturation: intensity. E.g. red vs. pink
Lightness: how bright
To the right: original, H, S, L

45. YIQ Used by NTSC TV Y = luma, same as black and white
Interactive Computer Graphics
YIQ
Used by NTSC TV
Y = luma, same as black and white
I = in-phase (red luma)
Q = quadrature (blue luma)
The eye is more sensitive to blue-orange than purple-green, so more bandwidth is allotted
Y = 4 MHz, I = 1.3 MHz, Q = 0.4 MHz, overall bandwidth 4.2 MHz
Linear transformation from RBG:
Y = R G B
I = R – G – B
Q = R – G B

46. Chromaticity Interactive Computer Graphics
Color researchers often prefer chromaticity coordinates (tristimulus values):
t1 = T1 / (T1 + T2 + T3)
t2 = T2 / (T1 + T2 + T3)
t3 = T3 / (T1 + T2 + T3)
Thus, t1+t2+t3 = 1.0.
Use t1 and t2; t3 can be computed as 1-t1-t2
an objective specification of the quality of a color regardless of its luminance. 
Chromaticity diagram uses XYZ color system based on human perception experiments
Y, luminance
X, redness (roughly)
Z, blueness (roughly)

47. CIE 1931 chromaticity diagram
Interactive Computer Graphics
Color temperature
Compute color temperature by comparing chromaticity with that of an ideal black-body radiator
Color temperature is that were the headed black-body radiator matches color of light source
Higher temperatures are “cooler” colors – more green/blue; warmer colors (yellow-red) have lower temperatures
CIE 1931 chromaticity diagram
(wikipedia)

48. Interactive Computer Graphics
Halftoning
How do you render a colored image when colors can only be on or off (e.g. inks, for print)?
Halftoning: dots of varying sizes
[But what if only fixed-sized pixels are available?]

49. Interactive Computer Graphics
Dithering
Dithering (patterns of b/w or colored dots) used for computer screens
Dither is an intentionally applied form of noise used to randomize quantization error
OpenGL can dither
But, patterns can be visible and bothersome. A better approach?

50. Floyd-Steinberg Error Diffusion Dither
Interactive Computer Graphics
Floyd-Steinberg Error Diffusion Dither
is an image dithering algorithm
Spread out “error term”
7/16 right
3/16 below left
5/16 below
1/16 below right
Note that you can also do this for color images (dither a color image onto a fixed 256-color palette)

51. Interactive Computer Graphics
Color Quantization
Color quantization: modifying a full-color image to render with a 256-color palette
For a fixed palette (e.g. web-safe colors), can use closest available color, possibly with error-diffusion dither
Algorithm for selecting an adaptive palette?
E.g. Heckbert Median-Cut algorithm
Make a 3-D color histogram
Recursively cut the color cube in half at a median
Use average color from each resulting box

52. Hardware Implementations
Interactive Computer Graphics
Hardware Implementations
Pipeline architecture for speed (but what about latency?)
Originally, whole pipeline on CPU
Later, back-end on graphics card
Then, whole pipeline in hardware on graphics card
Then, parts of pipeline done on GPUs(graphics processor unit)
Now, hardware pipeline is gone—done in software on a large number of GPUs

53. Interactive Computer Graphics
Future Architectures?
20 years ago, performance of 1 M polygons per second cost millions
Performance limited by memory bandwidth
Main component of price was lots of memory chips
Now an iPhone 5 does 115 million triangles (1.8 billion texels) per second
Fastest performance today achieved with many cores on each graphics chip (and several chips in parallel)
How to use parallel graphics cores/chips?
All vertices go through vertex shaders – can do in parallel
All fragments go through fragment shaders – can do in parallel
Resulting fragments combined into a single image – easy with sufficient memory bandwidth

54. NVidia GeForce GTX 590 Facts
Interactive Computer Graphics
NVidia GeForce GTX 590
Facts
“Most powerful graphics card ever built” (of 2011)
Dual 512-core GTX 500 GPUs (1024 CUDA unified shaders)
2 x 16 Streaming Multiprocessor (SM) processor groups, each with 32 Streaming Processors (SPs)
PhysX physics engine on the card (used for flowing clothing, smoke, debris, turbulence, etc)
Memory bandwidth: GB/sec
Texture fill rate: 77.7 billion/sec
4X antialiasing
3 billion transistors x 2
Real-time ray tracing
Power: 365 w