1. Real-Time Rendering
靜宜大學資工研究所
蔡奇偉副教授
2010©

2. 大綱 Introduction Graphics Rendering Pipeline Architecture
Application Stages
Geometry Stages
Rasterizer Stages

3. What are the results that are achievable with Computer Graphics?
Tomas Akenine-Mőller © 2002

4. State-of-the-Art Real-Time Rendering 2001
Tomas Akenine-Mőller © 2002

5. State-of-the-Art Real-Time Rendering 2001
Tomas Akenine-Mőller © 2002

6. Which one is a real photo, and which one is CG?
Tomas Akenine-Mőller © 2002

7. Complexity: 8 million triangles…
Tomas Akenine-Mőller © 2002

8. Photo-realistic Rendering…
Tomas Akenine-Mőller © 2002

9. Introduction
Real-time rendering is concerned with making images rapidly on the computer.
The rate at which images are displayed is measured in frames per second (fps) or Hertz (Hz).
Usually between 15 fps and 72 fps.
Interactivity, 3D rendering, graphics acceleration hardware.
Include applications such as computer games, CAD, 3D animation software (3D Max or Maya), …, etc.

10. Graphics Rendering Pipeline
The main function of the pipeline is to generate, or render, a two-dimensional image, given a virtual camera, 3D objects, light sources, shading equations, textures, and more.

11. Pipeline Architecture
A pipeline consists of several stages. The speed of the pipeline is determined by the slowest stage, no matter how fast the other stages may be.
conceptual stages
functional stages
pipeline stages

12. Conceptual Stages
Application
Geomrtry
Rasterizer

13. The APPLICATION stage Executed on the CPU Examples:
Geometry
Rasterizer
The APPLICATION stage
Executed on the CPU
Means that the programmer decides what happens here
Examples:
Collision detection
Speed-up techniques
Animation
Most important task: send rendering primitives (e.g. triangles) to the graphics hardware
Tomas Akenine-Mőller © 2002

14. Rendering Primitives Use graphics hardware for real time…
These can render points, lines, triangles.
A surface is thus an approximation by a number of such primitives.
Tomas Akenine-Mőller © 2002

15. You say that you render a ”3D scene”, but what is it?
First, of all to take a picture, it takes a camera – a virtual one.
Decides what should end up in the final image
A 3D scene is:
Geometry (triangles, lines, points, and more)
Light sources
Material properties of geometry
Textures (images to glue onto the geometry)
A triangle consists of 3 vertices
A vertex is 3D position, and may include normals and more.
Tomas Akenine-Mőller © 2002

16. Virtual Camera
Defined by position, direction vector, up vector, field of view, near and far plane.
point
dir
near
far
fov
(angle)
Create image of geometry inside gray region
Used by OpenGL, DirectX, ray tracing, etc.
Tomas Akenine-Mőller © 2002

17. Application
Geometry
Rasterizer
The GEOMETRY stage
Task: ”geometrical” operations on the input data (e.g. triangles)
Allows:
Move objects (matrix multiplication)
Move the camera (matrix multiplication)
Compute lighting at vertices of triangle
Project onto screen (3D to 2D)
Clipping (avoid triangles outside screen)
Map to window
Tomas Akenine-Mőller © 2002

18. Animate objects and camera
Application
Geometry
Rasterizer
Animate objects and camera
Can animate in many different ways with 4x4 matrices
Example:
Before displaying a torus on screen, a matrix that represents a rotation can be applied. The result is that the torus is rotated.
Same thing with camera (this is possible since motion is relative)
Tomas Akenine-Mőller © 2002

19. Application
Geometry
Rasterizer
The RASTERIZER stage
Main task: take output from GEOMETRY and turn into visible pixels on screen
Also, add textures and various other per-pixel operations
And visibility is resolved here: sorts the primitives in the z-direction
Tomas Akenine-Mőller © 2002

20. Application
Geometry
Rasterizer
In Summery
The programmer sends down primtives to be rendered through the pipeline (using API calls)
The geometry stage does per-vertex operations
The rasterizer stage does per-pixel operations
Next, scrutinize geometry and rasterizer
Tomas Akenine-Mőller © 2002

21. The GEOMETRY stages Application Geometry Rasterizer
Tomas Akenine-Mőller © 2002

22. GEOMETRY Model Transformation
Application
Geometry
Rasterizer
GEOMETRY Model Transformation
Originally, an object is in model space
Move, orient, and transform geometrical objects into world space
Example, a sphere is defined with origin at (0,0,0) with radius 1
Translate, rotate, scale to make it appear elsewhere
Done per vertex with a 4x4 matrix multiplication!
The user can apply different matrices over time to animate objects
Tomas Akenine-Mőller © 2002

23. GEOMETRY View Transform
Application
Geometry
Rasterizer
GEOMETRY View Transform
You can move the camera in the same manner
But apply inverse transform to objects, so that camera looks down negative z-axis
Tomas Akenine-Mőller © 2002

24. GEOMETRY Vertex Shading
Application
Geometry
Rasterizer
GEOMETRY Vertex Shading
shading equation
Materials + Lighting Shading
Compute color at vertices
Try to mimic how light in nature behaves
Hard to uses empirical models, hacks, and some real theory
light
Geometry
blue
red
green
Rasterizer

25. GEOMETRY Projection Two major ways to do it Orthogonal Perspective
Application
Geometry
Rasterizer
GEOMETRY Projection
Two major ways to do it
Orthogonal
Perspective
Mimics how humans perceive the world, i.e., objects’ apparent size decreases with distance
Tomas Akenine-Mőller © 2002

26. GEOMETRY Projection Also done with a matrix multiplication!
Application
Geometry
Rasterizer
GEOMETRY Projection
Also done with a matrix multiplication!
Pinhole camera (left), analog used in CG (right)
Tomas Akenine-Mőller © 2002

27. GEOMETRY Clipping Square (cube) after projection
Application
Geometry
Rasterizer
GEOMETRY Clipping
Square (cube) after projection
Clip primitives to square

28. GEOMETRY Screen Mapping
Application
Geometry
Rasterizer
GEOMETRY Screen Mapping
Screen mapping, scales and translates square so that it ends up in a rendering window
These screen space coordinates together with Z (depth) are sent to the rasterizer stage

29. GEOMETRY Summary camera space world space model space world space
Application
Geometry
Rasterizer
GEOMETRY Summary
camera space
world space
model space
world space
compute lighting
projection
image space
clip
map to screen
Tomas Akenine-Mőller © 2002

30. The RASTERIZER stages Application Geometry Rasterizer
Tomas Akenine-Mőller © 2002

31. The RASTERIZER Triangle Setup
Application
Geometry
Rasterizer
The RASTERIZER Triangle Setup
In this stage the differentials and other data for the triangle's surface are computed. This data is used for scan conversion, as well as for interpolation of the various shading data produced by the geometry stage. This process is performed by fixed-operation hardware dedicated to this task.
Tomas Akenine-Mőller © 2002

32. The RASTERIZER Triangle Traversal (Scan Conversion)
Application
Geometry
Rasterizer
The RASTERIZER Triangle Traversal (Scan Conversion)
Find pixels inside the triangle
Or on a line, or on a point
Do per-pixel operations on these pixels:
Interpolation
Texturing
Z-buffering
And more…
Tomas Akenine-Mőller © 2002

33. The RASTERIZER Interpolation
Application
Geometry
Rasterizer
The RASTERIZER Interpolation
Interpolate colors over the triangle
Called Gouraud interpolation
blue
red
green
Tomas Akenine-Mőller © 2002

34. The RASTERIZER Pixel Shading
Application
Geometry
Rasterizer
The RASTERIZER Pixel Shading
Using the interpolated shading data as input
The end result is one or more colors to be passed on to the next stage
is executed by programmable GPU cores
Texturing is one of the most important techniquies

35. The RASTERIZER Texturing
Application
Geometry
Rasterizer
The RASTERIZER Texturing
One application of texturing is to glue images onto geometrical object
Uses and other applications
More realism
Bump mapping
Pseudo reflections
Store lighting
Almost infinitely many uses + =

36. Example of texturing

37. The RASTERIZER Merging
Application
Geometry
Rasterizer
The RASTERIZER Merging
Pixels are stored in color buffer
Combine the fragment color with the pixel color
Highly configurable
Resolving visibility (done with Z-buffer)
alpha channel for transparency
stencil buffer for masking
accumulation buffer for motion blur, depth of field, soft shadows, or antialiasing

38. The RASTERIZER Z-buffering
Application
Geometry
Rasterizer
The RASTERIZER Z-buffering
The graphics hardware is pretty stupid
It just draws triangles
However, a triangle that is covered by a more closely located triangle should not be visible
Assume two equally large tris at different depths
Triangle 1
Triangle 2
Draw 1 then 2
incorrect
Draw 2 then 1
correct

39. The RASTERIZER Z-buffering
Application
Geometry
Rasterizer
The RASTERIZER Z-buffering
Would be nice to avoid sorting…
The Z-buffer (aka depth buffer) solves this
Idea:
Store z (depth) at each pixel
When scan-converting a triangle, compute z at each pixel on triangle
Compare triangle’s z to Z-buffer z-value
If triangle’s z is smaller, then replace Z-buffer and color buffer
Else do nothing
Can render in any order
Have difficulty to handle transparancy
Tomas Akenine-Mőller © 2002

40. The RASTERIZER Double Buffering
Application
Geometry
Rasterizer
The RASTERIZER Double Buffering
The monitor displays one image at a time
So if we render the next image to screen, then rendered primitives pop up
And even worse, we often clear the screen before generating a new image
A better solution is double buffering
Tomas Akenine-Mőller © 2002

41. The RASTERIZER Double Buffering
Application
Geometry
Rasterizer
The RASTERIZER Double Buffering
Use two buffers: one front and one back
The front buffer is displayed
The back buffer is rendered to
When new image has been created in back buffer, swap front and back
Tomas Akenine-Mőller © 2002

42. Double Buffering
Front Buffer 儲存螢幕目前顯示的內容，程式計算出的下一個畫面則先寫入 Back Buffer 中。
Copy
更新畫面時，在垂直回歸時間內，把 Back Buffer 拷貝至 Front Buffer：
back
front
程式寫入
顯示在螢幕上
back
front 拷貝
(vertical retrace)
back
front
顯示在螢幕上

43. register 的值是在垂直回歸時才更新。
Flipping
目前的顯示卡通常都具有緩衝區交換的硬體功能，可以避免拷貝而很有效率地把 Front Buffer 和 Back Buffer 互換：
back
front
程式寫入
顯示在螢幕上
register A B
register 的值是在垂直回歸時才更新。
front
back
程式寫入
顯示在螢幕上
register A B

44. And, lately… Programmable shading has become a hot topic
Vertex shaders
Pixel shaders
Adds more control and much more possibilities for the programmer
Application
Geometry
Rasterizer
HARDWARE
Vertex shader
program
Pixel shader