1. Lecture 18 and 19 GPU and Programmable Shaders
CSc4820/6820 Computer Graphics Algorithms Ying Zhu Georgia State University
Lecture 18 and 19
GPU and
Programmable Shaders

2. Outline What is GPU? Vertex shaders Pixel shaders
How to develop shaders?
High level shading languages
How to control the GPU from the CPU?
3D API and shader interfaces

3. Acknowledgement
Some of the pictures and texts in the following slides are taken from
NVIDIA, ATI, and 3Dlabs’ presentation slides
Tom’s Hardware Guide (

4. What is GPU?
Graphics Processing Unit (GPU) is the microprocessor on a graphics card.
GPU has access to graphics memory on the graphics card
GPU is connected to the CPU and main memory via AGP or PCI-Express bus
NVIDIA GeForce 7800 GTX
ATI Radeon X1800

5. A little history
Before 1995, SGI was the dominate graphics hardware company
Flagship product: Onyx Reality Engine server
Implements 3D graphics pipeline on hardware
Very expensive
Operating system: IRIX
PC graphics cards at that time were rather simple (compared with today’s standard)
No hardware 3D pipeline
Almost all graphics operations are handled by CPU

6. GPU
The first popular 3D graphics card was the Voodoo card by 3DFX (1995)
Only texture mapping and Z-buffer were implemented on graphics card

7. GPU
Gradually, graphics card companies began to implement more and more 3D pipeline functions on graphics cards
By 2000, transformation and lighting were also implemented on graphics card (e.g. GeForce2)
Hardware 3D pipeline on graphics card is complete

8. Programmable GPU
In 2001, NVDIA introduced the first programmable GPU in GeForce3
Still implements a fixed-function 3D pipeline
Add support for vertex shader and texture shader
Can write your own transformation & lighting programs
Only supports assembly language shaders

9. Programmable GPU
In 2003, NVIDIA added pixel shader support in GeForceFX
Pixel shader replaces texture shader
New high level shading language: Cg and HLSL

10. Programmable GPU In 2004, NVIDIA releases GeForce 6800 and 6600
Vertex shaders can now access texture images
PCI-Express bus

11. Geometry shader
Geometry shaders are a new feature of DirectX 10 that allow for dynamic creation of geometry on the GPU.
With the introduction of geometry shaders, the GPU can now create and destroy vertices output from the vertex shader.
This capability allows for new features such as displacement mapping with tessellation and GPU-generated shadow volumes.
Tutorial

12. United Shader Model Introduced in 2006
Supported in GeForce 8xxx, Radeon HD 2000 series and Intel GMA X3000 series
Also known as Shader Model 4.0
Uses a unified Instruction Set Architecture across all shader types

13. Why unified shader model?
NVIDIA designed a single floating point shader core with multiple independent processors.
Each of these independent processors is capable of handling any type of shading operation, including pixel shading, vertex shading, geometry shading, and physics shading.
GeForce 8800 GPUs can dynamically allocate processing power depending on the workload of the application, providing unprecedented performance and efficiency.

14. Why unified shader model?

15. OpenGL 2.0 In 2004, OpenGL 2.0 is officially released
Added OpenGL Shading Language to support programmable GPU

16. Major GPU companies NVIDIA (www.nvidia.com) ATI (www.ati.com)
GeForce and Quadro series cards
High level shading language: Cg
ATI (
Radeon and FireGL series cards
3DLabs (
Wildcat series cards
High level shading language: OpenGL Shading Language (GLSL)

17. Why programmable GPU? Pre-GPU age: fixed function graphics pipeline
You can change the pipeline parameters in OpenGL/DirectX but not the algorithms.
Programmable GPU: gives freedom to developers
Applications can take control of the processing on graphics hardware
Never-before-seen effects are possible

18. Why programmable GPU?
Much better graphics performance on GPU than on CPU.
GPUs are designed for extreme speed
Highly parallel optimized hardware for graphics processing
CPUs are designed for flexibility
Engineered for wide variety of applications
For many graphics applications, real-time performance now becomes possible.
Non-graphics applications can also take advantage of the powerful GPU.

19. Other Uses of GPU Beyond computer graphics application
Use the powerful GPU as a second CPU
Speed up non-realtime graphics algorithms
Real-time ray tracing on GPU
Real-time photon mapping on GPU
Real-time radiosity on GPU
GPU-based volume rendering, etc.
Non-graphics applications
Fast Fourier transform
Interactive collision detection
Bioinformatic computing
More information at

20. Major Trends Faster GPUs Performance doubles every 6 - 9 months
Increased parallelism with each new generation of graphics processors
Increasing programmability of graphics hardware
Programmable Processing is the future of graphics and visual and cinematic computing
Enhanced vertex and pixel shader capabilities
Future shaders may have access to more 3D pipeline operations
E.g. clipping, depth test, stencil test, etc.

21. NVIDIA GeForce 7800 GTX

22. GeForce 8800 GTX
GeForce 8800 GTX architecture

23. ATI Radeon X1800

24. What’s in a GPU?
The latest GPUs still implements a fixed-function graphics pipeline
Controlled by OpenGL/Direct3D functions
But there are two programmable units that can take over part of the fixed-function pipeline
Vertex processor
Pixel (fragment) processor
Vertex and pixel processors are controlled by shaders

25. Vertex Shader
A vertex shader is a small program that manipulates vertex data and runs on GPU’s vertex unit
Provides developer with almost unlimited control on how vertices are perturbed.
Vertex shader can be written in high level shading language: Cg, HLSL, or GLSL

26. Vertex Shader in Graphics Pipeline
CPU
GPU

27. What is in a vertex shader?
Vertex Program
Reads an untransformed, unlit vertex
Creates a transformed vertex
Optional operations
Calculate lighting for vertex
Creates texture coordinates
Does not create or delete vertices
1 vertex in and 1 vertex out
No topological information provided
No edge, face, nor neighboring vertex info

28. What’s in a vertex shader?
You can implement the following operations in a vertex shader:
Multiply model-view and projection matrices with vertex coordinates
Transform normals to eye coordinates
Normalize normals
Calculate lighting for each vertex
Multiply texture matrices with texture coordinates
Access a texture map (shader model 3.0)

29. What’s not in a vertex shader?
Currently the following operations cannot be implemented by a vertex shader
Perspective division
Clipping
Viewport mapping
Front face determination
Flat shading or Gouraud shading

30. Why use vertex shader?
Write your own transformation and lighting function
Visual quality is much better than the fixed function Transformation & Lighting
Graphics developers have more control
Take advantage of the powerful GPU

31. Why use vertex shader?
Complete control of transform and lighting hardware
Complex vertex operations accelerated in hardware
Custom vertex lighting
Custom skinning and blending
Custom texture coordinate generation
Custom texture matrix operations
Custom vertex computations of your choice
Offloading vertex computations frees up CPU
More physics and simulation possible!

32. Vertex Shader Example
Custom transform, lighting, and skinning

33. Vertex Shader Example
Custom cartoon-style lighting

34. Vertex Shader Example
Per-vertex set up for per-pixel bump mapping

35. Example
Character morphing & shadow volume projection

36. Example
Dynamic displacements of surfaces by objects

37. Vertex shader
You have to load and compile vertex shaders in your OpenGL (or DirectX) program
Can load multiple vertex shaders
Vertex shaders can be turned on and off like a state in OpenGL program
E.g. turn on shader for some objects
If turned on, each vertex shader will be executed once per frame for each of the vertices on the affected objects
Your vertex shader may run hundreds of times per frame

38. What is pixel shader?
A pixel (fragment) shader is a small program which processes pixels and executes on the GPU’s pixel (fragment) processor
A pixel shader can be written in high level shading language: Cg, HLSL, or GLSL

39. Pixel Shader in Graphics Pipeline
CPU
GPU

40. What’s in a pixel shader?
Pixel shader implements the following rasterization operations
Texture mapping (including filtering, wrapping, texture environments, etc.)
Color sum
Fog
Discuss standard GL texture ops

41. Switch between fixed function and pixel shader

42. What’s not in a pixel shader?
Currently the following operations are not supported by pixel shader
Blending
All of the per-fragment operations

43. Why use pixel shader?
Fully control the texture mapping/blending process instead of using the fixed-function texture mapping.
Can generate much better quality images
E.g. Per-pixel lighting
Can also use pixel shader for non-graphics applications.

44. What can you do with pixel shader?
Per-pixel lighting
Looks much better than per-vertex lighting
True phong shading
Anisotropic lighting
Non-Photorealistic-Rendering
cartoon shading, hatching, gooch lighting, image space techniques
Per-pixel fresnel term
Volumetric effects
Advanced bump mapping
Procedural textures and texture perturbation
Bidirectional reflectance distribution functions
And more …

45. Pixel Shader Example: Natural Light

46. Pixel Shader Example: Reflection

47. Pixel Shader Example: Bump Mapping

48. Pixel shader
You have to load and compile pixel shaders in your OpenGL (or DirectX) program
Can load multiple pixel shaders
Pixel shaders can be turned on or off like a state in OpenGL program
E.g. turn on shader for some objects
If turned on, each pixel shader will be executed once per frame for each of the pixels on the affected objects
Your pixel shader may run hundreds of thousands of times per frame

49. How to develop shader programs?
High level shading languages Cg
Microsoft High Level Shader Language (HLSL)
OpenGL Shading Language
Supplementary tools
ATI’s RenderMonkey
NVIDIA CgFX Composer
OpenGL Shader Designer
Assembly language (?)

50. High level shading languagesCg
Developed by NVIDIA
Best supported by NVIDIA GeForce cards
Works on Windows, Linux, and Mac OS
Looks like C language but with some restrictions
Can be loaded in both OpenGL and DirectX programs

51. High level shading languages
MS HLSL
Part of DirectX 9.0 (
Jointly designed by MS and NVIDIA
Almost identical to Cg
Supported by GeForce and Radeon cards
Only runs on Windows
Only works with DirectX programs

52. High level shading languages
OpenGL Shading Language (GLSL):
Developed by 3Dlabs, approved by OpenGL ARB
Now part of the OpenGL 2.0 specification
Supported by latest GeForce and Radeon cards
Expected to be cross platform
Only works with OpenGL programs

53. Other high level language for GPUSh
A metaprogramming language for GPU
Developed at University of Waterloo
Brook for GPU
A stream program language for general purpose computing on GPU
Developed at Stanford University

54. Shader development tools
ATI’s RenderMonkey:
A shader development environment for both programmers and artists
Can be used for rapid prototyping shaders
Supports HLSL and GLSL
Runs best on ATI Radeon cards
Free

55. Shader development tools
NVIDIA FX Composer:
Similar to RenderMonkey
Save files in CgFX or HLSL FX format (A metaprogramming file format)
Runs best on GeForce cards
Only supports Cg
Free

56. Interface between shader and 3D API
Shaders have to be loaded and enabled during runtime by an OpenGL or DirectX program
OpenGL/DirectX program need to interact with shaders
How to create/load/delete shaders?
How to compile shaders?
How to enable/disable shaders?
How to pass data to shaders?

57. Interface between shader and 3D API
DirectX 9.0 has native functions to interface with both HLSL and Cg shaders
OpenGL–Cg interface
NVIDIA Cg runtime library
NVIDIA OpenGL extensions
OpenGL—GLSL interface
Newly added OpenGL 2.0 functions
(older) OpenGL extensions
OpenGL—CgFX interface
NVIDIA CgFX runtime library

58. Shader developers’ options
Write shaders in GLSL, use OpenGL 2.0 functions to load/execute shaders
Not many cards support this yet
Write shaders in HLSL (e.g. using RenderMonkey), use native DirectX functions to load/execute shaders
Windows only solution
Write shaders in Cg, use NVIDIA OpenGL extensions to load/execute shaders
OpenGL extensions are poorly documented

59. Shader developers’ options
Write shaders in Cg, in OpenGL/DirectX program use NVIDIA Cg runtime to load/execute shaders
Perhaps a better choice for now
Better cross-platform support
Easier to learn/use
Develop shaders in NVIDIA FX Composer, save them in CgFX files, in OpenGL/DirectX program load/execute shaders using CgFX runtime

60. Cg and Cg Runtime: a brief introduction
Cg language
A C-like language
Has built-in functions and predefined structures to simplify GPU programming
Cg runtime library
API that allow OpenGL/DirectX programs to load, compile and link, and execute Cg programs at runtime

61. How to use Cg shaders?
Use 3D API calls to specify vertex and pixel shaders
Enable vertex and fragment shaders
Load/enable texture as usual
Draw geometry as usual
Set blend state as usual
Vertex shader will execute for each vertex
Pixel shader will execute for each pixel

62. Cg language syntax Date types: float: 32 bit floating point
half: 16 bit floating point
fixed: 12 bit fixed point
bool
sampler: handle to texture map
struct
No pointers … yet

63. Different kinds of variables
Uniform variables
Same for each vertex (in a vertex shader)
Same for each pixel (in a pixel shader)
Examples: model-view matrix, light color
Varying variables
Different for each vertex (in a vertex shader)
Different for each pixel (in a pixel shader)
Semantic binding
Examples: position, normal, texture coord.
Local variables
Used for intermediate computations within vertex or pixel shader

64. Example
Semantic
binding

65. Abstracting data flow
For varying variables, need to use special keywords, called semantics, to tell Cg compiler witch GPU register to store data in or to read data from
Can use in, out, and inout modifiers to describe varying variables
in: varying variables that are inputs to a shader
out: varying variables that are outputs by a shader
inout: varying variables that are inputs but also written by the shader

66. Example

67. Array / vector / matrix

68. Function overloading

69. Support for vectors and matrices

70. New operators

71. A simple Cg vertex shader
struct appdata {
float4 position : POSITION;
float3 normal : NORMAL;
float3 color : DIFFUSE;
float3 TestColor : SPECULAR; };
struct vfconn
float4 HPOS : POSITION;
float4 COL0 : COLOR0;

72. A simple Cg vertex shader
vfconn main(appdata IN,
uniform float4 Kd,
uniform float4x4 ModelViewProj) {
vfconn OUT;
OUT.HPOS = mul(ModelViewProj, IN.position);
OUT.COL0.xyz = Kd.xyz * IN.TestColor.xyz;
OUT.COL0.w = 1.0;
return OUT;
} // main

73. Two ways to compile Cg Offline compilation Runtime compilation
Compile Cg code into assembly code
Assembly code is linked to OpenGL program at runtime
Runtime compilation
Cg program is compiled and linked at runtime.
Good future compatibility
No dependency limitations
Flexible input parameter management

74. What is Cg runtime?
An API that allow application to compile and link Cg programs at runtime
Three parts
Core Cg runtime
OpenGL Cg runtime
Direct3D Cg runtime
We’ll focus on Core Cg and OpenGL Cg runtime

75. Where are the files?
Cg runtime comes with Cg Toolkit (
Normally under C:\Program Files\NVIDIA Corporation\Cg
Cg.dll, cgGL.dll, cgD3D9.dll, etc.
Cg.lib, cgGL.lib, cgD3D9.lib, etc.
Cg.h, cgGL.h, cgD3D9.h, etc.
Download and install Cg Toolkit
Make sure Cg runtime header files and lib files are on VC++’s search path
Make sure Cg runtime DLL files are on system search path

76. Basic Cg runtime concepts
Context
A container for multiple Cg programs
An application may have multiple contexts
Program
Compile a Cg program by adding it to a context
Parameters
A program has several parameters

77. Cg Language Profiles
Different GPUs support different set of graphics capabilities
A Cg profile defines a subset of full Cg language that is supported on a particular hardware platform or API
CG_PROFILE_VP30, CG_PROFILE_FP30, etc.
A full list can found on Cg User’s Manual
A Cg program has to be compiled for a particular profile

78. Basic Steps of using Cg Runtime
In your OpenGL program, do the following
Create Cg context(s)
Initialize profiles
Compile Cg program(s)
Load Cg program(s)
(a) Modify Cg program parameters
(b) Enable profile(s)

79. Basic Steps of using Cg Runtime
(c) Bind a Cg program
(d) Draw 3D object(s)
(e) Disable the profile
Repeat (a) – (e) if necessary
Destroy Cg program(s)
Destroy Cg context(s)

80. Create a Context
You have to create at least one context if you want to use Cg
It’s the first to be created and the last to be destroyed
CGcontext context = cgCreateContext();

81. Get Cg profiles
CGprofile vertexProfile = cgGLGetLatestProfile(CG_GL_VERTEX)
CGprofile fragmentProfile = cgGLGetLatestProfile(CG_GL_FRAGMENT)

82. Compiling a program
CGprogram vertexProgram = cgCreateProgram(context, CG_SOURCE, myVertexProgramString, vertexProfile, “main”, args);

83. Loading a program Pass the compiled object code to OpenGL
cgGLLoadProgram(vertexProgram)

84. Modify Cg Program Parameters
You need to set Cg program parameters before its execution
Get Cg program parameters
CGparameter baseTexture = cgGetNamedParameter(fragmentProgram, “baseTexture”)
Set uniform parameters that don’t change during program execution
cgGLSetTextureParameter(baseTexture, texture)
cgGLSetParameter4fv(someColor, constantColor)

85. Modify Cg Program Parameters
Set the varying parameters
cgGLEnableClientState(texCoord);
cgGLSetParameterPointer(texCoord, 2, GL_FLOAT, 0, vertexTexCoords);
Set the uniform parameters that may change every frame
cgGLSetStateMatrixParameter(modelViewMatrix, CG_GL_MODELVIEW_PROJECT_MATRIX, CG_GL_MATRIX_IDENTITY);

86. Enable the profiles cgGLEnableProfile(vertexProfile)
cgGLEnableProfile(fragmentProfile)

87. Bind the programs
You can bind only one vertex program and one fragment program at a time.
cgGLBindProgram(vertexProgram)
cgGLBindProgram(fragmentProgram)

88. Disable the profiles
This automatically unbind the corresponding vertex and/or fragment program
cgGLDisableProfile(vertexProfile);
cgGLDsiableProfile(fragmentProfile);

89. Destroy Cg program Release the resources allocated to the Cg program
cgDestroyProgram(vertexProgram)

90. Destroy Context Release the resources allocated to the Cg context
cgDestroyContext(context);

91. Cg Runtime Example Example from Cg User’s Manual, page 57 – 60
Vertex
shader

92. Cg Runtime Example (cont.)
Pixel
shader

93. Cg Runtime Example (cont.)
OpenGL program

94. Cg Runtime Example (cont.)

95. Cg Runtime Example (cont.)

96. Cg Runtime Example (cont.)

97. Examples A complete Cg/Cg runtime example in “Hello, Cg!”
More examples comes with Cg Toolkit
Installed under C:\Program Files\NVIDIA Corporation\Cg\examples\runtime_ogl
cgGL_vertex_example.c
cgGL_vertex_example.cg

98. Summary
Cg runtime is a bridge between Cg program and the host OpenGL/D3D application
Quite straightforward to learn/use
Perhaps the more difficult part is to learn how to get/set various Cg program parameters
A good understanding of various Cg profiles are also important

99. Summary
Programmable GPU is the biggest revolution in graphics in 10 years and the foundation for the next 10
Use shading languages to write vertex or pixel shaders that run on GPU
OpenGL Shading Language, Cg, High Level Shading Language, etc.

100. Readings “Introduction to the hardware graphics pipeline”
Hello, Cg!
“Cg in two pages” ( )
(Optional) Cg User’s Manual

101. GPU references NVIDIA technical presentations
ATI technical presentations
3Dlabs technical presentations

102. Cg and Cg runtime References
Cg toolkit
Cg User’s Manual 1.5
Comes with Cg Toolkit
A thorough discussion of Cg and Cg runtime.
Appendix A: Cg language specification
“The Cg Tutorial”, by R. Fernando and M. J. Kilgard, Addison-Wesley, 2003

103. GLSL references OpenGL 2.0 specification
GLSL Introductions & GLSL master class
“OpenGL Shading Language”, by R. J. Rost, 2nd edition, Addison-Wesley, 2006

104. Next lecture
Overview of modeling