Lecture 18 and 19 GPU and Programmable Shaders CSc4820/6820 Computer Graphics Algorithms Ying Zhu Georgia State University Lecture 18 and 19 GPU and Programmable Shaders

Outline What is GPU? Vertex shaders Pixel shaders How to develop shaders? High level shading languages How to control the GPU from the CPU? 3D API and shader interfaces

Acknowledgement Some of the pictures and texts in the following slides are taken from NVIDIA, ATI, and 3Dlabs’ presentation slides Tom’s Hardware Guide (http://www.tomshardware.com/)

What is GPU? Graphics Processing Unit (GPU) is the microprocessor on a graphics card. GPU has access to graphics memory on the graphics card GPU is connected to the CPU and main memory via AGP or PCI-Express bus NVIDIA GeForce 7800 GTX ATI Radeon X1800

A little history Before 1995, SGI was the dominate graphics hardware company Flagship product: Onyx Reality Engine server Implements 3D graphics pipeline on hardware Very expensive Operating system: IRIX PC graphics cards at that time were rather simple (compared with today’s standard) No hardware 3D pipeline Almost all graphics operations are handled by CPU

GPU The first popular 3D graphics card was the Voodoo card by 3DFX (1995) Only texture mapping and Z-buffer were implemented on graphics card

GPU Gradually, graphics card companies began to implement more and more 3D pipeline functions on graphics cards By 2000, transformation and lighting were also implemented on graphics card (e.g. GeForce2) Hardware 3D pipeline on graphics card is complete

Programmable GPU In 2001, NVDIA introduced the first programmable GPU in GeForce3 Still implements a fixed-function 3D pipeline Add support for vertex shader and texture shader Can write your own transformation & lighting programs Only supports assembly language shaders

Programmable GPU In 2003, NVIDIA added pixel shader support in GeForceFX Pixel shader replaces texture shader New high level shading language: Cg and HLSL

Programmable GPU In 2004, NVIDIA releases GeForce 6800 and 6600 Vertex shaders can now access texture images PCI-Express bus

Geometry shader Geometry shaders are a new feature of DirectX 10 that allow for dynamic creation of geometry on the GPU. With the introduction of geometry shaders, the GPU can now create and destroy vertices output from the vertex shader. This capability allows for new features such as displacement mapping with tessellation and GPU-generated shadow volumes. Tutorial http://appsrv.cse.cuhk.edu.hk/~ymxie/Geometry_Shader/

United Shader Model Introduced in 2006 Supported in GeForce 8xxx, Radeon HD 2000 series and Intel GMA X3000 series Also known as Shader Model 4.0 Uses a unified Instruction Set Architecture across all shader types

Why unified shader model? NVIDIA designed a single floating point shader core with multiple independent processors. Each of these independent processors is capable of handling any type of shading operation, including pixel shading, vertex shading, geometry shading, and physics shading. GeForce 8800 GPUs can dynamically allocate processing power depending on the workload of the application, providing unprecedented performance and efficiency.

Why unified shader model? http://www.extremetech.com/article2/0,1697,2053309,00.asp

OpenGL 2.0 In 2004, OpenGL 2.0 is officially released Added OpenGL Shading Language to support programmable GPU

Major GPU companies NVIDIA (www.nvidia.com) ATI (www.ati.com) GeForce and Quadro series cards High level shading language: Cg ATI (www.ati.com) Radeon and FireGL series cards 3DLabs (www.3dlabs.com) Wildcat series cards High level shading language: OpenGL Shading Language (GLSL)

Why programmable GPU? Pre-GPU age: fixed function graphics pipeline You can change the pipeline parameters in OpenGL/DirectX but not the algorithms. Programmable GPU: gives freedom to developers Applications can take control of the processing on graphics hardware Never-before-seen effects are possible

Why programmable GPU? Much better graphics performance on GPU than on CPU. GPUs are designed for extreme speed Highly parallel optimized hardware for graphics processing CPUs are designed for flexibility Engineered for wide variety of applications For many graphics applications, real-time performance now becomes possible. Non-graphics applications can also take advantage of the powerful GPU.

Other Uses of GPU Beyond computer graphics application Use the powerful GPU as a second CPU Speed up non-realtime graphics algorithms Real-time ray tracing on GPU Real-time photon mapping on GPU Real-time radiosity on GPU GPU-based volume rendering, etc. Non-graphics applications Fast Fourier transform Interactive collision detection Bioinformatic computing More information at www.gpgpu.org

Major Trends Faster GPUs Performance doubles every 6 - 9 months Increased parallelism with each new generation of graphics processors Increasing programmability of graphics hardware Programmable Processing is the future of graphics and visual and cinematic computing Enhanced vertex and pixel shader capabilities Future shaders may have access to more 3D pipeline operations E.g. clipping, depth test, stencil test, etc.

NVIDIA GeForce 7800 GTX

GeForce 8800 GTX GeForce 8800 GTX architecture

ATI Radeon X1800

What’s in a GPU? The latest GPUs still implements a fixed-function graphics pipeline Controlled by OpenGL/Direct3D functions But there are two programmable units that can take over part of the fixed-function pipeline Vertex processor Pixel (fragment) processor Vertex and pixel processors are controlled by shaders

Vertex Shader A vertex shader is a small program that manipulates vertex data and runs on GPU’s vertex unit Provides developer with almost unlimited control on how vertices are perturbed. Vertex shader can be written in high level shading language: Cg, HLSL, or GLSL

Vertex Shader in Graphics Pipeline CPU GPU

What is in a vertex shader? Vertex Program Reads an untransformed, unlit vertex Creates a transformed vertex Optional operations Calculate lighting for vertex Creates texture coordinates Does not create or delete vertices 1 vertex in and 1 vertex out No topological information provided No edge, face, nor neighboring vertex info

What’s in a vertex shader? You can implement the following operations in a vertex shader: Multiply model-view and projection matrices with vertex coordinates Transform normals to eye coordinates Normalize normals Calculate lighting for each vertex Multiply texture matrices with texture coordinates Access a texture map (shader model 3.0)

What’s not in a vertex shader? Currently the following operations cannot be implemented by a vertex shader Perspective division Clipping Viewport mapping Front face determination Flat shading or Gouraud shading

Why use vertex shader? Write your own transformation and lighting function Visual quality is much better than the fixed function Transformation & Lighting Graphics developers have more control Take advantage of the powerful GPU

Why use vertex shader? Complete control of transform and lighting hardware Complex vertex operations accelerated in hardware Custom vertex lighting Custom skinning and blending Custom texture coordinate generation Custom texture matrix operations Custom vertex computations of your choice Offloading vertex computations frees up CPU More physics and simulation possible!

Vertex Shader Example Custom transform, lighting, and skinning

Vertex Shader Example Custom cartoon-style lighting

Vertex Shader Example Per-vertex set up for per-pixel bump mapping

Example Character morphing & shadow volume projection

Example Dynamic displacements of surfaces by objects

Vertex shader You have to load and compile vertex shaders in your OpenGL (or DirectX) program Can load multiple vertex shaders Vertex shaders can be turned on and off like a state in OpenGL program E.g. turn on shader for some objects If turned on, each vertex shader will be executed once per frame for each of the vertices on the affected objects Your vertex shader may run hundreds of times per frame

What is pixel shader? A pixel (fragment) shader is a small program which processes pixels and executes on the GPU’s pixel (fragment) processor A pixel shader can be written in high level shading language: Cg, HLSL, or GLSL

Pixel Shader in Graphics Pipeline CPU GPU

What’s in a pixel shader? Pixel shader implements the following rasterization operations Texture mapping (including filtering, wrapping, texture environments, etc.) Color sum Fog Discuss standard GL texture ops

Switch between fixed function and pixel shader

What’s not in a pixel shader? Currently the following operations are not supported by pixel shader Blending All of the per-fragment operations

Why use pixel shader? Fully control the texture mapping/blending process instead of using the fixed-function texture mapping. Can generate much better quality images E.g. Per-pixel lighting Can also use pixel shader for non-graphics applications.

What can you do with pixel shader? Per-pixel lighting Looks much better than per-vertex lighting True phong shading Anisotropic lighting Non-Photorealistic-Rendering cartoon shading, hatching, gooch lighting, image space techniques Per-pixel fresnel term Volumetric effects Advanced bump mapping Procedural textures and texture perturbation Bidirectional reflectance distribution functions And more …

Pixel Shader Example: Natural Light

Pixel Shader Example: Reflection

Pixel Shader Example: Bump Mapping

Pixel shader You have to load and compile pixel shaders in your OpenGL (or DirectX) program Can load multiple pixel shaders Pixel shaders can be turned on or off like a state in OpenGL program E.g. turn on shader for some objects If turned on, each pixel shader will be executed once per frame for each of the pixels on the affected objects Your pixel shader may run hundreds of thousands of times per frame

How to develop shader programs? High level shading languages Cg Microsoft High Level Shader Language (HLSL) OpenGL Shading Language Supplementary tools ATI’s RenderMonkey NVIDIA CgFX Composer OpenGL Shader Designer Assembly language (?)

High level shading languages Cg http://developer.nvidia.com/page/cg_main.html Developed by NVIDIA Best supported by NVIDIA GeForce cards Works on Windows, Linux, and Mac OS Looks like C language but with some restrictions Can be loaded in both OpenGL and DirectX programs

High level shading languages MS HLSL Part of DirectX 9.0 (http://www.microsoft.com/windows/directx/) Jointly designed by MS and NVIDIA Almost identical to Cg Supported by GeForce and Radeon cards Only runs on Windows Only works with DirectX programs

High level shading languages OpenGL Shading Language (GLSL): Developed by 3Dlabs, approved by OpenGL ARB http://developer.3dlabs.com/openGL2/index.htm) Now part of the OpenGL 2.0 specification Supported by latest GeForce and Radeon cards Expected to be cross platform Only works with OpenGL programs

Other high level language for GPU Sh http://www.libsh.org/ A metaprogramming language for GPU Developed at University of Waterloo Brook for GPU http://graphics.stanford.edu/projects/brookgpu A stream program language for general purpose computing on GPU Developed at Stanford University

Shader development tools ATI’s RenderMonkey: http://www.ati.com/developer/rendermonkey/ http://www2.ati.com/developer/rendermonkey/movies/RenderMonkey-Siggraph2004SketchVideo.mov A shader development environment for both programmers and artists Can be used for rapid prototyping shaders Supports HLSL and GLSL Runs best on ATI Radeon cards Free

Shader development tools NVIDIA FX Composer: http://developer.nvidia.com/object/fx_composer_home.html http://developer.download.nvidia.com/presentations/2007/gdc/NVIDIA_FX_Composer_2.pdf Similar to RenderMonkey Save files in CgFX or HLSL FX format (A metaprogramming file format) Runs best on GeForce cards Only supports Cg Free

Interface between shader and 3D API Shaders have to be loaded and enabled during runtime by an OpenGL or DirectX program OpenGL/DirectX program need to interact with shaders How to create/load/delete shaders? How to compile shaders? How to enable/disable shaders? How to pass data to shaders?

Interface between shader and 3D API DirectX 9.0 has native functions to interface with both HLSL and Cg shaders OpenGL–Cg interface NVIDIA Cg runtime library NVIDIA OpenGL extensions OpenGL—GLSL interface Newly added OpenGL 2.0 functions (older) OpenGL extensions OpenGL—CgFX interface NVIDIA CgFX runtime library

Shader developers’ options Write shaders in GLSL, use OpenGL 2.0 functions to load/execute shaders Not many cards support this yet Write shaders in HLSL (e.g. using RenderMonkey), use native DirectX functions to load/execute shaders Windows only solution Write shaders in Cg, use NVIDIA OpenGL extensions to load/execute shaders OpenGL extensions are poorly documented

Shader developers’ options Write shaders in Cg, in OpenGL/DirectX program use NVIDIA Cg runtime to load/execute shaders Perhaps a better choice for now Better cross-platform support Easier to learn/use Develop shaders in NVIDIA FX Composer, save them in CgFX files, in OpenGL/DirectX program load/execute shaders using CgFX runtime

Cg and Cg Runtime: a brief introduction Cg language A C-like language Has built-in functions and predefined structures to simplify GPU programming Cg runtime library API that allow OpenGL/DirectX programs to load, compile and link, and execute Cg programs at runtime

How to use Cg shaders? Use 3D API calls to specify vertex and pixel shaders Enable vertex and fragment shaders Load/enable texture as usual Draw geometry as usual Set blend state as usual Vertex shader will execute for each vertex Pixel shader will execute for each pixel

Cg language syntax Date types: float: 32 bit floating point half: 16 bit floating point fixed: 12 bit fixed point bool sampler: handle to texture map struct No pointers … yet

Different kinds of variables Uniform variables Same for each vertex (in a vertex shader) Same for each pixel (in a pixel shader) Examples: model-view matrix, light color Varying variables Different for each vertex (in a vertex shader) Different for each pixel (in a pixel shader) Semantic binding Examples: position, normal, texture coord. Local variables Used for intermediate computations within vertex or pixel shader

Example Semantic binding

Abstracting data flow For varying variables, need to use special keywords, called semantics, to tell Cg compiler witch GPU register to store data in or to read data from Can use in, out, and inout modifiers to describe varying variables in: varying variables that are inputs to a shader out: varying variables that are outputs by a shader inout: varying variables that are inputs but also written by the shader

Example

Array / vector / matrix

Function overloading

Support for vectors and matrices

New operators

A simple Cg vertex shader struct appdata { float4 position : POSITION; float3 normal : NORMAL; float3 color : DIFFUSE; float3 TestColor : SPECULAR; }; struct vfconn float4 HPOS : POSITION; float4 COL0 : COLOR0;

A simple Cg vertex shader vfconn main(appdata IN, uniform float4 Kd, uniform float4x4 ModelViewProj) { vfconn OUT; OUT.HPOS = mul(ModelViewProj, IN.position); OUT.COL0.xyz = Kd.xyz * IN.TestColor.xyz; OUT.COL0.w = 1.0; return OUT; } // main

Two ways to compile Cg Offline compilation Runtime compilation Compile Cg code into assembly code Assembly code is linked to OpenGL program at runtime Runtime compilation Cg program is compiled and linked at runtime. Good future compatibility No dependency limitations Flexible input parameter management

What is Cg runtime? An API that allow application to compile and link Cg programs at runtime Three parts Core Cg runtime OpenGL Cg runtime Direct3D Cg runtime We’ll focus on Core Cg and OpenGL Cg runtime

Where are the files? Cg runtime comes with Cg Toolkit (http://developer.nvidia.com/object/cg_toolkit.html) Normally under C:\Program Files\NVIDIA Corporation\Cg Cg.dll, cgGL.dll, cgD3D9.dll, etc. Cg.lib, cgGL.lib, cgD3D9.lib, etc. Cg.h, cgGL.h, cgD3D9.h, etc. Download and install Cg Toolkit Make sure Cg runtime header files and lib files are on VC++’s search path Make sure Cg runtime DLL files are on system search path

Basic Cg runtime concepts Context A container for multiple Cg programs An application may have multiple contexts Program Compile a Cg program by adding it to a context Parameters A program has several parameters

Cg Language Profiles Different GPUs support different set of graphics capabilities A Cg profile defines a subset of full Cg language that is supported on a particular hardware platform or API CG_PROFILE_VP30, CG_PROFILE_FP30, etc. A full list can found on Cg User’s Manual A Cg program has to be compiled for a particular profile

Basic Steps of using Cg Runtime In your OpenGL program, do the following Create Cg context(s) Initialize profiles Compile Cg program(s) Load Cg program(s) (a) Modify Cg program parameters (b) Enable profile(s)

Basic Steps of using Cg Runtime (c) Bind a Cg program (d) Draw 3D object(s) (e) Disable the profile Repeat (a) – (e) if necessary Destroy Cg program(s) Destroy Cg context(s)

Create a Context You have to create at least one context if you want to use Cg It’s the first to be created and the last to be destroyed CGcontext context = cgCreateContext();

Get Cg profiles CGprofile vertexProfile = cgGLGetLatestProfile(CG_GL_VERTEX) CGprofile fragmentProfile = cgGLGetLatestProfile(CG_GL_FRAGMENT)

Compiling a program CGprogram vertexProgram = cgCreateProgram(context, CG_SOURCE, myVertexProgramString, vertexProfile, “main”, args);

Loading a program Pass the compiled object code to OpenGL cgGLLoadProgram(vertexProgram)

Modify Cg Program Parameters You need to set Cg program parameters before its execution Get Cg program parameters CGparameter baseTexture = cgGetNamedParameter(fragmentProgram, “baseTexture”) Set uniform parameters that don’t change during program execution cgGLSetTextureParameter(baseTexture, texture) cgGLSetParameter4fv(someColor, constantColor)

Modify Cg Program Parameters Set the varying parameters cgGLEnableClientState(texCoord); cgGLSetParameterPointer(texCoord, 2, GL_FLOAT, 0, vertexTexCoords); Set the uniform parameters that may change every frame cgGLSetStateMatrixParameter(modelViewMatrix, CG_GL_MODELVIEW_PROJECT_MATRIX, CG_GL_MATRIX_IDENTITY);

Enable the profiles cgGLEnableProfile(vertexProfile) cgGLEnableProfile(fragmentProfile)

Bind the programs You can bind only one vertex program and one fragment program at a time. cgGLBindProgram(vertexProgram) cgGLBindProgram(fragmentProgram)

Disable the profiles This automatically unbind the corresponding vertex and/or fragment program cgGLDisableProfile(vertexProfile); cgGLDsiableProfile(fragmentProfile);

Destroy Cg program Release the resources allocated to the Cg program cgDestroyProgram(vertexProgram)

Destroy Context Release the resources allocated to the Cg context cgDestroyContext(context);

Cg Runtime Example Example from Cg User’s Manual, page 57 – 60 http://developer.nvidia.com/object/cg_toolkit.html Vertex shader

Cg Runtime Example (cont.) Pixel shader

Cg Runtime Example (cont.) OpenGL program

Cg Runtime Example (cont.)

Cg Runtime Example (cont.)

Cg Runtime Example (cont.)

Examples A complete Cg/Cg runtime example in “Hello, Cg!” http://developer.nvidia.com/object/hello_cg_tutorial.html More examples comes with Cg Toolkit Installed under C:\Program Files\NVIDIA Corporation\Cg\examples\runtime_ogl cgGL_vertex_example.c cgGL_vertex_example.cg

Summary Cg runtime is a bridge between Cg program and the host OpenGL/D3D application Quite straightforward to learn/use Perhaps the more difficult part is to learn how to get/set various Cg program parameters A good understanding of various Cg profiles are also important

Summary Programmable GPU is the biggest revolution in graphics in 10 years and the foundation for the next 10 Use shading languages to write vertex or pixel shaders that run on GPU OpenGL Shading Language, Cg, High Level Shading Language, etc.

Readings “Introduction to the hardware graphics pipeline” http://developer.nvidia.com/object/eg_2004_presentations.html Hello, Cg! http://developer.nvidia.com/object/hello_cg_tutorial.html “Cg in two pages” (http://developer.nvidia.com/attach/7076 ) (Optional) Cg User’s Manual http://developer.nvidia.com/object/cg_toolkit.html

GPU references NVIDIA technical presentations http://developer.nvidia.com/page/documentation.html ATI technical presentations http://www.ati.com/developer/techpapers.html 3Dlabs technical presentations http://developer.3dlabs.com/openGL2/presentations/index.htm

Cg and Cg runtime References Cg toolkit http://developer.nvidia.com/object/cg_toolkit.html Cg User’s Manual 1.5 Comes with Cg Toolkit A thorough discussion of Cg and Cg runtime. Appendix A: Cg language specification www.shadertech.com “The Cg Tutorial”, by R. Fernando and M. J. Kilgard, Addison-Wesley, 2003

GLSL references OpenGL 2.0 specification http://www.opengl.org/documentation/specs/version2.0/glspec20.pdf GLSL Introductions & GLSL master class http://developer.3dlabs.com/openGL2/presentations/index.htm “OpenGL Shading Language”, by R. J. Rost, 2nd edition, Addison-Wesley, 2006 http://www.3dshaders.com/

Next lecture Overview of modeling