1. Real-Time Shading Using Programmable Graphics Hardware Introduction, Setup and Examples Wan-Chun Ma National Taiwan University

2. Course Infomation Instructor  Wan-Chun Ma, Alex  Dept. of Computer Science and Information Engineering, National Taiwan University  http://graphics.csie.ntu.edu.tw/~firebird http://graphics.csie.ntu.edu.tw/~firebird Course  4/28, 5/5, 5/12, 5/14  Suggested readings R. Fernando and M. J. Kilgard. The Cg Tutorial: The Definitive Guide to Programmable Real-Time Graphics, Addison-Wesley, 2003 (beginners only!) R. J. Rost. OpenGL Shading Language, Addison-Wesley, 2004  http://graphics.csie.ntu.edu.tw/~firebird/dokuwiki/doku.p hp?id=tech:courses:dci_rts:home http://graphics.csie.ntu.edu.tw/~firebird/dokuwiki/doku.p hp?id=tech:courses:dci_rts:home

3. The Student... The student should be familiar with  C/C++ programming  Graphics basics Transformations in 3D (translation, rotation, modelview, projection) Rasterization Texturing  OpenGL GLUT, GLUI Use texturing in OpenGL

4. Today’s Schedule Introduction Setup of Programming Environment Real-Time Shading Examples

5. Introduction

6. Evolution of GPUs Virtual Fighter SEGA Dead or Alive 3 Temco Dawn Demo NVIDIA NV1Xbox (NV2A)GeForce FX (NV30) 50K triangles/sec 1M pixel ops/sec 1M transistors 100M triangles/sec 1G pixel ops/sec 20M transistors 200M triangles/sec 2G pixel ops/sec 120M transistors 199520012003

7. The 5 Generations of GPU 1 st generation (up to 1998)  NVIDIA TNT2, ATI Rage, 3dfx Voodoo3  Lack of transform vertices of 3D objects. Vertex transformation are done by CPU  Limited math operations for combining textures to compute the color of pixels 2 nd generation (1999-2000)  NVIDIA GeForce 256, GeForce 2, ATI Radeon 7500  GPU has the ability to do transformation and lighting. Both OpenGL and DirectX 7 support vertex transformation by hardware  Configurable (in driver level) but not programmable

8. The 5 Generations of GPU 3 rd generation (2001)  NVIDIA GeForce 3, GeForce 4 Ti, Xbox, ATI Radeon 8500  Vertex programmability: DirectX 8 vertex shader and OpenGL ARB vertex program  Pixel-level configurable 4 th generation (2002)  NVIDIA GeForce FX, ATI Radeon 9700  Vertex and pixel programmability  High-level shading language (NVIDIA Cg, Microsoft HLSL, OpenGL GLSL) 5 th generation (2004)  NVIDIA GeForce 6, ATI Radeon X  Infinite length shader program  Dynamic flow control

9. GPU Model (Old) Fixed function pipeline

10. GPU Model (Current) Programmability!

11. GPU Process Vertex Processing Fragment Processing

12. Programming GPU However, programming in assembly is painful DP3 R0, c[11].xyzx, c[11].xyzx; RSQ R0, R0.x; MUL R0, R0.x, c[11].xyzx; MOV R1, c[3]; MUL R1, R1.x, c[0].xyzx; DP3 R2, R1.xyzx, R1.xyzx; RSQ R2, R2.x; MUL R1, R2.x, R1.xyzx; ADD R2, R0.xyzx, R1.xyzx; DP3 R3, R2.xyzx, R2.xyzx; RSQ R3, R3.x; MUL R2, R3.x, R2.xyzx; DP3 R2, R1.xyzx, R2.xyzx; MAX R2, c[3].z, R2.x; MOV R2.z, c[3].y; MOV R2.w, c[3].y; LIT R2, R2; DX8 shader instructions Basic: mov, add, mul, mad, rsq… Vector operation: dp3, dp4… Miscellaneous: lit, exp, log, min, max, tex… Oh my god!

13. Programming GPU The need of high level shading language Compile DP3 R0, c[11].xyzx, c[11].xyzx; RSQ R0, R0.x; MUL R0, R0.x, c[11].xyzx; MOV R1, c[3]; MUL R1, R1.x, c[0].xyzx; DP3 R2, R1.xyzx, R1.xyzx; RSQ R2, R2.x; MUL R1, R2.x, R1.xyzx; ADD R2, R0.xyzx, R1.xyzx; DP3 R3, R2.xyzx, R2.xyzx; RSQ R3, R3.x; MUL R2, R3.x, R2.xyzx; DP3 R2, R1.xyzx, R2.xyzx; MAX R2, c[3].z, R2.x; MOV R2.z, c[3].y; MOV R2.w, c[3].y; LIT R2, R2; // A Phong model shader COLOR c = k_a + k_d * dot(N, L) + k_s * pow(max(0, dot(N, H)), k_exp); High level shading language Easier to read and modify Cross-platform Code reuse

14. Cg: A Shading Language Cg is a high level language from NVIDIA for programming GPUs, developed in close collaboration with Microsoft Cg stands for “C for Graphics” Cg enables a dramatic productivity increase for graphics development developers of:  Games  CAD tools  Scientific visualizations

15. Cg: A C-like Language Syntax, operators, functions from C Conditionals and flow control (for, if) Particularly suitable for GPUs:  Express data flow of pipeline/stream architecture of GPUs (e.g. vertex-to-pixel)  Vector and matrix operations  Support hardware data types for maximum performance  Exposes GPU functions for convenience and speed: Intrinsic: (mul, dot, sqrt, exp, pow) Built-in: extremely useful and GPU optimized math, utility and geometric functions (noise, ddx, ddy, reflect) Compiler uses hardware profiles to subset Cg as required for particular hardware feature sets

16. Cg Workflow Architecture

17. Cg Workflow Shader Development Application Cg program source code // Diffuse lighting float d = dot(normalize(N), normalize(L)); if (d < 0) d = 0; c = d*tex2D(texture, T)*diffuse; 1.Load/bind program 2.Specify program parameter 3.Specify vertex inputs 4.Render Cg Compiler Shader program assembly code DP3 r0.x, f[TEX0], f[TEX0]; RSQ r0.x, r0.x; MUL r0, r0.x, f[TEX0]; DP3 r1.x, f[TEX1], f[TEX1]; RSQ r1.x, r1.x; MUL r1, r1.x, f[TEX1]; DP3 r0, r0, r1; MAX r0.x, r0.x, 1.0; MUL r0, r0.x, DIFFUSE; TEX r1, f[TEX1], 0, 2D; MUL r0, r0, r1; Shader Compiler Shader binary 0000h: 54 68 69 73 20 69 73 20 65 2D 54 65 58 2C 20 56 0010h: 65 72 73 69 6F 6E 20 33 2E 31 34 31 35 39 32 2D 0020h: 32 2E 31 20 28 4D 69 4B 54 65 58 20 32 2E 34 29 0030h: 20 28 70 72 65 6C 6F 61 64 65 64 20 66 6F 72 6D 0040h: 61 74 3D 6C 61 74 65 78 20 32 30 30 34 2E 36 2E

18. What Cg can do? Real-time visual effects

19. What Cg can do? Lots of effects…

20. Coffee Break Next section: Setup of Programming Environment

21. Setup of Programming Environment

22. Requirement Hardware  The computer should be equipped with programmable graphics hardware  NVIDIA FX, NVIDIA 6, ATI 9x00, ATI X series Software  Microsoft Visual Studio.NET 2003  GLUT, GLUI...

23. Installation Cg Toolkit 1.3 (10MB)  http://developer.nvidia.com/object/cg_toolkit.ht ml http://developer.nvidia.com/object/cg_toolkit.ht ml  Check the “Cg Installer for Windows” NVIDIA SDK 9.0 (340MB, not required)  http://developer.nvidia.com/object/sdk_home.ht ml http://developer.nvidia.com/object/sdk_home.ht ml FX Composer 1.6 (60MB, not required)  http://developer.nvidia.com/object/fx_composer _home.html http://developer.nvidia.com/object/fx_composer _home.html  Check the “FX Composer 1.6 Installer”

24. Installation If default installation locations are used, all the packages are installed in the folder of C:\Program Files\NVIDIA Corporation\ C:\Program Files\NVIDIA Corporation\  Cg\  NVIDIA FX Composer\  SDK 9.0\

25. My Stuff Several useful codes I collect  http://graphics.csie.ntu.edu.tw/~firebird/ download/dci_rts/class.zip http://graphics.csie.ntu.edu.tw/~firebird/ download/dci_rts/class.zip Download it and unpack it into a folder, say  D:\My Projects\Class\  Any folder is ok, but remember where you put it

26. VC++ Directories Execute visual studio  Tools, Options, Projects, VC++ Directories  Show the directories for:  Include files D:\My Project\Class (remember My Stuff?) C:\Program Files\NVIDIA Corporation\Cg\include  Library files C:\Program Files\NVIDIA Corporation\Cg\lib

27. Ready to Go A small engine  http://graphics.csie.ntu.edu.tw/~firebird/ download/dci_rts/env.zip http://graphics.csie.ntu.edu.tw/~firebird/ download/dci_rts/env.zip  I will use this engine for shader development during these courses The first example  http://graphics.csie.ntu.edu.tw/~firebird/ download/dci_rts/ex01.zip http://graphics.csie.ntu.edu.tw/~firebird/ download/dci_rts/ex01.zip

28. Compilation cgc –profile profiles filename  profiles: graphics hardware profiles Vertex: arbvp1, vp20, vp30, vp40... Fragment: arbfp1, fp20, fp30, fp40...  filename: filename of the shader Examples  cgc –profile vp30 test_vtx.cxx  cgc –profile fp30 test_frg.cxx

29. Debugging Debugging is very hard (it is GPU, not CPU) However, you may still use intermediate visualization to debug your program  Output intermediate data (e.g. position, normal, textures…) as color

30. Coffee Break Next section: Real-time Shading Examples

31. Real-Time Shading Examples

32. Progression Games push hardware, hardware advances games

33. Effects in Games Shadows Level of detail Reflection Shading Smoke

34. Effects in Games Bump mapping Light mapping Per-pixel lighting Multi-texturing

35. Multi-pass Rendering The rendering pass is not fixed anymore. A single rendering pass may consists of many functional programs

36. Multi-pass Rendering Each different program (effect) is handled individually, and finally summed up to become rendering result

37. Cg Samples Check out the effect samples in NVIDIA SDK Browser

38. The First Cg Example A Phong model shader with color texture Shaders  Vertex: ex1_vtx.cxx  Fragment: ex1_frg.cxx Texture  Diffuse: wood.bmp

39. Vertex Shader struct v2f { float4 P2D: POSITION;// projected 2D position float4 C: COLOR0;// color float4 T: TEXCOORD0;// texture coord float3 P3D: TEXCOORD1;// vertex 3D position float3 N: TEXCOORD2;// normal float3 G: TEXCOORD3;// tangent float3 B: TEXCOORD4;// binormal }; Vertex-to-fragment data structure

40. Vertex Shader Main (application-to-vertex) arguments v2f main( float4 C: COLOR, float4 P: POSITION, float4 N: NORMAL, float4 T: TEXCOORD0, uniform float4x4 ModelViewProj, uniform float4x4 ModelView, uniform float4x4 ModelViewIT)

41. Vertex Shader Main body { v2f OUT; OUT.P2D = mul(ModelViewProj, P); OUT.P3D = P.xyz; OUT.T = T; OUT.N= normalize(N.xyz); // normal OUT.G= normalize(2.0*C.xyz - 1.0);// tangent OUT.B= normalize(cross(OUT.G, OUT.N)); return OUT; }

42. Fragment Shader Fragment-to-screen data structure struct f2s { float4 C: COLOR0; };

43. Fragment Shader Main (vertex-to-fragment) arguments f2s main( v2f IN, uniform sampler2Dtex01, // texture 01 uniform float3L, uniform float3V)

44. Fragment Shader Main body { f2s OUT; OUT.rgb = 0; L = normalize(L); V = normalize(V); float3 H = normalize(L+V); float diff = dot(normalize(IN.N), L); if(diff > 0) { float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = diff*tex2D(tex01, IN.T.xy) + spec; } return OUT; }

45. Result

46. Try This... Output red color for all fragments { f2s OUT; OUT.rgb = 0; // L = normalize(L); V = normalize(V); // float3 H = normalize(L+V); // float diff = dot(normalize(IN.N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = float3(1.0, 0.0, 0.0); // } return OUT; }

47. Try This... Visualize normal vectors { f2s OUT; OUT.rgb = 0; // L = normalize(L); V = normalize(V); // float3 H = normalize(L+V); // float diff = dot(normalize(IN.N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = (IN.N+1)/2; // } return OUT; }

48. End