Shading Languages Yung-feng Chiu

2 Agenda Introduction Pixar’s RenderMan, MS’s HLSL, NV’s CgFX, ATI’s RenderMonkey Demos.fx file Comparsion

3 Key Idea of a Shading Language Image synthesis can be divided into two basic concerns –Shape: Geometric Objects, Coordinates, Transformations, Hidden-Surface Methods… –Shading: Light, Surface, Material, Texture, … Control shading not only by adjusting parameters and options, but by telling the shader what you want it to do directly in the form of a procedure

4 Pixar’s RenderMan Separation of Modeling and Rendering –RenderMan serves as the interface. Scene = Shape + Shading The power of RenderMan is in the shading part.

5 Pixar’s RenderMan -2 Rman Geom Code cc Rman Program Rman Shader.sl Shader (slc) Byte-code Shader.slc RIB File.rib render Program (rendrib) TIFF image Rman texture Image File txmake

6 RenderMan’s RIB File Structure Options global to the entire animation Frame Block Image options Camera options World Block Attributes, lights, primitives Changed Options Another world block Next frame block

7 RenderMan’s Shader Phong shader surface phong( float Ka = 1, Kd =1, Ks = 0.5; float roughness = 0.1; color specularcolor = 1; ) { normal Nf = faceforward( normalize(N), I ); vector V = -normalize(I); color C = 0; illuminance( P ) { vector R = 2*normalize(N)* (normalize(N). normalize( L )) - normalize( L ); C += Ka*Cs + Kd*Cs*( normalize(N). normalize(L) ) + Ks*specularcolor* pow(( R. V ), 10); } Ci = C*Cs; }

8 RenderMan’s Shader -2 Attaching to the RIB file *********** AttributeBegin Translate Rotate Color [ ] Surface "phong" "Ka" [.1] "Kd" [.8] "Ks" [1] "roughness" [0.1] "specularcolor" [1 1 1] Basis "bezier" 3 "bezier" 3 PatchMesh "bicubic" 13 "nonperiodic" 10 "nonperiodic" "P" [ **************

9 Misrosoft’s HLSL (.fx) Assembly … dp3 r0, r0, r1 max r1.x, c5.x, r0.x pow r0.x, r1.x, c4.x mul r0, c3.x, r0.x mov r1, c2 add r1, c1, r1 mad r0, c0.x, r1, r0... HLSL … float4 cSpec = pow(max(0, dot(Nf, H)), phongExp).xxx; float4 cPlastic = Cd * (cAmbi + cDiff) + Cs * cSpec; … Simple Blinn-Phong shader expressed in both assembly and HLSL

10 nVidia’s CgFX Overview

11 nVidia’s CgFX Overview -2 Supports Microsoft.fx files Cg plus: –Multi-pass –Hardware fallbacks (techniques) –Complete Hardware states –Tweakables MS.fx plus: –DirectX8 and OpenGL

12 nVidia’s CgFX Overview -3 CgFX (.fx) –Manages whole rendering process –Handles render states – cross API support –Convenient exposure of tweakables & artist controls Cg Shaders (.cg) –semantics directives to match your C++ and other custom hardware shaders –Bind textures/parameters to specific HW registers Cg Runtime –Thin API to compile on demand at runtime –Optimizes & manages.Cg for range of target HW

13 nVidia’s CgFX Viewer

14 nVidia’s CgFX Viewer -2 Scene graph GUI.fx parameters edition Error reporting for easy.fx file problem identification Runs OpenGL, DirectX8, DirectX9 –Switch between devices at any point

15 Production Pipeline with CgFX

16 ATI’s RenderMonkey

17 ATI’s RenderMonkey -2 Try to solve some of the problems developers face when designing software on emerging hardware Create a flexible, extensible shader development environment that allows easy incorporation of existing APIs –Support for low level DirectX8/9 in the current version –Extensible framework to support emerging HLSL standards DirectX9 HLSL OpenGL 2.0 Shading Language RenderMan Maya Shade Trees ……

18 ATI’s RenderMonkey -3 Encapsulate all effect data in a single XML text file Each Effect Workspace consists of –Effect Group(s) Effect(s) –Pass(es) Render state Pixel Shader Vertex Shader Geometry Textures –Variables and stream mapping nodes

19 Demos !

20 Bump Map Example Bump mapping simulates detail with a surface normal that varies across a surface

21 RenderMan Example displacement lumpy ( float Km = 1, frequency = 1, maxoctaves = 6; string shadingspace = "shader"; float truedisp = 1;) { point Pshad = transform (shadingspace, frequency*P); float dPshad = filterwidthp(Pshad); float magnitude = fBm (Pshad, dPshad, maxoctaves, 2, 0.5); N = Displace (normalize(N), shadingspace, Km*magnitude, truedisp); }

22

23 Pixel Shader Example ps.1.1 tex t0 // base map tex t1 ;// bump map tex t2 ;// light vector from normalizer cube map dp3_sat r1, t1_bx2, t2_bx2 ; // N.L mul r1, r1, c0 ; // N.L * diffuse_light_color mul_sat r0, t0, r1;// (N.L * diffuse_light_color) * base

24 Cg Example f2fb DiffuseBumpPS(v2f IN, uniform sampler2D DiffuseMap, uniform sampler2D NormalMap, uniform float4 bumpHeight) { f2fb OUT; float4 color = tex2D(DiffuseMap); //fetch base color //fetch bump normal float4 bumpNormal = expand(tex2D(NormalMap)) * bumpHeight; //expand iterated light vector to [-1,1] float4 lightVector = expand(passthrough(IN.LightVector)); //compute final color (diffuse + ambient) float4 bump = uclamp(dot3_rgba(bumpNormal.xyz, lightVector.xyz)); OUT.col = color * bump; return OUT; }

25

26 Fixed Function Example Texture[0] = ; Target[0] = Texture2D; MinFilter[0] = Linear; MagFilter[0] = Linear; MipFilter[0] = Linear; Texture[1] = ; Target[1] = Texture2D; MinFilter[1] = Linear; MagFilter[1] = Linear; MipFilter[1] = Linear; ColorOp[0] = DotProduct3; AlphaOp[0] = SelectArg1; ColorArg1[0] = Texture; ColorArg2[0] = Diffuse; AlphaArg1[0] = Texture; AlphaArg2[0] = Diffuse; ColorOp[1] = Modulate; ColorArg1[1] = Current; ColorArg2[1] = Texture; AlphaOp[1] = SelectArg1;

27

28 GL2 Logical Diagram

29 OpenGL Fixed Function Vertex Transform [MVP],[MV],[MV] -T Lighting [0,1] [0,1] Texgen Texture Matrixn Color SecondaryColor TexCoordn EdgeFlag Normal Vertex (object) TexCoordn EdgeFlag Vertex (eye) Vertex (clip) Front&Back Color Front&Back SecondaryColor

30 GL2 Vertex Processor Temporaries Vertex Shader Uniform Color SecondaryColor TexCoordn EdgeFlag Normal Vertex (object) TexCoordn EdgeFlag Vertex (eye) Vertex (clip) Front&Back Color Front&Back SecondaryColor

31 FrontFacing Color Coord Depth OpenGL Fixed Function Fragment Tex n TE n SumFog [0,1] Coord FrontFacing Color SecondaryColor TexCoord[n] zz e z (|z e |,f ) Depth

32 GL2 Fragment Processor TexCoord[n] FrontFacing zz e z (|z e |,f ) Coord FrontFacing Color Coord Color SecondaryColor Depth Temporaries Fragment Shader Uniform Texture

33 Using FX in Your Application Load effect Validate technique for hardware Detect parameters for technique Render Loop (for each object in scene): –Set technique for the object –Set parameter values for technique –For each pass in technique Set state for pass Draw object

34 Using FX – The FX API LPD3DXBUFFER pError = NULL; D3DXCreateEffectFromFile(m_pd3dDevice, _T("simple.fx"), NULL, NULL, 0, NULL, &m_pEffect, &pError); SAFE_RELEASE(pError);... UINT iPass, cPasses; m_pEffect->SetTechnique(“Simple"); m_pEffect->SetVector(“var1", v); m_pEffect->Begin(&cPasses, 0); for (iPass = 0; iPass < cPasses; iPass++) { m_pEffect->Pass(iPass); m_pMesh->Draw(); } m_pEffect->End();

35 Effect File Structure An effect is made up of multiple rendering algorithms (techniques) each made up of one or more passes Effect File Structure: Variable declarations Technique 1 Pass 1 … Pass n … Technique n Pass 1 … Pass n

36.fx file float4x4 worldMatrix : World;// World or model matrix float4x4 mvpMatrix : WorldViewProjection;// Model * View * Projection float4x4 worldViewMatrix : WorldView;// World * View texture diffuseTexture : DiffuseMap // Diffuse Map semantic < string File = "default_color.dds"; // Default texture file annotation >; texture normalMap : NormalMap // Normal Map semantic < string File = "default_bump_normal.dds";// Default texture file annotation >; float4 ambientColor : Ambient = {0.1, 0.1, 0.1, 1.0}; float bumpHeight < // GUI annotations string gui = "slider"; float min = 0; float max = 1; float step = 0.1; > = 0.5;

37.fx file void DiffuseBumpVS(float4 Position : POSITION,// Position in object space float2 TexCoord : TEXCOORD0,// Texture coordinates … out float4 TexCoord0 : TEXCOORD0, // Texture coordinates out float4 Position : POSITION)// Position in projection space { TexCoord0.xy = IN.TexCoord.xy;// Pass texture coordinates for the diffuse map … Position = mul(WorldViewProj, IN.Position);// Compute position in projection space } void DiffuseBumpPS(float4 Position : POSITION,// Position in projection space … out float4 col : COLOR ) { float4 color = tex2D(DiffuseMap, UV);// Look up the diffuse map … col = color * light;// Modulate the diffuse color by the light }

38.fx file sampler2D diffuseSampler = sampler_state { Texture = ; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; }; sampler2D normalSampler = sampler_state { Texture = ; MinFilter = Linear; MagFilter = Linear; MipFilter = Linear; }; technique CgTechnique// Both the vertex and the fragment shaders are in Cg { pass p0 { ZEnable = true; ZWriteEnable = true; CullMode = None; VertexShader = compile vs_1_1 DiffuseBumpVS(mvpMatrix, WorldIMatrix, lightPos); PixelShader = compile ps_1_1 DiffuseBumpPS(diffuseSampler, normalSampler, bumpHeight); } pass p1 {... }

39 Cg Pipeline Graphics programs are written in Cg and compiled to low-level assembly code... Cg Runtime API... that runs on any GPU compatible with DirectX or OpenGL

40 Using the Cg Compiler // // Diffuse lighting // float d = dot(normalize(frag.N), normalize(frag.L)); if (d < 0) d = 0; c = d*f4tex2D(t, frag.uv)*diffuse; … DP3 r0.x, f[TEX0], f[TEX0]; RSQ r0.x, r0.x; MUL r0, r0.x, f[TEX0]; DP3 r1.x, f[TEX1], f[TEX1]; RSQ r1.x, r1.x; MUL r1, r1.x, f[TEX1]; DP3 r0, r0, r1; MAX r0.x, r0.x, 1.0; MUL r0, r0.x, DIFFUSE; TEX r1, f[TEX1], 0, 2D; MUL r0, r0, r1; … Cg program source code Shader program assembly code Application DevelopmentYour Application 1)Load/bind program 2)Specify program parameters 3)Specify vertex inputs 4)Render Cg Compiler Shader Compiler (nvasm.exe, psa.exe) Shader Binary

41 Game Image Application (game, renderer, …) DCC Image Typical Production Pipeline DCC tool (Maya, Max, SoftImage, …) Scene exporter plug-in Scene manager App Scene Manager hard-coded to choose at run-time the appropriate ASM shaders + state for the hardware Artists create models, textures, maps, … in DCC tool of choice ASM Shaders (HW1) Programmer swrite assembly for different hardware ASM Shaders (HW2) Not the same! Models, Textures, Maps, …

42 Game Image Application (game, renderer, …) DCC Image FX-Enabled Production Pipeline DCC tool (Maya, Max, SoftImage, …) Scene exporter plug-in For any FX, App Scene Manager chooses at run-time the appropriate technique for the hardware Artists assign FX files to scene objects and tweak parameters for each object in real-time Same Image! FX material plug-in FX files Programmers and/or artists write FX effects FX runtime Scene manager Models, Textures, Maps, FX effects + parameters

43 Comparison RenderManOpenGL 2.0D3D VS (2.0/3.0)D3D PS (2.0/3.0) Program SizeNo limit 256/256 No limit>=1616/16 No limit>= 40 interpolators10/10 No limit 16/16 No limit 12/16 Yes Yes/YesNo/Yes Yes No/No Vertex Attributes No limit VS:>=512 floats PS: >= 64 floats 128/25632/128Constants Varying Parameters Texture Samplers Temp Registers Constant-Based Flow Control Variable-Based Flow Control