1. Week 8 - Monday
CS361

2. Last time What did we talk about last time? Mipmapping
Summed area tables
Anisotropic filtering

3. Questions?

4. Project 2

5. MonoGame Shader Examples

6. Effect files
We have been using BasicEffect to achieve most of our shading
BasicEffect gets so much done that it's tempting not to move any further
But more complex effects can be achieved by writing shader code ourselves
To start, open up the MonoGame pipepline, right-click on the Content folder, and choose Add > New Item
Choose Effect from the wizard and name it something that ends with .fx

7. Shader declarations
We need to declare the variables we are going to use at the top of the file
These usually include at least the following (which are already given in the template)
We're also going to add an ambient color and an ambient intensity for this shader
float4x4 World;
float4x4 View;
float4x4 Projection;
float4x4 WorldInverseTranspose;
float4 AmbientColor = float4(1, 1, 1, 1);
float AmbientIntensity = 0.1;

8. Shader data structures
We also have to define structures to take the input and the output
These vary because different vertex formats include different data (position, normals, colors, texture coordinates)
The simplest possible input and output would have position only
The POSITION0 is called a semantic
Semantics are used to tell the shader what the purpose of a variable is so that it can pass the right data in and out
struct VertexShaderInput {
float4 Position : POSITION0;
float4 Normal : NORMAL0; };
struct VertexShaderOutput {
float4 Color : COLOR0;

9. Vertex shader
The job of the vertex shader is, at the very least, to transform a vertex from model space to world space to view space to clip space
It can also do normal and color calculations
VertexShaderOutput VertexShaderFunction(VertexShaderInput input){
VertexShaderOutput output;
float4 worldPosition = mul(input.Position, World);
float4 viewPosition = mul(worldPosition, View);
output.Position = mul(viewPosition, Projection);
float4 normal = mul(input.Normal, WorldInverseTranspose);
float lightIntensity = dot(normal,
normalize(DiffuseLightDirection));
output.Color = saturate(DiffuseColor * DiffuseIntensity *
lightIntensity);
return output; }

10. Pixel shader
The pixel shader must find the final color of the pixel fragment
This pixel shader uses a diffuse shading model
The computed lighting is added to the ambient lighting
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0 {
return saturate(input.Color + AmbientColor *
AmbientIntensity); }

11. Techniques
You're allowed to name your vertex and pixel shaders anything you want
You specify which you're going to use in a technique
At this level, techniques only have one pass, but it is possible to use multiple techniques to achieve interesting effects
technique Diffuse {
pass Pass1 {
VertexShader = compile VS_SHADERMODEL
VertexShaderFunction();
PixelShader = compile PS_SHADERMODEL
PixelShaderFunction(); }

12. Using the shader in the game
You have to load the shader like other content
Then, we run a loop similar to the earlier one, setting the parameters for the effect object
effect = Content.Load<Effect>("Diffuse");
foreach( ModelMesh mesh in model.Meshes ) {
foreach (ModelMeshPart part in mesh.MeshParts) {
part.Effect = effect;
effect.Parameters["World"].
SetValue(mesh.ParentBone.Transform * world);
effect.Parameters["View"].SetValue(view);
effect.Parameters["WorldInverseTranspose"].
SetValue(Matrix.Transpose(Matrix.Invert(mesh.ParentBone.Transform * world)));
effect.Parameters["Projection"].
SetValue(projection); }
mesh.Draw();

13. Other Texturing Issues

14. Volume textures
Image textures are the most common, but 3D volume textures can be used
These textures store data in a (u, v, w) coordinate space
Even volume textures can be mipmapped
Quadrilinear interpolation!
In practice, volume textures are usually used for fog, smoke, or explosions
3D effects that are inconsistent over the volume

15. Cube maps A cube map is a kind of texture map with 6 faces
Cube maps are used to texture surfaces based on direction
They are commonly used in environment mapping
A ray is made from the center of the cube out to the surface
The component with the largest magnitude selects which of the 6 faces
The other components are used for (u,v) coordinates
Cube maps can cause awkward seams when jumping between faces

16. Texture caching
You will never need to worry about this in this class, but texture memory space is a huge problem
There are many different caching strategies, similar ones used for RAM:
Least Recently Used (LRU): Swap out the least recently used texture, very commonly used
Most Recently Used (MRU): Swap out the most recently used texture, use only during thrashing
Prefetching can be useful to maintain consistent frame rates

17. Texture compression
JPEG and PNG are common compression techniques for regular images
In graphics hardware, these are too complicated to be decoded on the fly
That's why the finished MonoGame projects have pre-processed.tkb files
Most DirectX texture compression divides textures into 4 x 4 tiles
Two 16-bit RGB values are recorded for each tile
Each texel uses 2 bits to select one of the two colors or two interpolated values between them

18. More texture compression
Ericsson texture compression (ETC) is used in OpenGL
It breaks texels into 2 x 4 blocks with a single color
It uses per-pixel luminance information to add detail to the blocks
Normal maps (normals stored as textures) allow for interesting compression approaches
Only x and y components are needed since the z component can be calculated
The x and y can then be stored using the BC5 format for two channels of color data

19. Procedural texturing
A procedural texture is made by computing a function of u and v instead of looking up a texel in an image
Noise functions are often used to give an appearance of randomness
Volume textures can be generated on the fly
Values can be returned based on distance to certain feature points (redder colors near heat, for example)

20. Texture animation Textures don't have to be static
The application can alter them over time
Alternatively, u and v values can be remapped to make the texture appear to move
Matrix transformations can be used for zoom, rotation, shearing, etc.
Video textures can be used to play back a movie in a texture
Blending between textures can allow an object to transform like a chameleon

21. Material Mapping
The lighting we have discussed is based on material properties
Diffuse color
Specular color
Smoothness coefficient m
A texture can be used to modify these values on a per-pixel basis
A normal image texture can be considered a diffuse color map
One that affects specular colors is a specular color map (usually grayscale)
One that affects m is a gloss map

22. Alpha Mapping Alpha values allow for interesting effects
Decaling is when you apply a texture that is mostly transparent to a (usually already textured) surface
Cutouts can be used to give the impression of a much more complex underlying polygon
1-bit alpha doesn't require sorting
Cutouts are not always convincing from every angle

23. Student Lecture: Bump Mapping

24. Bump Mapping

25. Bump mapping
Bump mapping refers to a wide range of techniques designed to increase small scale detail
Most bump mapping is implemented per-pixel in the pixel shader
3D effects of bump mapping are greater than textures alone, but less than full geometry

26. Macro, meso, and micro
Macro-geometry is made up of vertices and triangles
Limbs and head of a body
Micro-geometry are characteristics shaded in the pixel shader, often with texture maps
Smoothness (specular color and m parameter) based on microscopic smoothness of a material
Meso-geometry is the stuff in between that is too complex for macro-geometry but large enough to change over several pixels
Wrinkles
Folds
Seams
Bump mapping techniques are primarily concerned with mesoscale effects

27. Blinn's methods
James Blinn proposed the offset vector bump map or offset map
Stores bu and bv values at each texel, giving the amount that the normal should be changed at that point
Another method is a heightfield, a grayscale image that gives the varying heights of a surface
Normal changes can be computed from the heightfield

28. Normal maps
The results are the same, but these kinds of deformations are usually stored in normal maps
Normal maps give the full 3-component normal change
Normal maps can be in world space (uncommon)
Only usable if the object never moves
Or object space
Requires the object only to undergo rigid body transforms
Or tangent space
Relative to the surface, can assume positive z
Lighting and the surface have to be in the same space to do shading
Filtering normal maps is tricky

29. Parallax mapping
Bump mapping doesn't change what can be seen, just the normal
High enough bumps should block each other
Parallax mapping approximates the part of the image you should see by moving from the height back to the view vector and taking the value at that point
The final point used is:

30. Parallax mapping continued
At shallow viewing angles, the previous approximation can look bad
A small change results in a big texture change
To improve the situation, the offset is limited (by not scaling by the z component)
It flattens the bumpiness at shallow angles, but it doesn't look crazy
New equation:

31. Relief mapping
The weakness of parallax mapping is that it can't tell where it first intersects the heightfield
Samples are made along the view vector into the heightfield
Three different research groups proposed the idea at the same time, all with slightly different techniques for doing the sampling
There is still active research here
Polygon boundaries are still flat in most models

32. Heightfield texturing
Yet another possibility is to change vertex position based on texture values
Called displacement mapping
With the geometry shader, new vertices can be created on the fly
Occlusion, self-shadowing, and realistic outlines are possible and fast
Unfortunately, collision detection becomes more difficult

33. Upcoming

34. Next time…
Radiometry
Photometry
Colorimetry
BRDFs

35. Reminders
Start reading Chapter 7
Finish Project 2
Due on Friday