1. GPU Graphics Gary J. Katz University of Pennsylvania CIS 665
Skin Rendering
GPU Graphics
Gary J. Katz
University of Pennsylvania CIS 665
Adapted from David Gosselin’s
Power Point and article,
Real-time skin rendering, ShaderX article

2. Overview Background Offline rendering Texture space lighting
Blur and dilation
Shadows
Specular with shadows

3. Why Skin is Hard
Most lighting from skin comes from sub-surface scattering
Skin color mainly from epidermis
Pink/red color mainly from blood in dermis
Lambertian model designed for “hard” surfaces with little sub-surface scattering so it doesn’t work real well for skin
Slide from Glossin GDC 2004

4. Rough skin Cross Section
Air
Incoming Light
Outgoing Light
Epidermis
Dermis

5. Offline Rendering Approach from Matrix Reloaded
Render a 2D light map
Simulate subsurface diffusion in image domain (different for each color component)
The illumination of skin is rendered to a texture map
Map is blended to simulate the effect of subsurface scattering
Used traditional ray tracing for areas where light can pass all the way through (Ears)
Render a 2D light map
Simulate subsurface diffusion in image domain (different for each color component)
The illumination of skin is rendered to a texture map
Map is blended to simulate the effect of subsurface scattering
Used traditional ray tracing for areas where light can pass all the way through (Ears)
Right image is real, left image is fake
The frame in the background is a complete rendered frame

6. Algorithm Create shadow map from the point of view of the key light
Structure for texture space algorithm
Create shadow map from the point of view of the key light
Render diffuse illumination to a 2D light map
Dilate the boundaries and blur the light map to approximate subsurface scattering
Render final mesh (use the blurred light map for diffuse illumination)
Write distance from light
into shadow map
Light in
texture space
Blur
Geometry
Sample texture space light
Back
Buffer

7. Texture Space Subsurface Scattering
From “The Matrix Reloaded”
From Sushi Engine (ATI)
Current skin in Real Time

8. Creating the Shadow Map
Compute view-projection matrix to render from point of view of light
Use a frustum that tightly bounds the bounding sphere of the geometry
Use bounding sphere of head to ensure the most texture space is used
Store the depth values in the alpha channel
Vertex shader passes depth to the pixel shader.
Pixel Shader outputs interpolated depth
Blur effect later will provide the soft shadows
Only setting the alpha channel here, the RGB is still set to zero, we’ll explain later why (translucent materials, sun glasses)

9. Creating the Shadow Map
Translucent Shadows
Use a second pass
Render translucent geometry
Have z-testing turned on
Have z-writing turned off
Accumulate opacity of the samples on the RGB channels of the shadow map texture using additive blending
This renders to the RGB but not to the alpha channel
The z-testing turned on and the z-writing turned off forces all translucent geometry that is hidden by the opaque geometry to be z-culled and thus not rendered

10. Computing Shadows from Map
If the sample is not shadowed by a translucent object its diffuse contribution is attenuated by the value in the alpha channel
Opaque geometry shadows itself
Translucent geometry shadows opaque geometry

11. Shadow Map and Shadowed Lit Texture
From GDC 2004
Shadows in Texture Space
Shadow Map
(depth)

13. Render to Light Map The Algorithm
Render diffuse lighting into an off-screen texture using texture coordinates as position
Blur the off-screen diffuse lighting
Read the texture back and add specular lighting in subsequent pass
Only use bump map for the specular lighting pass
Diffuse lighting same as in phong shading. Light is scattered equally with no bias depending on viewpoint

14. Rendering Diffuse Illumination Creating the 2D Light Map
Vertex Shader
Sets the output positions to be the texture coordinates of
the vertices
Moves Texture Coordinates from [0,1] to [-1, 1]
Computes position of the vertex from point of view of light in the resized [0,1] space and passes result to pixel shader for depth test
What is going on:
Each vertex contains a texture coordinate that is in the [0,1] range. This is pretty much the light map. It is a 2d representation of the geometry which is totally independent of the camera position.
The code is only recentering the texture coordinates to put it in the [-1, 1] range
More explanation on the next slide
// Vertex Shader Code
o.pos.xy = i.texCoord*2.0 – 1.0;
o.pos.z = 0.0;
o.pos.w = 1.0;
Example Light Map

15. Texture Coordinates as Position
Need to light as a 3D model but draw into texture
By passing texture coordinates as “position” the rasterizer does the unwrap
Compute light vectors based on 3D position and interpolate
Slide directly taken from GDC 2004 Real Time Skin Render presentation

16. Rendering Diffuse Illumination Creating the 2D Light Map
Pixel Shader
Computes diffuse contribution of light that is
attenuated by the shadow
// Pixel Shader Code
// if light is not in shadow
If(lightPos.z < textureMap[x,y] &&
dot(Normal,Light) > 0) {
// compute translucency
alpha = textureMap[x,y]^ShadowCoeficient
shadowFactor = lerp(occluded, white, alpha); }
else
shadowFactor = occluded;
diffuse = shadowFactor *
saturate(dot(Normal, Light) * lightColor)
Example Light Map

17. Blur Used to simulate the subsurface component of skin lighting
Used a grow-able Poisson disc filter
Read the kernel size from a texture
Allows varying the subsurface effect
Higher for places like ears/nose
Lower for places like cheeks
Taken from GDC 2004

18. Blur Size Map and Blurred Lit Texture
Shows how the ears and nose are going to be blurred more than the cheeks
Taken from GDC 2004
Created by the artist

19. Blur All work is done in the pixel shader
Poisson distribution is created
(hard-coded in shader)
Read center sample and blur kernel size from alpha channel
Scale other samples by the kernel size
Sum the samples and divide by the kernel size
Set the alpha channel with the blur size to allow for further blurring passes

20. Texture Boundary Dilation
The Problem:
Boundary artifacts are create when fetching from the light map
Solution:
Dilate the texture prior to blurring
Put image from textbook here

21. Texture Boundary Dilation
Modify the Poisson disk filter shader
Check whether a given sample is just outside the boundary of useful data
If outside, copy from an interior neighboring sample instead
More expensive, but only used in first blurring pass
This is accomplished by assuming that the alpha value that holds the blur kernel size is never Therefore, if it is 1.0 we know there was a boundary artifact. A flag is set if there was a boundary artifact (ie. One of the blur kernel sizes= 1.0), If the flag is set, the blur is set to the sample with the smallest alpha value.

22. Specular Use bump map for specular lighting Per-pixel exponent
Need to shadow specular
Hard to blur shadow map directly
Modulate specular from shadowing light by luminance of texture space light
Darkens specular in shadowed areas but preserves lighting in unshadowed areas
The luminance of the blurred light map can be used to attenuate the specular term of the shadow casting light to obtain a natural look.
// compute luminance of light map sample
float lum = dot(float3(.2125, .7154, .0721), cLightMap.rgb);
// possibly scale and bias lum here depending on the light setup
// multiply specular by lum
specular *= lum;

23. Specular Shadow Dim Results
Specular Without Shadows
Specular With Shadows

24. Acceleration Techniques
Frustrum Culling
Before rendering the light map clear z-buffer to 1
Perform texture space rendering pass where the z value is set to 0 for all rendered samples
On all further texture space passes
set the z value to 0 in the vertex shader
Set the z test to ‘equal’
If the model lies outside the view frustrum the hardware early-z culling will prevent the pixels from being processed
Means that if the bounding box of the model’s head lies outside the view frustrum and is culled by the graphics engine the z value is not modified

25. Acceleration Techniques
Backface Culling
Dot product of view vector and normal is computed in vertex shader
If frontfacing a 0 is written to the z-buffer
Else pixel is clipped, leaving z-buffer untouched
Note: bias result of dot product by .3 to have slightly backfaced samples not culled
Perform bias because some of the samples that are slightly backfacing may still be in the blurring radius of the frontfacing samples
Insert image from textbook here

26. Acceleration Techniques
Distance Culling
If model is very far away from camera
Z value is set to 1
Light map from previous rendered frame is used
Specular illumination is still computed
All acceleration techniques can be performed in one pass

27. Demo