GPU Graphics Gary J. Katz University of Pennsylvania CIS 665 Skin Rendering GPU Graphics Gary J. Katz University of Pennsylvania CIS 665 Adapted from David Gosselin’s Power Point and article, Real-time skin rendering, ShaderX article

Overview Background Offline rendering Texture space lighting Blur and dilation Shadows Specular with shadows

Why Skin is Hard Most lighting from skin comes from sub-surface scattering Skin color mainly from epidermis Pink/red color mainly from blood in dermis Lambertian model designed for “hard” surfaces with little sub-surface scattering so it doesn’t work real well for skin Slide from Glossin GDC 2004

Rough skin Cross Section Air Incoming Light Outgoing Light Epidermis Dermis

Offline Rendering Approach from Matrix Reloaded Render a 2D light map Simulate subsurface diffusion in image domain (different for each color component) The illumination of skin is rendered to a texture map Map is blended to simulate the effect of subsurface scattering Used traditional ray tracing for areas where light can pass all the way through (Ears) Render a 2D light map Simulate subsurface diffusion in image domain (different for each color component) The illumination of skin is rendered to a texture map Map is blended to simulate the effect of subsurface scattering Used traditional ray tracing for areas where light can pass all the way through (Ears) Right image is real, left image is fake The frame in the background is a complete rendered frame

Algorithm Create shadow map from the point of view of the key light Structure for texture space algorithm Create shadow map from the point of view of the key light Render diffuse illumination to a 2D light map Dilate the boundaries and blur the light map to approximate subsurface scattering Render final mesh (use the blurred light map for diffuse illumination) Write distance from light into shadow map Light in texture space Blur Geometry Sample texture space light Back Buffer

Texture Space Subsurface Scattering From “The Matrix Reloaded” From Sushi Engine (ATI) Current skin in Real Time

Creating the Shadow Map Compute view-projection matrix to render from point of view of light Use a frustum that tightly bounds the bounding sphere of the geometry Use bounding sphere of head to ensure the most texture space is used Store the depth values in the alpha channel Vertex shader passes depth to the pixel shader. Pixel Shader outputs interpolated depth Blur effect later will provide the soft shadows Only setting the alpha channel here, the RGB is still set to zero, we’ll explain later why (translucent materials, sun glasses)

Creating the Shadow Map Translucent Shadows Use a second pass Render translucent geometry Have z-testing turned on Have z-writing turned off Accumulate opacity of the samples on the RGB channels of the shadow map texture using additive blending This renders to the RGB but not to the alpha channel The z-testing turned on and the z-writing turned off forces all translucent geometry that is hidden by the opaque geometry to be z-culled and thus not rendered

Computing Shadows from Map If the sample is not shadowed by a translucent object its diffuse contribution is attenuated by the value in the alpha channel Opaque geometry shadows itself Translucent geometry shadows opaque geometry

Shadow Map and Shadowed Lit Texture From GDC 2004 Shadows in Texture Space Shadow Map (depth)

Render to Light Map The Algorithm Render diffuse lighting into an off-screen texture using texture coordinates as position Blur the off-screen diffuse lighting Read the texture back and add specular lighting in subsequent pass Only use bump map for the specular lighting pass Diffuse lighting same as in phong shading. Light is scattered equally with no bias depending on viewpoint

Rendering Diffuse Illumination Creating the 2D Light Map Vertex Shader Sets the output positions to be the texture coordinates of the vertices Moves Texture Coordinates from [0,1] to [-1, 1] Computes position of the vertex from point of view of light in the resized [0,1] space and passes result to pixel shader for depth test What is going on: Each vertex contains a texture coordinate that is in the [0,1] range. This is pretty much the light map. It is a 2d representation of the geometry which is totally independent of the camera position. The code is only recentering the texture coordinates to put it in the [-1, 1] range More explanation on the next slide // Vertex Shader Code o.pos.xy = i.texCoord*2.0 – 1.0; o.pos.z = 0.0; o.pos.w = 1.0; Example Light Map

Texture Coordinates as Position Need to light as a 3D model but draw into texture By passing texture coordinates as “position” the rasterizer does the unwrap Compute light vectors based on 3D position and interpolate Slide directly taken from GDC 2004 Real Time Skin Render presentation

Rendering Diffuse Illumination Creating the 2D Light Map Pixel Shader Computes diffuse contribution of light that is attenuated by the shadow // Pixel Shader Code // if light is not in shadow If(lightPos.z < textureMap[x,y] && dot(Normal,Light) > 0) { // compute translucency alpha = textureMap[x,y]^ShadowCoeficient shadowFactor = lerp(occluded, white, alpha); } else shadowFactor = occluded; diffuse = shadowFactor * saturate(dot(Normal, Light) * lightColor) Example Light Map

Blur Used to simulate the subsurface component of skin lighting Used a grow-able Poisson disc filter Read the kernel size from a texture Allows varying the subsurface effect Higher for places like ears/nose Lower for places like cheeks Taken from GDC 2004

Blur Size Map and Blurred Lit Texture Shows how the ears and nose are going to be blurred more than the cheeks Taken from GDC 2004 Created by the artist

Blur All work is done in the pixel shader Poisson distribution is created (hard-coded in shader) Read center sample and blur kernel size from alpha channel Scale other samples by the kernel size Sum the samples and divide by the kernel size Set the alpha channel with the blur size to allow for further blurring passes

Texture Boundary Dilation The Problem: Boundary artifacts are create when fetching from the light map Solution: Dilate the texture prior to blurring Put image from textbook here

Texture Boundary Dilation Modify the Poisson disk filter shader Check whether a given sample is just outside the boundary of useful data If outside, copy from an interior neighboring sample instead More expensive, but only used in first blurring pass This is accomplished by assuming that the alpha value that holds the blur kernel size is never 1.0. Therefore, if it is 1.0 we know there was a boundary artifact. A flag is set if there was a boundary artifact (ie. One of the blur kernel sizes= 1.0), If the flag is set, the blur is set to the sample with the smallest alpha value.

Specular Use bump map for specular lighting Per-pixel exponent Need to shadow specular Hard to blur shadow map directly Modulate specular from shadowing light by luminance of texture space light Darkens specular in shadowed areas but preserves lighting in unshadowed areas The luminance of the blurred light map can be used to attenuate the specular term of the shadow casting light to obtain a natural look. // compute luminance of light map sample float lum = dot(float3(.2125, .7154, .0721), cLightMap.rgb); // possibly scale and bias lum here depending on the light setup // multiply specular by lum specular *= lum;

Specular Shadow Dim Results Specular Without Shadows Specular With Shadows

Acceleration Techniques Frustrum Culling Before rendering the light map clear z-buffer to 1 Perform texture space rendering pass where the z value is set to 0 for all rendered samples On all further texture space passes set the z value to 0 in the vertex shader Set the z test to ‘equal’ If the model lies outside the view frustrum the hardware early-z culling will prevent the pixels from being processed Means that if the bounding box of the model’s head lies outside the view frustrum and is culled by the graphics engine the z value is not modified

Acceleration Techniques Backface Culling Dot product of view vector and normal is computed in vertex shader If frontfacing a 0 is written to the z-buffer Else pixel is clipped, leaving z-buffer untouched Note: bias result of dot product by .3 to have slightly backfaced samples not culled Perform bias because some of the samples that are slightly backfacing may still be in the blurring radius of the frontfacing samples Insert image from textbook here

Acceleration Techniques Distance Culling If model is very far away from camera Z value is set to 1 Light map from previous rendered frame is used Specular illumination is still computed All acceleration techniques can be performed in one pass

Demo