David Luebke 1 9/13/2015 Texture Mapping

David Luebke 2 9/13/2015 Recap: Texture Map Rendering ● Rendering uses the mapping: ■ Find the visible surface at a pixel ■ Find the point on that surface corresponding to that pixel ■ Find the point in the texture corresponding to that point on the surface ■ Use the parameters associated with that point on the texture to shade the pixel ● Using triangulated meshes reduces the problem to mapping a portion of the image to each triangle:

Recap: Texture Map Rendering

David Luebke 4 9/13/2015 Recap: User-Generated Mappings ● For complex 3-D objects, mapping textures is still something of an art…so we often let the user do it

David Luebke 5 9/13/2015 Recap: Naive Texture Mapping ● A first cut at a texture-mapping rasterizer: ■ For each pixel: ○ Interpolate u & v using standard edge equations techniques ○ Look up nearest texel in texture map ○ Color pixel according to texel color (possibly modulated by lighting calculations) ● A serious artifact is warping at the edges of triangles making up the mesh ■ An obvious example:

David Luebke 6 9/13/2015 Recap: Interpolating Parameters ● The problem turns out to be fundamental to interpolating parameters in screen-space ■ Uniform steps in screen space  uniform steps in world coords

David Luebke 7 9/13/2015 Recap: Perspective-Correct Interpolation ● The solution: ■ Rather than interpolating u and v directly, interpolate u/z and v/z ○ These do interpolate correctly in screen space ○ Also need to interpolate z and multiply per-pixel ■ Problem: we don’t know z anymore ■ Solution: we do know w  1/z ■ So…interpolate uw and vw and w, and compute u = uw/w and v = vw/w for each pixel ○ This unfortunately involves a divide per pixel

David Luebke 8 9/13/2015 Recap: Perspective-Correct Texturing ● Known as perspective-correct texture mapping ■ Early PC cards and game consoles didn’t support it ■ Compensated by using more, smaller polygons ● As mentioned, other interpolation schemes really ought to use perspective correction ■ E.g., Gouraud shading ■ Generally get away without it because it is more important to be smooth than correct ● Java code fragment from McMillan’s edge- equation triangle rasterizer:

David Luebke 9 9/13/2015 Perspective-Correct Texturing: Code Fragment... PlaneEqn(uPlane, (u0*w0), (u1*w1), (u2*w2)); PlaneEqn(vPlane, (v0*w0), (v1*w1), (v2*w2)); PlaneEqn(wPlane, w0, w1, w2);... for (y = yMin; y <= yMax; y += raster.width) { e0 = t0; e1 = t1; e2 = t2; u = tu; v = tv; w = tw; z = tz; boolean beenInside = false; for (x = xMin; x <= xMax; x++) { if ((e0 >= 0) && (e1 >= 0) && (e2 >= 0))) { int iz = (int) z; if (iz <= raster.zbuff[y+x]) { float denom = 1.0f / w; int uval = (int) (u * denom + 0.5f); uval = tile(uval, texture.width); int vval = (int) (v * denom + 0.5f); vval = tile(vval, texture.height); int pix = texture.getPixel(uval, vval); if ((pix & 0xff000000) != 0) { raster.pixel[y+x] = pix; raster.zbuff[y+x] = iz; } beenInside = true; } else if (beenInside) break; e0 += A0; e1 += A1; e2 += A2; z += Az; u += Au; v += Av; w += Aw; } t0 += B0; t1 += B1; t2 += B2; tz += Bz; tu += Bu; tv += Bv; tw += Bw; }

David Luebke 10 9/13/2015 Texture Tiling ● It is often handy to tile a repeating texture pattern onto a surface ● The previous code does this via tile() : int uval = (int) (u * denom + 0.5f); uval = tile(uval, texture.width); int vval = (int) (v * denom + 0.5f); vval = tile(vval, texture.height); int pix = texture.getPixel(uval, vval); int tile(int val, int size) { while (val >= size) val -= size; while (val < 0) val += size; } See

David Luebke 11 9/13/2015 Texture Transparency ● McMillan’s code also includes a “quick fix” for handling transparent texture: if ((pix & 0xff000000) != 0) { raster.pixel[y+x] = pix; raster.zbuff[y+x] = iz; } ■ Note that this doesn’t handle partial transparency ■ Demo at:

David Luebke 12 9/13/2015 Recap: Texture Coordinates ● Give each vertex of the triangle a texture coordinate (u, v) ● For other points on the triangle, interpolate texture coordinate from the vertices ● Problem: interpolating in screen-space (a la Gouraud shading) is incorrect ■ Perspective foreshortening should compress the texture image on distant regions of the surface ■ Demo at

David Luebke 13 9/13/2015 Recap: Perspective-Correct Interpolation ● Skipping some math… ■ Rather than interpolating u and v directly, interpolate u/z and v/z ○ These do interpolate correctly in screen space ○ Also need to interpolate z and multiply per-pixel ■ Problem: we don’t know z anymore ■ Solution: we do know w  1/z ■ So…interpolate uw and vw and w, and compute u = uw/w and v = vw/w for each pixel ○ This unfortunately involves a divide per pixel

David Luebke 14 9/13/2015 Texture Map Aliasing ● Naive texture mapping looks blocky, pixelated ■ Problem: using a single texel to color each pixel: int uval = (int) (u * denom + 0.5f); int vval = (int) (v * denom + 0.5f); int pix = texture.getPixel(uval, vval); ■ Actually, each pixel maps to a region in texture ○ If the pixel is larger than a texel, we should average the contribution from multiple texels somehow ○ If the pixel is smaller than a texel, we should interpolate between texel values somehow ○ Even if pixel size  texel size, a pixel will in general fall between four texels ■ An example of a general problem called aliasing

David Luebke 15 9/13/2015 Recap: Texture Map Antialiasing ● Use bilinear interpolation to average nearby texel values into a single pixel value (Draw it) ■ Find 4 nearest texture samples ○ Round u & v up and down ■ Interpolate texel values in u ■ Interpolate resulting values in v ● Also addresses the problem of many pixels projecting to a single texel (Why?)

David Luebke 16 9/13/2015 Recap: Texture Map Antialiasing ● What if a single pixel covers many texels? ■ Problem: sampling those texels at a single point (the center of the pixel): ○ Produces Moire patterns in coherent texture (checkers) ○ Leads to flicker or texture crawling as the texture moves ○ Show OpenGL Texturing tutor ■ Approach: blur the image under the pixel, averaging the contributions of the covered texels ○ But calculating which texels every pixel covers is way too expensive, especially as the texture is compressed ■ Solution: pre-calculate lower-resolution versions of the texture that incorporate this averaging

David Luebke 17 9/13/2015 MIP-maps ● For a texture of 2 n x 2 n pixels, compute n-1 textures, each at ½ the resolution of previous: ● This multiresolution texture is called a MIP-map

Generating MIP-maps ● Generating a MIP-map from a texture is easy ■ For each texel in level i, average the values of the four corresponding texels in level i-1 ● If a texture requires n bytes of storage, how much storage will a MIP-map require? ● Answer: 4n/3

GR B RG B RG B B GR Representing MIP-maps Trivia: MIP = Multum In Parvo (many things in a small place)

David Luebke 20 9/13/2015 Using MIP-maps ● Each level of the MIP-map represents a pre-blurred version of multiple texels ■ A texel at level n represents 2 n original texels ● When rendering: ■ Figure out the texture coverage of the pixel (i.e., the size of the pixel in texels of the original map) ■ Find the level of the MIP map in which texels average approximately that many original texels ■ Interpolate the value of the four nearest texels

David Luebke 21 9/13/2015 Using MIP-maps ● Even better: ■ Likely, the coverage of the pixel will fall somewhere between the coverage of texels in two adjacent levels of the MIP map ■ Find the pixel’s value in each of the two textures using two bilinear interpolations ■ Using a third interpolation, find a value in between these two values, based on the coverage of the pixel versus each of the MIP-map levels ■ This is (misleadingly?) called trilinear interpolation

David Luebke 22 9/13/2015 Using MIP-maps ● How many interpolations does a texture lookup using trilinear interpolation in a MIP-mapped texture involve? ● How many texel values from the MIP-map must be fetched for such a lookup?

David Luebke 23 9/13/2015 MIP-map Example ● No filtering: ● MIP-map texturing:

David Luebke 24 9/13/2015 Can We Do Better? ● What assumption does MIP-mapping implicitly make? ● A: The pixel covers a square region of the texture ■ More exactly, the compression or oversampling rate is the same in u and v ● Is this a valid assumption? Why or why not?

David Luebke 25 9/13/2015 MIP-maps and Signal Processing ● An aside: aliasing and antialiasing are properly topics in sampling theory ■ Nyquist theorem, convolution and reconstruction, filters and filter widths ■ Textures are particularly difficult because a tiled texture can easily generate infinite frequencies ○ E.g., a checkered plane receding to an infinite horizon ■ Using a MIP-map amounts to prefiltering the texture image to reduce artifacts caused by sampling at too low a rate

David Luebke 26 9/13/2015 Summed-Area Tables ● A technique called summed-area tables lets us integrate texels covered by the pixel more exactly (but still quickly) ■ Details in the book ● Example: MIP-map texturingSummed-area table texturing

David Luebke 27 9/13/2015 Texture Mapping Variations ● A texture can modulate any parameter in the rendering process: Texture as R,G,B: Texture as diffuse lighting coefficients:

David Luebke 28 9/13/2015 Bump Mapping ● The texture map can even modulate the surface normal used for shading Sphere w/ diffuse textureSwirly bump map Sphere w/ diffuse texture and swirly bump map

David Luebke 29 9/13/ = More Bump Mapping ● How can you tell a bumped-mapped object from an object in which the geometry is explicitly modeled?

David Luebke 30 9/13/2015 Last Bump Mapping Example

David Luebke 31 9/13/2015 Displacement Map ● A displacement map actually displaces the geometry ■ Treats the texture as a height field to be applied to the surface ■ Starting to appear in the interactive graphics pipeline ○ First supported in Matrox Parhelia card ○ Can sort of implement with beta drivers in ATI & NVIDIA cards ○ Will soon appear in all cards ■ Implemented by recursive subdivision of triangles/quads

David Luebke 32 9/13/2015 Displacement Map Example ● What is the biggest visual difference between displacement mapping and bump mapping?

David Luebke 33 9/13/2015 Illumination Maps ● Quake introduced illumination maps or light maps to capture lighting effects in video games Texture map: Texture map + light map: Light map

David Luebke 34 9/13/2015 Illumination Maps ● Illumination maps differ from texture maps in that they: ■ Usually apply to only a single surface ■ Are usually fairly low resolution ■ Usually capture just intensity (1 value) rather than color (3 values) ● Illumination maps can be: ■ Painted by hand: Disney’s Aladdin ride ■ Calculated by a global illumination process: Nintendo64 demo, modern level builders

David Luebke 35 9/13/2015 Other Texture Applications ● Lots of other interesting applications of the texture- map concept (we’ll return to some): ■ Shadow maps ■ 3-D textures (marble, wood, clouds) ■ Procedural textures ■ Environment maps & cube maps ● For a neat explanation of the first three (with cool applets, as usual) check out: