Ying Zhu Georgia State University Computer Graphics Algorithms Advanced Texture Mapping Techniques: displacement mapping, multi-texture, bump mapping, and projective texture mapping Ying Zhu Georgia State University

Heightmap and displacement mapping Heightmap is an image used to store surface elevation data for display in 3D computer grahics. In displacement mapping, a heightmap can be used to displace the position of vertices. In bump mapping, a heightmap can be used for generating the normal map.

Heightmap

Heightmap Heightmaps are widely used in games, terrain rendering software, GIS, scientific visualization, etc. Another name for heightmaps is “digital elevation models” Heightmaps are good for storing digital terrain elevations because they require less memory for a given level of detail than regular polygon mesh models.

Heightmap How to generate a heightmap? Terragen Picogen Many video games have editors for generating heightmaps See http://en.wikipedia.org/wiki/Heightmap for a list

Displacement mapping in OpenGL/GLSL Create a heightmap Load the heightmap in an OpenGL program, and then pass it to the vertex shader as a texture image. In the vertex shader, retrieve an RGB or RGBA vector from the heightmap for the vertex. This is called vertex texture.

Displacement mapping Convert the RGB or RGBA vector to a grayscale value. This is the amount of displacement for this vertex. Multiply the grayscale value to the normal vector of this vertex, and then add the result to the current vertex position. The normal vector guides the direction of the displacement. Multiply the new vertex position with the model-view-projection matrix.

Displacement mapping After vertex displacement, the original normal is no longer valid. You may need to recalculate the normal vectors, or create a normal map from the heightmap, or simply don’t do lighting at all. http://en.wikipedia.org/wiki/Displacement_mapping

Displacement mapping

Multi-texture mapping Multi-texture is quite easy with GLSL Load multiple texture images in the OpenGL program and then pass them to the shader. In the pixel shader, you can multiple, add, or mix the texture colors from multiple texture images. http://www.opengl.org/wiki/Multitexture_with_GLSL

Multi-texture mapping

Bump mapping Create the illusion of a bumpy, uneven surface without using a lot of small polygons Bump mapping is actually a per-pixel lighting technique

Bump mapping Basic idea Create a normal map that stores the normals for each pixel The “bumpiness” is encoded in the normal map. Load the normal map in the OpenGL program and then pass the normal map to the shader program as a texture image. In the fragment shader, retrieve the normal for the pixel from the normal map and then calculate lighting color with this normal vector. The key idea: Instead of interpolating the normals from the vertices, the normals are retrieved from the normal map.

Normal maps

Normal maps How to create a normal map? Convert a picture to a normal map using image editors (e.g. Photoshop, GIMP) Create a high resolution version and a low resolution version of the same 3D model, and then generate a normal map from the high resolution version for the low resolution version. Advanced modeling tools: Maya, Blender, etc. Nvidia’s Melody (http://developer.nvidia.com/object/melody_home.htm)

Normal maps How to generate normal map in GIMP? Install GIMP (http://gimp-win.sourceforge.net/) Download the GIMP Normalmap plugin (http://code.google.com/p/gimp-normalmap/) and follow the instructions in the readme file to install it. Open an image in GIMP. From the menu, choose Filters  Map  Normalmap Change the “Scale” value if you want to increase the “bumpiness”. Create the normal map and save it to an image file.

Simple bump mapping Bump mapping on a flat surface is quite easy Simply use the normal retrieved from the normal map for per-pixel lighting calculation. There is no need to transform the normal. This is because the coordinate system of the normal map and the coordinate system of the pixel is the same.

Simple bump mapping

General bump mapping Bump mapping on a curved surface is more complicated Retrieve the normal from the normal map Transform the pixel position to the tangent space (i.e. the normal map’s coordinate space) Calculate lighting in the tangent space. This is called tangent space bump mapping

General bump mapping Tutorials on the tangent space bump mapping http://wiki.gamedev.net/index.php/OpenGL:Tutorials:GLSL_Bump_Mapping http://www.blacksmith-studios.dk/projects/downloads/bumpmapping_using_cg.php http://http.developer.nvidia.com/CgTutorial/cg_tutorial_chapter08.html

Projective texture mapping Project a texture from a (sometimes invisible) projector. Projective texture mapping is the basis for shadow mapping, lightmap, texture decal, etc.

Projective texture mapping

Projective texture mapping In regular texture mapping, the texture coordinates from each vertex is static, often calculated in the modeling software such as Maya, 3DS Max, or Blender. In projective texture mapping, the texture coordinates are calculated dynamically from the position of the vertex. Ignore the texture coordinates provided by the 3D model file.

Projective texture mapping The basic idea Suppose you look from the projector’s viewpoint, you will always see the texture image -- front and center. If you can project a 3D vertex to this 2D image, then the 2D location of the projected vertex is its projective texture coordinate. So the key is to construct a matrix to project any vertex to a 2D image from the projector’s viewpoint.

Projective texture mapping The steps: Construct a model-view-projection-bias matrix from the projector's viewpoint, using GLM’s lookat() and perspective() functions. A scale and bias matrix is used to scale the eventual texture coordinates to [0, 1] range. Pass this projectorMVPB matrix to the vertex shader.

Projective texture mapping The steps (continued): In the vertex shader, multiply the vertex position with this projectorMVPB matrix, and then use the first two component's of the transformed position as the texture coordinates for this vertex. Pass the texture coordinates to the fragment shader. Carry out texture mapping in the fragment shader as usual.

Projective texture mapping Since the texture may be clamped at the border, it’s better to use a texture with a black border.

Projective texture mapping An (old) tutorial on projective texture mapping from Nvidia http://developer.nvidia.com/object/Projective_Texture_Mapping.html This tutorial describes the matrices for transforming the vertex position to projective texture coordinates.