Texture Mapping CSE167: Computer Graphics Instructor: Steve Rotenberg UCSD, Fall 2006 Modified by R. Cook 2014

Texture Mapping A way to render realistic picture w/o using too many polygons Parallax mapping Fall 2009 revised

Texture Mapping Texture mapping is the process of mapping an image onto a triangle in order to increase the detail of the rendering This allows us to get fine scale details without resorting to rendering tons of tiny triangles The image that gets mapped onto the triangle is called a texture map or texture and is usually a regular color image

Texture Map We define our texture map as existing in texture space, which is a normal 2D space The lower left corner of the image is the coordinate (0,0) and the upper right of the image is the coordinate (1,1) The actual texture map might be 512 x 256 pixels for example, with a 24 bit color stored per pixel

Texture Coordinates To render a textured triangle, we must start by assigning a texture coordinate to each vertex A texture coordinate is a 2D point [tx ty] in texture space that is the coordinate of the image that will get mapped to a particular vertex

Texture Mapping v1 t1 v0 t2 t0 v2 (1,1) y (0,0) x Texture Space Triangle (in any space)

Vertex Class We can extend the concept of a Model to include texture coordinates We can do this by simply extending the Vertex class: class Vertex { Vector3 Position; Vector3 Color; Vector3 Normal; Vector2 TexCoord; void Draw() { glColor3f(Color.x, Color.y, Color.z); glNormal3f(Normal.x, Normal.y, Normal.z); glTexCoord2f(TexCoord.x, TexCoord.y); glVertex3f(Position.x, Position.y, Position.z); // This has to be last } };

Texture Coordinate System Texture: a 2D array of color values, each unit is called texel (texture element) Every texture map has (s,t) coordinates [0,0] to [1,1] Each vertex in a polygon specifies its texture coordinates (s,t), then map to the given image to determine the corresponding texture Fall 2009 revised

Example Screen space view Texture Map Texture space view OpenGL code Fall 2009 revised

Example BACK Screen space view Texture space view Texture Map OpenGL code BACK Fall 2009 revised

Texture Interpolation The actual texture mapping computations take place at the scan conversion and pixel rendering stages of the graphics pipeline During scan conversion, as we are looping through the pixels of a triangle, we must interpolate the txty texture coordinates in a similar way to how we interpolate the rgb color and z depth values As with all other interpolated values, we must precompute the slopes of each coordinate as they vary across the image pixels in x and y Once we have the interpolated texture coordinate, we look up that pixel in the texture map and use it to color the pixel

Tiling The image exists from (0,0) to (1,1) in texture space, but that doesn’t mean that texture coordinates have to be limited to that range We can define various tiling or wrapping rules to determine what happens when we go outside of the 0…1 range

Tiling (1,1) x y (0,0) Texture Space

Clamping (1,1) x y (0,0) Texture Space

Mirroring (1,1) x y (0,0) Texture Space

Combinations One can usually set the tiling modes independently in x and y Some systems support independent tiling modes in x+, x-, y+, and y-

Magnification What happens when we get too close to the textured surface, so that we can see the individual texels up close? This is called magnification We can define various magnification behaviors such as: Point sampling Bilinear sampling Bicubic sampling

Magnification With point sampling, each rendered pixel just samples the texture at a single texel, nearest to the texture coordinate. This causes the individual texels to appear as solid colored rectangles With bilinear sampling, each rendered pixel performs a bilinear blend between the nearest 4 texel centers. This causes the texture to appear smoother when viewed up close, however, when viewed too close, the bilinear nature of the blending can become noticeable Bicubic sampling is an enhancement to bilinear sampling that actually samples a small 4x4 grid of texels and performs a smoother bicubic blend. This may improve image quality for up close situations, but adds some memory access costs

Minification Minification is the opposite of magnification and refers to how we handle the texture when we view it from far away Ideally, we would see textures from such a distance as to cause each texel to map to roughly one pixel If this were the case, we would never have to worry about magnification and minification However, this is not a realistic case, as we usually have cameras moving throughout an environment constantly getting nearer and farther from different objects When we view a texture from too close, a single texel maps to many pixels. There are several popular magnification rules to address this When we are far away, many texels may map to a single pixel. Minification addresses this issue Also, when we view flat surfaces from a more edge on point of view, we get situations where texels map to pixels in a very stretched way. This case is often handled in a similar way to minification, but there are other alternatives

Mipmapping Minification Mipmapping was first published in 1983, although the technique had been in use for a couple of years at the time In addition to storing the texture image itself, several mipmaps are precomputed and stored Each mipmap is a scaled down version of the original image, computed with a medium to high quality scaling algorithm Usually, each mipmap is half the resolution of the previous image in both x and y For example, if we have a 512x512 texture, we would store up to 8 mipmaps from 256x256, 128x128, 64x64, down to 1x1 Altogether, this adds 1/3 extra memory per texture (1/4 + 1/16 + 1/64…= 1/3)

Mipmapping When rendering a mipmapped texture, we still compute an interpolated tx,ty per pixel, as before Instead of just using this to do a point sampled (or bilinear…) lookup of the texture, we first determine the appropriate mipmap to use Once we have the right mipmap, we can either point sample or bilinear sample it

Basic Texture Mapping To render a texture mapped triangle, we must first assign texture coordinates to the vertices These are usually assigned at the time the object is created, just as the positions, normals, and colors The tx and ty texture coordinates are interpolated across the triangle in the scan conversion process, much like the z value, or the r, g, and b. However, unlike the other values, it is highly recommended to do a perspective correct interpolation for texture coordinates Once we have a texture coordinate for the pixel, we perform some sort of texture lookup. This will make use of the minification or magnification rules as necessary, and commonly involves looking up and combining 8 individual texels for trilinear mipmapping (min), or 4 texels for bilinear sampling (mag) This texture color is then usually blended with the interpolated rgb color to generate the final pixel color

Solid Textures It is also possible to use volumetric or solid textures Instead of a 2D image representing a colored surface, a solid texture represents the coloring of a 3D space Solid textures are often procedural, because they would require a lot of memory to explicitly store a 3D bitmap A common example of solid texture is marble or granite. These can be described by various mathematical functions that take a position in space and return a color Texturing a model with a marble solid texture can achieve a similar effect to having carved the model out of a solid piece of marble

Texture Combining Textures don’t just have to represent the diffuse coloring of a material- they can be used to map any property onto a triangle One can also combine multiple textures onto the same triangle through a variety of combination rules For example, if we shine a slide projector onto a brick wall, we could map the surface with both the brick texture and the projected image The triangle would have two sets of texture coordinates, and both would be interpolated in the scan conversion process At the final pixel rendering stage, the textures are combined according to certain rules that can be set up to achieve a variety of effects

Bump Mapping As we said, a texture doesn’t always have to represent a color image The technique of bump mapping uses a texture to represent the variable height of a surface For example, each texel could store a single 8 bit value which represents some variation from a mean height This perturbed normal is then used for per-pixel lighting This allows one to use a bump map to fake small surface irregularities that properly interact with lighting

sketches 50864 27242 2306 7205

Specify the Texture glTexImage1D() glTexImage2D() glTexImage3D()                                                           Specify the Texture glTexImage1D() glTexImage2D() glTexImage3D() Fall 2009 revised

Two Dimensional Texture glTexImage2D (GLenum target, GLint level, GLint components, GLsizei width, GLsizei height, GLint border, GLenum format, GLenum type, const GLvoid *pixels) target：GL_TEXTURE_2D level：LOD number (0: base image) components：number of color components (1|2|3|4) Format：pixel data format Border: 0 or 1 Width & height 2m + 2(border bit) w and h can be different There are new extensions that removes this restriction Fall 2009 revised

glTexImage*D supplement In GL version 1.1 or greater, pixels may be a null pointer. In this case texture memory is allocated to accommodate a texture of width width and height height. You can then download subtextures to initialize this texture memory. The image is undefined if the user tries to apply an uninitialized portion of the texture image to a primitive. Fall 2009 revised

Setting Texture Environment glTexEnv{if}{v}(GLenum target, GLenum pname, TYPEparam); Setting how textures are to be interpreted: Target：GL_TEXTURE_ENV Pname: GL_TEXTURE_ENV_MODE Param: modes (DECAL|REPLACE|MODULATE|BLEND) Fall 2009 revised

Texture Environment Modes GL_REPLACE GL_MODULATE (default) GL_DECAL GL_BLEND New environment modes: GL_ADD: Cv = Cf + Ct GL_COMBINE (ARB, see here) Fall 2009 revised

GL_MODULATE Color of polygon affects the display of texture Tree: (r,g,b,a) acutout = 0 Polygon: (1,0,0) Fall 2009 revised

GL_BLEND Use with: glTexEnvfv (GL_TEXTURE_ENV, GL_TEXTURE_ENV_COLOR, colorPtr) Cc: texture environment color Fall 2009 revised

GL_REPLACE Appearance solely determined by texture Tree: (r,g,b,a) acutout = 0 Polygon: (1,0,0) FOR TEXTURE CUT-OUTS Fall 2009 revised

GL_DECAL Cp: replace Tree: (r,g,b,a) acutout = 0 Polygon: (1,0,0) RGB: DECAL=REPLACE Fall 2009 revised

Texture + Lighting To show fragment color, use GL_MODULATE Apply specular color AFTER texture mapping: glLightModeli (GL_LIGHT_MODEL_COLOR_CONTROL, GL_SEPARATE_SPECULAR_COLOR); See FAQ 21.040 GL_SEPARATE_SPECULAR_COLOR GL_SINGLE_COLOR Fall 2009 revised

Texture Coordinates Texture coordinate: Associate texture location (in texture space) with vertices in the polygon glTexCoord2i (s, t); glVertex2i (x, y); Order cannot be reversed! [Think of TexCoord as state assignment] Fall 2009 revised

Texture Coordinates of Quadrics Most quadric primitives have default setting for texture coordinates To turn on the default setting: gluQuadricTexture (qobj, GL_TRUE) Fall 2009 revised

Polygon (in screen space) and texture coordinates Texture Access and Lookup Polygon (in screen space) and texture coordinates (1,1) (1,0) (0,.75) linear nearest Filters (1,1) (1,0) (0,.75) Texture map (4x4) rasterization Interpolate (s,t) and lookup (r,g,b,a) in the texture map Fall 2009 revised

Magnification & Minification 1 Pixels Texels Magnification Minification Nature of problem: Mismatch between texels and pixels Fall 2009 revised

Options Magnification Minification Chooses the mipmap that most closely matches the size of the pixel being textured Magnification GL_NEAREST GL_LINEAR Minification GL_NEAREST, GL_LINEAR, GL_NEAREST_MIPMAP_NEAREST GL_LINEAR_MIPMAP_NEAREST GL_NEAREST_MIPMAP_LINEAR GL_LINEAR_MIPMAP_LINEAR Nearest: in Manhattan distance to the center of the pixel Chooses the two mipmaps that most closely match the size of the pixel being textured Fall 2009 revised

Magnification Options GL_LINEAR Interpolate the texels More time consuming but more accurate GL_NEAREST Snap to the nearest texel Fall 2009 revised

Problems of Minification Can use nearest neighbor or bilinear interpolation,but these two may cause severe aliasing problems Nearest Neighbor Bilinear Interpolation or ?? Texels Pixels No Influence Fall 2009 revised

Problem with Minification without mipmap with mipmap Fall 2009 revised

Mipmapping The most popular method of anti-aliasing for textures Mipmapping was invented by Lance Williams in 1983 and is described in his paper Pyramidal parametrics. The most popular method of anti-aliasing for textures ‘Mip’ stands for “multum in parvo” = “many things in a small place” The texture is downsampled to a quarter of the original area. 2m2n m, n∈+ 2m-12n-1 d = the Level Of Detail (LOD) Level 0 Level 1 Level 2 Level 3 Level 0 Texture If a pixel covers 2222 texels, go to level 2 and get a texel. 2121, level 1 Fall 2009 revised

[Point Sampled Texture Aliasing] Texture map Polygon far from the viewer in perspective projection Rasterized and textured Note that the back row is a very poor representation of the true image Fall 2009 revised

Minification Options glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR, GL_NEAREST, GL_LINEAR_MIPMAP_LINEAR, GL_LINEAR_MIPMAP_NEAREST, GL_NEAREST_MIPMAP_LINEAR, GL_NEAREST_MIPMAP_NEAREST); Remarks: Mipmap selection is done per pixel Using not-yet-ready mipmap disables the texture mapping function. Fall 2009 revised

Mipmap Generation gluBuild2DMipmaps Storage overhead Fall 2009 revised

Texture Wrap When the texture coordinates are outside [0,1] glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S,param);  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T,param);  param: GL_CLAMP|GL_REPEAT Fall 2009 revised

Parameters for Wrapping GL_CLAMP use the border texel Diagonal: corner texel GL_REPEAT repeat the texels in [0,1] Fall 2009 revised

Texture Object A texture object stores texture data and makes it readily available You can now control many textures and go back to textures that have been previously loaded into your texture resources Using texture objects is usually the fastest way to apply textures, resulting in big performance gains it is almost always much faster to bind (reuse) an existing texture object than it is to reload a texture image using glTexImage*D() Fall 2009 revised

Texture Object API glGenTextures glBindTexture glDeleteTextures Allocate n textures (integer identifiers) glBindTexture Bind the current texture to specified identifier glDeleteTextures Free the allocated texture identifiers Reverse of glGenTextures Fall 2009 revised

Texture Objects Texture properties saved in texture objects: Minification and magnification filters, wrapping modes, border color and texture priority When a texture object is bound again, one may edit the contents of the bound texture object. Any commands that change the above properties will change the currently bound texture as well as the current texture state. Texture environment, texgen, etc. are NOT stored in texture objects Fall 2009 revised

Texture Transforms Texture coordinates are multiplied by a 4 by 4 matrix before any texture mapping occurs. Texture animation: using this, you can make the texture slide over the surface, rotate around it, stretch and shrink, … All matrix operations apply: Push/Pop/Mult/… Fall 2009 revised

Texture Suite explode cloud smoke fire water Fall 2009 revised

Automatic TEXture Coordinate GENeration void glTexGen{ifd}{v}(GLenum coord, GLenum pname, TYPEparam); Coord: GL_S, GL_T, GL_R, GL_Q Pname: GL_TEXTURE_GEN_MODE Type: GL_OBJECT_LINEAR, GL_EYE_LINEAR, or GL_SPHERE_MAP Used when the tex coords are too cumbersome to specify, or will be changed with time. Texture environment, texgen, etc. are NOT stored in texture objects. Fall 2009 revised

Usages of Multitexture Fall 2009 revised

ARB_multitexture (ref) (First) ARB-approved extension (1998) Support up to 32 texture units* Texture matrix and texgen affects the active texture environment can cascade up to 32 TE’s Note: default blending mode is GL_MODULATE * glGetIntegerv (GL_MAX_TEXTURE_UNITS, &units); My platform has only 4! OpenGL 3.1 requires minimum of 16 texture units Fall 2009 revised

Details OpenGL's multitexture support requires that every texture unit be fully functional and maintain state that is independent of any other texture units. Each texture unit has its own texture coordinate generation state, texture matrix state, texture enable state, and texture environment state. However, each texture unit within an OpenGL context shares the same set of texture objects. Fall 2009 revised

Multi Texture Fall 2009 revised

TexGen with Multitexture When the GL_ARB_multitexture extension is supported, glTexGen set the texture generation parameters for the currently active texture unit, selected with glActiveTexture.  TexUnit0: texture2D;(s,t) given TexUnit1: texture1D; texGen Fall 2009 revised

Fall 2009 revised

Fall 2009 revised

Fall 2009 revised