Mesostructure Rendering Techniques Presented by Keith Yerex

Mesostructure Mesostructure is used to refer to details that are too large to be modeled by reflectance functions, and too small to be modeled efficiently by polygons Commonly rendered with bump-mapping, horizon mapping, and/or displacement mapping.

Bump-mapping [Blinn 78] Store the normals of the true surface in a texture map, and use these normals in lighting equations. (dot3) Normals can be stored in object space, or tangent space. In either case, the light must be transformed into that coordinate system.

Tangent space Repeating bump/horizon/displacement maps must be represented in tangent space Tangent space is a coordinate system made of the normal N, the tangent T, and the binormal B Where T is in the direction of increasing texture coordinate u and B=NxT

Horizon mapping [Max 88] The horizon angle is precomputed for a discrete number of directions An incoming light angle must be above the horizon in order to illuminate the surface

Displacement Mapping A displacement map specifies displacement in the direction of the surface normal, for each point on a surface from Klautz & Seidel

Displacement Mapping Generally expressed as a function of u,v texture coordinates (or parametric surface parameters) Stored as a 2d texture, and/or computed procedurally

The Problem How can we render a model given as a set of polygons, and a displacement map?

Current solutions Geometric Image Space Subdivide and displace Volume slice rendering Ray tracing Image Space Parallax mapping Relief textures View dependent texturing / BDTF View dependent displacement mapping

Subdivide and Displace [Cook 84] Subdivide each polygon into many tiny polygons, and displace the new vertices along their normals using the displacement map Produces many new vertices and triangles, which all need to be transformed and rendered. Improvements include adaptive subdivision, and hardware implementation.

Volume Slicing [Kautz & Seidel] Generate polygons that slice through the displacement volume Render each slice, but draw only pixels where the displacement map value is greater than height of the slice at that position Slices alignment varies (bump-out shells, orthogonal, screen aligned)

Volume Slicing Alpha test can be used for efficient hardware implementation Still generates a lot of geometry and high overdraw/fill-rate from Klautz & Seidel

Ray tracing [Smits et al] In ray tracing, rays are simply intersected with the displacement map, rather than the polygon Reference Plane Max Displacement

Image Space Methods For a given viewing direction and texture coordinates P find the displacement in texture space l from Wang et al

Parallax mapping Makes the approximation that l=V*H(P)*scale+bias [Kaneko et al] Makes the approximation that l=V*H(P)*scale+bias Where H(P) is the height at the point of incidence. Very efficient, and pretty convincing, but not accurate Movie

Parallax mapping

[Oliveira, Bishop, McAllistor] Relief Textures [Oliveira, Bishop, McAllistor] Instead of inverse mapping, use a forward mapping (from texture to image) Warping is implemented efficiently by a 1d pre-warp, followed by standard texture mapping.

View Dependent Texturing [Debevec] Store textures from all possible views, and then just pick the best one from a list when rendering a new view (or blend the best few) BDTF also stores different lighting conditions for all views Very large memory requirements [Dana et al]

View Dependent Displacement Mapping [Wang et al] Store the depth maps along all possible viewing angles. Computing the texture coordinate offset then becomes as simple as

VDM - curvature For curved surfaces, the situation is slightly more complicated So depths are pre-computed for variations in curvature as well as viewing direction

VDM – function The vdm function dVDM(x,y,,,c), represents the depth to the surface along any ray to a surface with any curvature. Note that if texture coordinates are unique, the curvature dimension doesn’t need to be stored.

VDM - shadows Self shadowing is easily computed given the vdm function This is used for curved surfaces also, even though it isn’t correct.

VDM - silhouette A maximum view polar angle function MVM(x,y,,c) is used to store silhouette information Any rays where  < MVM(x,y,,c) are not rendered

VDM - compression The 5D vdm function is sampled in this paper at 128x128 x 32x8 x 16 Plus another 4MB for the mvm function. Taking 68MB of memory!

VDM - compression First, flatten dVDM and  MVM into one 2d matrix A=[AVDM,AMVM] Rows of A are indexed by x,y, and columns are indexed by ,c (or just c for ,AMVM) SVD is applied to A: A=UET=WET Where E= [EVDM,EMVM]

VDM – compression The two functions can now be reconstructed as: Only a the first few eigenfunctions Ei need to be used since the eigenvalues (and Wi) decrease rapidly with i

VDM – compression Using 8 eigenfunctions EVDM and 4 eigenfunctions EMVM reduces the data size to 4MB

VDM – results Movie

References James F. Blinn, Simulation of wrinkled surfaces, Proceedings of the 5th annual conference on Computer graphics and interactive techniques, p.286-292, August 23-25, 1978 Robert L. Cook, Shade trees, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.223-231, January 1984 Kristin J. Dana , Bram van Ginneken , Shree K. Nayar , Jan J. Koenderink, Reflectance and texture of real-world surfaces, ACM Transactions on Graphics (TOG), v.18 n.1, p.1-34, Jan. 1999 Jan Kautz , Hans-Peter Seidel, Hardware accelerated displacement mapping for image based rendering, No description on Graphics interface 2001, p.61-70, June 07-09, 2001, Ottawa, Ontario, Canada Max, N. 1988. Horizon mapping: shadows for bump-mapped surfaces. The Visual Computer 4, 2, 109--117. Manuel M. Oliveira , Gary Bishop , David McAllister, Relief texture mapping, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.359-368, July 2000 Brian E. Smits , Peter Shirley , Michael M. Stark, Direct Ray Tracing of Displacement Mapped Triangles, Proceedings of the Eurographics Workshop on Rendering Techniques 2000, p.307-318, June 26-28, 2000 Lifeng Wang, Xi Wang, Xin Tong, Stephen Lin, Shimin Hu, Baining Guoand Heung-Yeung Shum, View-dependent displacement mapping, ACM Trans. Graph.22 (3), 2002

Questions?