1. Mesostructure Rendering Techniques
Presented by Keith Yerex

2. Mesostructure
Mesostructure is used to refer to details that are too large to be modeled by reflectance functions, and too small to be modeled efficiently by polygons
Commonly rendered with bump-mapping, horizon mapping, and/or displacement mapping.

3. Bump-mapping
[Blinn 78]
Store the normals of the true surface in a texture map, and use these normals in lighting equations. (dot3)
Normals can be stored in object space, or tangent space. In either case, the light must be transformed into that coordinate system.

4. Tangent space
Repeating bump/horizon/displacement maps must be represented in tangent space
Tangent space is a coordinate system made of the normal N, the tangent T, and the binormal B
Where T is in the direction of increasing texture coordinate u and B=NxT

5. Horizon mapping
[Max 88]
The horizon angle is precomputed for a discrete number of directions
An incoming light angle must be above the horizon in order to illuminate the surface

6. Displacement Mapping
A displacement map specifies displacement in the direction of the surface normal, for each point on a surface
from Klautz & Seidel

7. Displacement Mapping
Generally expressed as a function of u,v texture coordinates (or parametric surface parameters)
Stored as a 2d texture, and/or computed procedurally

8. The Problem
How can we render a model given as a set of polygons, and a displacement map?

9. Current solutions Geometric Image Space Subdivide and displace
Volume slice rendering
Ray tracing
Image Space
Parallax mapping
Relief textures
View dependent texturing / BDTF
View dependent displacement mapping

10. Subdivide and Displace
[Cook 84]
Subdivide each polygon into many tiny polygons, and displace the new vertices along their normals using the displacement map
Produces many new vertices and triangles, which all need to be transformed and rendered.
Improvements include adaptive subdivision, and hardware implementation.

11. Volume Slicing
[Kautz & Seidel]
Generate polygons that slice through the displacement volume
Render each slice, but draw only pixels where the displacement map value is greater than height of the slice at that position
Slices alignment varies
(bump-out shells, orthogonal, screen aligned)

12. Volume Slicing
Alpha test can be used for efficient hardware implementation
Still generates a lot of geometry and high overdraw/fill-rate
from Klautz & Seidel

13. Ray tracing
[Smits et al]
In ray tracing, rays are simply intersected with the displacement map, rather than the polygon
Reference Plane
Max Displacement

14. Image Space Methods
For a given viewing direction and texture coordinates P find the displacement in texture space l
from Wang et al

15. Parallax mapping Makes the approximation that l=V*H(P)*scale+bias
[Kaneko et al]
Makes the approximation that l=V*H(P)*scale+bias
Where H(P) is the height at the point of incidence.
Very efficient, and pretty convincing, but not accurate
Movie

16. Parallax mapping

17. [Oliveira, Bishop, McAllistor]
Relief Textures
[Oliveira, Bishop, McAllistor]
Instead of inverse mapping, use a forward mapping (from texture to image)
Warping is implemented efficiently by a 1d pre-warp, followed by standard texture mapping.

18. View Dependent Texturing
[Debevec]
Store textures from all possible views, and then just pick the best one from a list when rendering a new view (or blend the best few)
BDTF also stores different lighting conditions for all views
Very large memory requirements
[Dana et al]

19. View Dependent Displacement Mapping
[Wang et al]
Store the depth maps along all possible viewing angles.
Computing the texture coordinate offset then becomes as simple as

20. VDM - curvature
For curved surfaces, the situation is slightly more complicated
So depths are pre-computed for variations in curvature as well as viewing direction

21. VDM – function
The vdm function dVDM(x,y,,,c), represents the depth to the surface along any ray to a surface with any curvature.
Note that if texture coordinates are unique, the curvature dimension doesn’t need to be stored.

22. VDM - shadows Self shadowing is easily computed given the vdm function
This is used for curved surfaces also, even though it isn’t correct.

23. VDM - silhouette
A maximum view polar angle function MVM(x,y,,c) is used to store silhouette information
Any rays where  < MVM(x,y,,c) are not rendered

24. VDM - compression
The 5D vdm function is sampled in this paper at 128x128 x 32x8 x 16
Plus another 4MB for the mvm function.
Taking 68MB of memory!

25. VDM - compression
First, flatten dVDM and  MVM into one 2d matrix A=[AVDM,AMVM]
Rows of A are indexed by x,y, and columns are indexed by ,c (or just c for ,AMVM)
SVD is applied to A: A=UET=WET
Where E= [EVDM,EMVM]

26. VDM – compression The two functions can now be reconstructed as:
Only a the first few eigenfunctions Ei need to be used since the eigenvalues (and Wi) decrease rapidly with i

27. VDM – compression
Using 8 eigenfunctions EVDM and 4 eigenfunctions EMVM reduces the data size to 4MB

28. VDM – results
Movie

29. References
James F. Blinn, Simulation of wrinkled surfaces, Proceedings of the 5th annual conference on Computer graphics and interactive techniques, p , August 23-25, 1978
Robert L. Cook, Shade trees, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p , January 1984
Kristin J. Dana , Bram van Ginneken , Shree K. Nayar , Jan J. Koenderink, Reflectance and texture of real-world surfaces, ACM Transactions on Graphics (TOG), v.18 n.1, p.1-34, Jan. 1999
Jan Kautz , Hans-Peter Seidel, Hardware accelerated displacement mapping for image based rendering, No description on Graphics interface 2001, p.61-70, June 07-09, 2001, Ottawa, Ontario, Canada
Max, N Horizon mapping: shadows for bump-mapped surfaces. The Visual Computer 4, 2,
Manuel M. Oliveira , Gary Bishop , David McAllister, Relief texture mapping, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p , July 2000
Brian E. Smits , Peter Shirley , Michael M. Stark, Direct Ray Tracing of Displacement Mapped Triangles, Proceedings of the Eurographics Workshop on Rendering Techniques 2000, p , June 26-28, 2000
Lifeng Wang, Xi Wang, Xin Tong, Stephen Lin, Shimin Hu, Baining Guoand Heung-Yeung Shum, View-dependent displacement mapping, ACM Trans. Graph.22 (3), 2002

30. Questions?