Texture Sprites: Texture Elements Splatted on Surfaces Sylvain Lefebvre Samuel Hornus Fabrice Neyret GRAVIR/IMAG-INRIA ACM SIGGRAPH 2005 Symposium on Interactive 3D Graphics and Games 2005

Abstract

Introduction Texturing 3D models with very high resolutions create two major difficulties –Storage cost of high resolution texture maps can easily exceed the available texture memory –Creating high resolution textures proves to be a difficult and tedious task Lack of representation for textures composed of sparse elements –Wastes a lot of memory –Discontinuities –Flickering due to Z-fighting

Introduction Texture sprites –The texture element live on the object surface and can be updated dynamically Provides a general and convenient scheme for sprite-based texturing –Homogeneous appearances obtained by overlapping many sprites –Interactive addition of local details Geometry and composite texture are independent –Not require: 1) mesh to conform to any constraint 2) global planar parameterization 3) modify the initial mesh

Contributions A parameterization-free texture representation allowing efficient storage and filtered rendering of composite textures An efficient solution to dynamic decal management and animated texture elements New appealing texturing effects Complete GPU-implementation

Sprite-Based Textures Key idea –Store the sprite attributes (position, orientation, etc) in a 3D hierarchical structure surrounding the mesh –Follow the idea of octree textures The sprites attributes in the leaves of the hierarchical grid –The entire data-structure is compact Many sprites share a same texture pattern Information is stored only around the surface

Sprite-Based Textures Easy to author –add, drag, drop, change size, and orientation The system is very flexible –Sprite ordering is controllable –Sprite overlapping can be mixed Each sprite can be independently animated –For instance: react to surface deformations

Difficulties How to manage the hierarchical 3D grid on the GPU (fragment program) How to use the hierarchical 3D grid to store and retrieve sprites attributes How to filter the composite texture and blend overlapping sprites together

3D Hierarchical Grid (in GPU) Problem –Store the hierarchical grid in texture memory –Retrieve data from the grid at a given 3D location –(This is implemented in a fragment program) N 3 -treeWe call this structure N 3 -tree –N in each dimension –Octree → N = 2 –Implementation: N = 4

N 3 -tree Memory Layout Tree structure –Node: indirection grid consisting of N 3 cells –Cell content: (cell type is determined by a flag stored in the A-channel) A = 1, data (leaf) A = ½, a pointer A = 0, empty –Store our structure in a 3D texture called the indirection pool Data values and pointer values are stored as RGB-triples

Notations N*N*N each tree nodeN*N*N: resolution of the indirection grid corresponding to each tree node N l *N l *N lN l *N l *N l : resolution of virtual 3D grid at level l At level l a point M in space ( M [0, 1) 3 ) lies in a cell located at and has a local coordinate within this cell. We have

Notations Node are packed in the indirection pool S = (S u, S v, S w ): size of indirection pool NS u *NS v *NS w : resolution of the texture hosting the indirection pool A texture value in the indirection pool is indexed by a real valued P [0, 1) 3 P = P i +P f

Notations P i identifies the node which is stored in the area P i is called the node coordinates P f is the local coordinate of P within the node’s indirection grid (!! Packing the cells does not change the local coordinates.) Equation(1)

Tree Lookup For rendering, we need to retrieve the data corresponding to a given location M in space (more precisely, on the object surface) –This is done per-pixel in a fragment program –Start from level 0 in the tree –At level l let P l,i be the coordinate of the current node in the indirection pool –The RGBA value corresponding to M is found in P l,i at the location P l,f, obtained by Equation(1) –A –A : in a leaf (return), an empty node (discard) or an internal node (continue to the next level (l+1) )

Implementation Details To avoid precision issues, and minimize storage requirements we store S ‧ P l,i instead of P l,i in the indirection cells S ‧ P l,i is an integer –Optimized the encoding: choose the number of bits to stored in the RGB channels to exactly represent at most S indices –The values of the pointers do not depend on the size of the indirection pool

Precision Limitations The limited precision of 32 bits floating point: –Limit on the depth, resolution, number of nodes of the hierarchical grid

Managing the Texture Sprites Each sprite is associated with –various parameters –a bounding volume V defining the spatial limit of the sprite influence on the composite texture Bounding volume V is used to –determine which cells are covered by sprite –detect whether two sprites overlap

Storing Sprite Parameters Sprite parameter table is a texture containing all the parameters for all the sprites –Each sprite corresponds to an index into this texture; each texel encodes a parameter set –Store the index of the sprite parameters in the tree leaf data field The RGB value encodes a pointer to the sprite parameter table

Dealing with Overlapping Sprites Leaf vectors table –This 3D texture implements a table of vectors storing the sprite parameters associated with a leaf –2 dimensions: the vector –1 dimension: to index the vector of sprite parameters Instead of directly storing the index of one sprite in the leaves, we now store an index to the leaf vector

Adding a Sprite to the Tree Leaf splitting occurs only in full leaves (O max ) –C max < O max Insertion failures –The maximum depth level is reached only when more than O max sprites are crowded in the same very small region Ordering of sprites –Order the sprites in the leaf vectors

Positioning the Sprites on the Surface Associate a frame to each sprite and use a parallel projection to surface The positioning parameters –A center point c ( sprite handle ) –Two vectors l, h defining the plane, scale, orientation of the pattern –A normal vector n (direction of projection) –M: (l, h, n) U = M -1 ‧ (P-c) –U = (u, v, w): texture coordinates within the pattern

Positioning the Sprites on the Surface Deformation-proof texturing –Store a 3D parameterization (u, v, w) at each vertex and interpolated as usual for any fragment Blending sprites –When multiple sprite overlap, the resulting color is computed by blending together their contributions –Non standard blending modes Blobby-paintingcellular textures

Filtering Linear interpolation (for close viewpoint) –standard texture unites of the GPU MIP-mapping of sprites –standard texture unites of the GPU

Applications and Result Texture authoring –Interactively scaled, rotated, or moved above or below the already existing sprites Lapped texture approximation –The sample is stored only once –N 3 -tree minimizes the memory space required for positioning information –Rendering does not suffer from filtering issues created by atlases

Applications and Result Animated sprites –Modify the positioning parameters (position, orientation, scaling) –A sprite can cycle over a set of texture patterns, simulating an animation in a cartoon-like fashion Snake scales –Estimate the local geometric distortion and scale the sprites accordingly to compensate for the deformation of the animated mesh

Performance # of spritesNode sizeTree depthMax. overlapFPS (800*600) Lapped Gears Stars Blobby Voronoi

Storage Requirements Size of structureAllocated memoryTotal memory Lapped1 MB1.6 MB1.9 MB Gears0.012 MB0.278 MB0.442 MB Stars0.007 MB0.285 MB5.7 MB Blobby0.141 MB0.418 MB0.424 MB Voronoi0.180 MB0.538 MB0.554 MB

Conclusions High texturing resolution at low memory cost Without need for a global planar parameterization The sprite’s attributes are efficiently stored in a hierarchical grid surrounding the object’s surface Flexible –Each sprite can be independently animated, moved, scaled, rotated –Create new texturing effects, such as animated texture, textures reacting to mesh deformations Complete GPU implementation, real-time performance

Future Work Painterly rendering Sprite projector functions For point-based and volumetric representation