Billboard Clouds Xavier Décoret† Frédo Durand† Francois Sillion Julie Dorsey‡ †MIT-CSAIL Artis (INRIA/CNRS/UJF/INPG) ‡ Yale university

What this is not about!

What this is about New representation: Rectangles  global shape Textures with a  finer details (silhouette) + appearance Take more to explain what it is Explain why we use the term “cloud” Geometry is captured with plane AND textures

Mesh Simplification Clustering [RB93,LT97] Hierarchical Dynamic Simplification [LE97] Decimation of Triangle Meshes [SZL92] Re-tiling [Tur92] Progressive Meshes [Hop96,PH97] Quadric Error Metrics [GH97] Out of Core Simplification [Lin00] Comprehensive Voxel based reconstruction [HHK+95] Multiresolution analysis [EDD+95] Superfaces [KT96], face cluster [WGH00]

Mesh Simplification Constraints on models Error control Simplification envelopes [CVM96] Permission Grids [ZG02] Image driven [LT00] Handling of attributes (textures and colors) Integration to the metric[GH98][Hop99] Re-generation [CMRS98,COM98] Extreme Simplification Silhouette Clipping [SGG+00] Go faster + images (oversimplified) “Error that you incur”

Alternative Rendering Image-based rendering Lightfield,Lumigraph [LH96,GGRC96] Impostors [Maciel95,Aliaga96,DSSD99] Relief Textures [OB00] Point-based rendering Surfels [PZBG00] Pointshop 3D [ZPKG02]

Classic billboards A modelling “trick” [RH94] Generalization to many planes / formalism Automatic construction

Classic Billboards Pause For a very non convex model

Classic Billboards

Classic Billboards Hide the gaps under the carpet

Principle Illustrated in 2D polygonal model

Simplification by planes Principle Simplification by planes polygonal model

Principle (1) Allow vertex displacement Maximum displacement P

Valid approximation by a plane Principle (2) Project faces onto planes Valid approximation by a plane Face

Valid approximation by a plane Principle (2) Project faces onto planes Valid approximation by a plane

Problem How many planes? Which planes?

Overview Express as an optimization problem Represent the space of planes Measure a plane’s relevance Find a set of planes

Optimization problem Define over the set of Billboard clouds: An error function Vertex displacement A cost function Number of planes Error Based: bound max error  minimize cost

Overview Express as an optimization problem Represent the space of planes Dual representation Discretization Measure a plane’s relevance Find a set of planes

Dual representation Dual space  Primal space Illustrated in 2D Hough transform [Hough62] Dual space   2p Dual of line = point  Line    Origin Primal space

Dual of a point Primal space Dual space Set of lines going through the point  (xP,yP)  Origin 2p Primal space Dual space

Dual of a point Primal space Dual space Set of lines going through the point  (xP,yP)  Origin 2p Primal space Dual space

Dual of a point Primal space Dual space Set of lines going through the point  (xP,yP)  Origin 2p Primal space Dual space

Dual of a point Primal space Dual space Set of lines going through the point  (xP,yP)  Origin 2p Primal space Dual space

Dual of a point Primal space Dual space Set of lines going through the point  r = xPcosq +yP sinq (xP,yP)  Origin 2p Primal space Dual space

Dual of a point Primal space Dual space Set of lines going through the point  r = xPcosq +yP sinq r  0 (xP,yP)  Origin 2p Primal space Dual space

Dual of a sphere Primal space Dual space Set of lines intersecting the sphere  R P  Origin 2p Primal space Dual space

Dual of a sphere Primal space Dual space Set of lines intersecting the sphere  R Dual of center P P  Origin 2p Primal space Dual space

Dual of a sphere Primal space Dual space Set of lines intersecting the sphere  R 2R Dual of center P P  Origin 2p Primal space Dual space

Dual of sphere = 2R-thick slice Dual of a sphere Set of lines intersecting the sphere  Dual of sphere = 2R-thick slice R P  Origin 2p Primal space Dual space

Dual of a face Primal space Dual space Planes intersecting all vertices’ spheres  R P’ R P  Origin 2p Primal space Dual space

Dual of a face Primal space Dual space Planes intersecting all vertices’ spheres  R P’ R P Don’t’ mention dual space when you say planes  Origin 2p Primal space Dual space

Dual of a face How to work with this complex set of planes?  R P’ R P  2p How to work with this complex set of planes?

Discretization How to work with this complex set of planes?  Bins R  2p How to work with this complex set of planes?

Discretization How to work with this complex set of planes?  R P’ R P  2p How to work with this complex set of planes?

Overview Express as an optimization problem Represent the space of planes Dual representation Discretization Measure a plane’s relevance Density function Find a set of planes

Discretization How to work with this complex set of planes?  R P’ R P  2p How to work with this complex set of planes?

Discretization R P P’ R P P’ R P P’ R P P’ R P P’ R P P’  R P P’  2p

There is (at least) one plane valid for the face Discretization  A tagged bin indicates that: There is (at least) one plane valid for the face  2p Relevance of this plane?

Use projected area of face (on central plane) Relevance  Grey plane is a better approximation of face ! Use projected area of face (on central plane)  Density function 2p

Density function Compute in plane space (a float per bin) Represent the relevance of each plane Accumulate face contributions into the bins

Density function + - Faces   Density Planes valid for face Vary -> vari  

Density function Faces Planes valid for face Density + -  

Density function Faces Planes valid for face Density + -  

Density function Faces Planes valid for face Density + -  

Density function Faces Planes valid for face Density + -  

Density function Faces Planes valid for face Density + -  

Density function Faces Planes valid for face Density + -  

Density function Faces Planes valid for face Density +  - 

Density function Faces Planes valid for face Density + -  

Density function Faces Density + -  

Overview Express as an optimization problem Represent the space of planes Hough transform Discretization Measure a plane’s relevance Density function Find a set of planes Greedy selection of best plane

Density function Faces Bin with highest density  

Density function Faces Faces for which bin is valid Bin with highest  

Density function How to find this plane?  Faces High density There is probably a plane valid for all the faces  How to find this plane?

Adaptive search How to find this plane?  Faces Test central plane High density There is probably a plane valid for all the faces  Subdivide Local density recomputation How to find this plane?

Algorithm Details more the Build Billboard construction Adaptive “For example using OpenGL”

2D 3D The Full Monty (3D) Faces Segments Triangles Primal Lines Planes Dual , ,f, Density function  r Dual space Primal space  f Origin q,f

A Simple Example Talk all the time

Texture optimization Working in dual space can group faces that are far away in primal space + No need for connectivity - Can potentially waste texture space & fill rate Billboard Wasted space (transparent)

Texture optimization Working in dual space can group faces that are far away in primal space + No need for connectivity - Can potentially waste texture space & fill rate Billboards

Texture Optimization

Texture Optimization Connected component

Results (1) Shadows = trick And no self shadowing Synchronize We are using a cloud of # billboards Mention the gaps now Realistic Emphasize we show full screen

Results (2)

Results (2)

Results (2) Billboard Cloud Polygons

Limitations Gaps Texture memory If # of planes is too small Project faces on multiple planes Texture memory Other methods need resampling as well Overhead = transparent texels (~30%) Use texture packing

Conclusions New representation Arbitrary models Automatic construction Rectangles  global shape Textures  finer details (silhouette) + appearance Arbitrary models Automatic construction Simple error criterion Extreme simplification

Future work Texture compression View-dependent billboard clouds Hardware compression Adaptive Texture Maps [KrausErtl02] Texture packing View-dependent billboard clouds Multi-meshed impostors [DSSD99] Stochastic density computation Application to collision detection e.g intersection of a ray ~ texture lookup

Acknowledgments INRIA NSF EIA-9802220 NTT (partnership MIT9904-30) Addy Ngan, Ray Jones, Eric Chan people at MIT Reviewers Thank you