1. Multi-Layered Impostors for Accelerated Rendering Xavier Decoret, iMAGIS This is joint work with Gernot Schaufler and Julie Dorsey at MIT and François Sillion at iMAGIS

2. Multi layered impostors for accelerated rendering Complex Environments Paris: 411537 vertices 137179 triangles 32 textures (most 256x256) 6.1 MB geometry

3. Multi layered impostors for accelerated rendering IBR in RT-Graphics Image-based representations simplify the scene Images are unaffected by the depicted scene’s complexity Images are fast to render as textured triangles Images themselves can be generated with hardware This works if a geometric model is available

4. Multi layered impostors for accelerated rendering Previous Work Pre-generated Representations Grossman98, Dally96, Maciel95, Aliaga96, Chen95, McMillan95, Xiong96, Sillion97, Darsa98, Pulli97, Laveau94, Max96, Rafferty98 Dynamically Updated Representations Torborg96, SGI97, Lengyel97, Regan94, Shade96, Schaufler96, Mark97, Mann97 Aliaga99

5. Multi layered impostors for accelerated rendering Impostors (pre-generated) SGI Performer (Billboards) Id Software (3D Sprites) Maciel95

6. Multi layered impostors for accelerated rendering Impostors (dynamically generated) Schaufler95 Schaufler Stürzlinger ‘96

7. Multi layered impostors for accelerated rendering Meshed Impostors [Sillion97] Distant Geometry Depth Map DiscontinuitiesTriangulation

8. Multi layered impostors for accelerated rendering Artefacts in IBR Deformation caused by mesh Resolution mismatch Geometry

9. Multi layered impostors for accelerated rendering Artefacts in IBR Incomplete representation Rubber sheet effects Geometry ?

10. Multi layered impostors for accelerated rendering Artefacts in IBR Image cracks and more … –Static Shading: no highlights no reflections –No moving Objects Geometry

11. Multi layered impostors for accelerated rendering Our Contributions Reducing Rubber Sheet Triangles –Multi Mesh Impostors Reducing distortions to improve quality –Dynamic Update Reduce some of the identified artefacts

12. Multi layered impostors for accelerated rendering The database The geometry is organized into objects We have a set of viewcells The street graph One edge Another edge

13. Multi layered impostors for accelerated rendering Model segmentation Complete GeometryPVS Local model Distant model

14. Multi layered impostors for accelerated rendering Single mesh impostor Distant model......replaced by impostor Combined with local model

15. Multi layered impostors for accelerated rendering Rubber Sheet Triangles due to parallax impostor viewpoint View from above

16. Multi layered impostors for accelerated rendering Locating Rubber Sheet Triangles Parallax creates rubber sheets between objects when objects overlap in depth View from above Front view In urban walkthrough, parallax is mainly horizontal Assume a 2D problem

17. Multi layered impostors for accelerated rendering Multi Mesh Impostor The critical zone identifies overlaps between 2 objects When overlapping occurs in image space, one object can uncover the other one Edge O1 Critical zone O2 Edge O1 O2 If uncovering is too much, objects must not be on the same mesh

18. Multi layered impostors for accelerated rendering Multi Mesh Impostor Quantifying overlapping between 2 objects Place those objects in different layers which are too distant in depth: –construct a relation graph –partition the graph Single meshSeveral meshes

19. Multi layered impostors for accelerated rendering Objects and Layers Two objects must go into different layers if two points on their geometry can be seen under sufficiently different viewing angles  min and  max. Q P A B  max edge  min Object 1 Object 2 y PQ A B x M 1, M 2  [PQ] such as AM 1 B minimal AM 2 B maximal

20. Multi layered impostors for accelerated rendering Relation Graph Edges joining overlapping objects Coloriate the graph so that joined vertices have different colors Each color represent a layer Non unique One node per object

21. Multi layered impostors for accelerated rendering All Geometry Results & Example Edge PVS Single Mesh Multi Mesh

22. Multi layered impostors for accelerated rendering Results & Examples Start point From which the impostor is computed End point Notice rubber sheet triangles

23. Multi layered impostors for accelerated rendering Results & Examples Start point From which the impostor is computed End point The view is no longer blocked

24. Multi layered impostors for accelerated rendering Results & Examples Geometry Single MeshMultiMesh

25. Multi layered impostors for accelerated rendering Offline vs Online If the user stops, the correct image should appear. This requires knowledge of the current viewpoint. Offline approaches do not have this information Motion hides small artifacts of stored impostors.

26. Multi layered impostors for accelerated rendering Combining preprocessing with dynamic updates Single mesh impostor replace too much geometry to be updated. Layers and their contents are suitable for regeneration of part of the distant model. Layers are updated front to back to improve image quality: –silhouettes –distortions –resolution mismatches

27. Multi layered impostors for accelerated rendering System Architecture Preprocessing: Take geometry and view cells Find visible geometry for cells Split into near and far part Create impostors for far part Store as scene per view cell Walkthrough: Page in geometry and textures Do dynamic updates if possible LOD management Minimal Scenegraph per viewcell Visibility Extractor Model Segmentation MMI Extractor Offline View CellsScene Dynamic Update Online Rendering

28. Multi layered impostors for accelerated rendering Results & Video (Paris) Storage requirements : 70 Mbs Computation time : 100 edges per hour Achieved frame rate: 50 Hz

29. Multi layered impostors for accelerated rendering Future Work Smooth transitions between different representations Automatic generation of street graph from street mesh Extension of viewcells from edges to areas and volumes Reverse approach: construction of viewcells to optimize use of impostors.

30. Multi layered impostors for accelerated rendering Acknowledgements This is a joint project between iMAGIS, Grenoble and MIT, Cambridge. It was supported in part by a joint collaborative research grant of NSF and INRIA (INT-9724005), an Alfred P. Sloan Foundation Research Fellowship (BR-3659), and by a grant from Intel Corporation. The following people contributed to the presented results : Julie Dorsey, François Sillion, Gernot Schaufler, Max Chen, Byong Oh Mok, Yann Argotti and Sami Shalabi.