Real-time Walkthrough of Virtual Space using Environment Map CG Lab. ChungAng University Ryoo, Seung Taek 2000. 10. 4 I am from Computer Graphics Lab. In ChungAng University, Korea I’m talking about Real-time walkthrough of virtual space using Environment map.

Introduction: Real-time Walkthrough High quality architectural walkthroughs require large complex models with many geometry primitives Renders real-time walkthrough of virtual space difficult Visibility Culling Close objects from viewpoint: use geometry models Distant objects Applying the LODs(Level Of Details) Replacing them with images (Image Based Representation) High quality architectural walkthroughs require large complex models with many geometry primitives It is difficult to render them real-time walkthrough To overcom this problem, Mechanisms such as visibility culling, using geometry models for close objects from viewpoint and applying the LODs or replacing them with images for distant objects are used The thechnique replacing with image is called Image Based Representation Image-based representation expresses the object using a 2-dimensional image instead of a geometry model to make photorealistic and real-time rendering possible VSMM 2000

Introduction: Image Based Representation Related Works Sprite Imposter Nailboards Portal Textures image caches Geometry model LDI Layered Sprite Sprite Texture mapping Sprite with depth Viewpoint Examples of real-time walkthrough algorithms using IBR techniques are imposters, image caches, nailboards, sprites, portal textures and so forth. The right figure show how to use image based primitives according to distance between viewpoint and objects VSMM 2000

Realtime Walkthrough System Realtime walkthrough algorithm Virtual Space Modeling Preprocessing Cell & Portal Division Images Capture Walkthrough Collision Detection I suggests the real-time walkthrough algorithm of virtual space using environment maps This figure shows Overview of realtime walkthrough algorithm First, the virual space is modelled into a hierarchical tree structure. And then, The transformed model is classified into cells and portals Cells means room, portals means doors or windows in buliding The scene seen through the portal is captured to express the outler layer of the cell. Next step is Culling. The data that is not necessary for rendering is discarded through view-frustum culling and portal culling After Culling, The models still have a large amount of geometry data. To resolve this, only a cell with a current viewpoint is expressed as geometry models and the neighboring cells visible through the portal are replaced with images. The algorithms replacing geometry visible through portal are texture map, image warping, layer depth image, and environment map method. I’ll express details about this repacement algorithm next cahpter. And also, collision detection is carried out through an intersection test using the bounding volume. The above process is applied repeatedly to create a continuous walkthrough scene. View-Frustum Culling Geometry Model Cell Cell + Portal Texture morphing Renderer Portal Culling Cell/Portal Replacement Portal Texture map Image Warping Layered Depth Image Environment Map VSMM 2000

Virtual Space Modeling hierarchical structuring of the models buildings, rooms(cell), objects in a room, faces of an object, and each point that consists of a face The current cell : geometry models the rest : replacement with images Portal Cell1 Cell2 Wall 2 7 Wall1 4 5 Wall6 Wall 3 Wall8 Cell1_Obj1 Cell1_Obj2 Cell2_Obj1 Cell2_Obj2 Images Hierarchical structuring means structuring the virtual space model into a hierarchical tree consisting of layers such as buildings, rooms, objects in a room, faces of an object, and each point that consists of a face Left tree show whole geometry model. Right tree shows the appearance the virtual space model has been replaced with the images The current cell has been expressed using geometry models and the rest has been replaced with images VSMM 2000

Replacement algorithms: current algorithms and its drawback Replacing geometry visible through portal Texture Map Single Texture Simple and easy to implement But, appears more like a large painting hanging on the wall than like another room Multiple Textures popping effect problem image warping method gaps that appear in the warped image cannot guarantee correct visibility LDIs method extra cost of applying the warping algorithm A B In this section, we will take a look at the characteristcs of the replacement algorithm and its drawbacks. The most common method is to use a single image for each portal. This method is simple and easy to implement but the single texture appears more like a large painting hanging on the wall than like another room. Multiple Textures are applied in order to overcome the above problem. However, this method creates a popping effect problem The warping method and the LDI method are used to resolve this problem. But, gaps appear in the warped image such as the right figure some areas of the scene that should have become visible were not sampled in the reference image And cannot guarantee correct visibility for the image created from the required viewpoint Also, the extra cost of applying the warping algorithm is required during walkthrough of virtual space VSMM 2000

Replacement algorithms: Environment Map Replacement algorithm using environment maps Environment Map A map that has been created by stitching a multiple number of images acquired by rotating the camera by 360 degrees from a fixed viewpoint to simplify the computations of specular reflections on a shiny object from a distant scene. to express new images with a composition of 2D images as an image-based representation mechanism To resolve the above mentioned problems, we propose a portal replacement algorithm using an environment map. An environment map is a map that has been created by stiching a multiple number of images acquired by rotating the camera by 360 degrees from a fixed viewpoint. The environment map was initially used in computer graphics to simplify the computations of specular reflections on a shiny object from a distant scene. It has been used to express new images with a composition of 2-D images as an image-based representation mechanism. VSMM 2000

Replacement algorithms: Environment Mapping The objects used for mapping the environment map Spherical projection Not suitable for storage on a computer. those which map onto a plane with consistent neighborhood relationships are generally quite distored. Cube oriented planar projections easily stored and accessed by a computer oversampled in the edges and corners. Cylindrical projection easily unrolled into a simple planar map. without boundaries in the azimuth direction. the boundary conditions introduced at the top and bottom. The objects used for mapping the environment map are a sphere, cylinder and a cube. Spherical projection is due to the lack of representation that is suitable for storage on a computer. This is particularly difficult if a uniform discrete sampling is required. Furthermore, those which map onto a plane with consistent neighborhood relationships are generally quite distored. Cube oriented planar projections can be easily stored and accessed by a computer but it is oversampled in the edges and corners. For cylindrical projection, a cylinder can be easily unrolled into a simple planar map. The surface is without boundaries in the azimuth direction. One shortcoming of a projection on a finite cylindrical surface is the boundary conditions introduced at the top and bottom Referenced image from Plenoptic modeling by Leonard McMillian VSMM 2000

Replacement algorithm using environment maps: Cylindrical Mapping Hemi-Cylindrical Mapping the height of the viewpoint is constant Using a hemi-cylinder the environment map applied to the portal only requires 180 degrees of the image visible through the portal The environment map has been mapped onto the hemi-cylinder hemiCylinder is set in the back of the portal that resides in the current cell ViewPoint Env Map Portal Current Cell Neighbor Cell The height of the viewpoint in the walkthrough of the virtual space is constant and therefore is not greatly effected by the problems caused by cylinder mapping In this study, a hemi-cylinder has been used for mapping the environment map. The reason for using a hemi-cylinder is that the environment map applied to the portal only requires 180 degrees of the image visible through the portal The environment map has been mapped onto the hemi-cylinder HemiCylinder is set in the back of the portal that resides in the current cell, as shown in the right figure The scene visible through the portal is a hemi-cylinder to which an environment map has been mapped onto VSMM 2000

Creation of Environment Map Replacement algorithm using environment maps: Creation of Environment Map Creation of Environment Map the cylinder which the environment map is to be mapped onto is set to the center of the portal. The viewpoint and the view volume is set to the boundary of the cylinder to acquire the scene visible through the portal. The scene from the viewpoint of the boundary of the cylinder is gathered by a certain degree. The gathered image is combined to create the environment map which is then mapped onto the hemi-cylinder. The process for creating an environment map is shown in figure below. First of all, the cylinder to which the environment map is to be mapped onto is set to the center of the portal. The viewpoint and the view volume is set to the boundary of the cylinder to acquire the scene visible through the portal. The scene from the viewpoint of the boundary of the cylinder is gathered by a certain degree. The gathered image is combined to create the environment map which is then mapped onto the hemi-cylinder. VSMM 2000

Our Walkthrough System Applying the environment map to the portal Eliminate the popping effect Remove the holes in the image No extra cost needed to render the virtual space The creation of the environment map in the preprocessing phase Problem cannot guarantee the correct visibility of the image created from the required viewpoint. re-structuring the environment map using depth information. Our method using environment map resolve some problem. The popping effect can be eliminated and the holes caused by the parallax have been eliminated by applying the environment map on replacement algorithm. Also, because the environment maps create in the preprocessing phase No the extra cost needed to render the virtual space The method using environment maps cannot guarantee the correct visibility of the image created from the required viewpoint. To resolve the problem caused by the parallax, re-structuring the environment map using depth information is needed for future work.   VSMM 2000

Wireframe: Geometry vs. Portal Image vs. Environment Map This figure compares the virtual space model with the geometry model, portal texture and the proposed method of applying an environment map to express the scene. Geometry Model Portal Image VSMM 2000 Env Map

Rendering: Geometry vs. Portal Image vs. Environment Map The image acquired using the environment map method is closer to that of the image expressed using the geometry model than the image acquired using portal texture method. Geometry Model Portal Image VSMM 2000 Env Map

Collision Detection Collision detection of the viewpoint is carried out objects within the cell Each Object: Bounding Sphere walls consisiting cell Bottom Wall: Bounding Plane portals that make up the cell Each Portal: Bounding Sphere Collision detection of the viewpoint is carried out on the objects within the cell, walls consisiting cell and the portals that make up the cell, as shown in right figure. Collision detection on the objects within the cell is carried out through the bounding sphere. For walls, collision test is carried out on the bottom wall through an inside/outside test A bounding sphere is applied to the portal area for collision detection of the portal. If the viewpoint exists outside the bounding sphere area of the portal, the walkthrough scene should be created using geometry models and portal images of the current cell. if the viewpoint exists inside the bounding sphere area of the portal, the walkthrough scene is created using geometry models and the portal images of the current cell and the next cell But, the portals shared by the current cell and the next cell are not replaced with portal images. VSMM 2000

Implementation Implementation Test Model VRML file as the input data for modeling A hierarchical structuring of the models OpenGL for rendering geometry model Bounding volume for collision detection Test Model The number of cells: 13 consists of 36 portals and 180,000 polygons We have used VRML file as the input data for modeling the virtual space and OpenGL for rendering geometry model Bounding volume has been used for collision detection. To implement and test the repalcement algorithm of geometry visible through portal. The inside of the virtual building has been modelled as shown in table below The number of cells of the virtual building is 13, and consists of 36 portals and 180,000 polygons. VSMM 2000

Geometry Model vs Portal Images vs Environment Map Result 1 Geometry Model vs Portal Images vs Environment Map This figure is the result of the realtime walkthtough algorithm based on the virtual building model explained above. It shows the resulting images of applying the geometry model, portal texture algorithm and the environment map algorithm respectively. As you can see from comparing the images, the method of applying the environment map can create a continuous image from a new viewpoint without any extra cost and exposure caused by the lack of information. VSMM 2000

Result 2 VSMM 2000 Geometry Model (Flythrough) Geometry Model (Walkthrough) Environment Map(Flythrough) Environment Map (Walkthrough) This figure compares the virtual space model with the geometry model and environment map algorithm Left figure shows the view of the flythrough, Right figure shows the view of the walkthrough VSMM 2000

Demo This one is the scene that represent the virtual space model with the geometry It shows the scene using environment maps It is the scene that show how to use the environment map method This one compares the virtual space model with the geometry model and environment map algorithm using bird eye’s view VSMM 2000

Conclusion & Future Work Real-time walkthrough algorithm of virtual space using an environment map The popping effect and the holes in the image can be eliminated It eliminates the extra cost needed to render the virtual space. Future Work the problem caused by the parallax re-structuring the environment map using depth information. Extending the algorithm to the outside of the builing is needed. Extending the applicable range to the dynamic environment should also be investigated in the future. I have proposed the real-time walkthrough algorithm of virtual space using an environment map The popping effect and the holes in the image can be eliminated It eliminates the extra cost needed to render the virtual space. The method using an environment map cannot guarantee the correct visibility of the image created from the required viewpoint. To resolve the problem caused by the parallax, re-structuring the environment map using depth information would be required. Also, further investigation on extending the algorithm to the outside of the builing is needed. An algorithm for further extending the applicable range to the dynamic environment should also be investigated in the future. VSMM 2000