Image-Based Techniques Hierarchical Image Caching Michael Chung

Papers Hierarchical Image Caching for Accelerated Walkthroughs of Complex Environments –Jonathan Shade, Dani Lischinski, David H. Salesin, Tony DeRose, John Snyder (SIGGRAPH, 1996) A Three Dimensional Image Cache for Virtual Reality –Gernot Schaufler, Wolfgang Sturzlinger (EUROGRAPHICS, 1996)

Motivation Unable to render large and complex environments at a sufficiently high frame rate. Would be nice to be able to. (ex: crowded forests, cities…)

Insights Dynamically Generated Impostors (Schaufler, 1995) –Path coherence between frames –No need to render every frame from scratch

Impostor – What is it? An impostor is a billboard that mimics and replaces 3d geometry in a scene.

Impostor – Why use it? Fast to render Reusable for objects that are coherent between frames –Especially distant objects Essentially, a high quality image at the cost of a very low LOD model (a quad) –But the impostor is short-lived.

Impostor Generation Face viewpoint towards center of bounding box Project bounding box into screen space Generate tight rectangle Reverse project rectangle into world space Render impostor texture onto rectangle

Example: Virtual Dublin Project Geopostors: A Real-Time Geometry / Impostor Crowd Rendering System Simon Dobbyn, John Hamill, Keith O’Conor, Carol O’Sullivan

Insights Dynamically Generated Impostors (Schaufler, 1995) –Path coherence between frames –No need to render every frame from scratch

Insights Dynamically Generated Impostors (Schaufler, 1995) –Path coherence between frames –No need to render every frame from scratch New insight: –Further optimize rendering using a hierarchical cache of impostors Creating impostors out of smaller impostors saves time over rendering actual geometry If possible, draw fewer impostors per frame

Proposed Contributions Shade, Lischinski, Salesin, DeRose, Snyder “New method for accelerating walkthroughs of geometrically complex and largely unoccluded static scenes by hierarchically caching images of scene portions.” “New simple error metric that provides automatic quality control.”

Proposed Contributions Schaufler, Sturzlinger “New approach to software accelerated rendering which improves and generalizes the concept of impostors” –Handles intersecting objects –Handles indoor as well as outdoor scenes –Deals with primitives, does not rely on organization of scene into object groupings

Proposed Contributions Schaufler, Sturzlinger Indoor intersecting objects problem:

Proposed Contributions Schaufler, Sturzlinger With the 3d image cache:

Method Overview Partition scene into a tree (BSP-tree or k-d-tree) For each frame: –First traversal of tree Cull nodes outside of the view frustum (optional) Update the cached impostors of nodes –Second traversal of tree Render scene into framebuffer, back to front

Algorithm Details Shade, Lischinski, Salesin, DeRose, Snyder Partition scene into BSP-tree –All splitting planes perpendicular to x and z axes. –Place splitting planes in the “best” gap found. Compute a cost for each gap that is a function of: –the number of its active objects –and, the ratio of the number of primitives on either side of the gap –Choose best gap such that the longest side of the bounding box is split.

Algorithm Details Schaufler, Sturzlinger Partition scene into k-d-tree –Recursively subdivide along longest side. –If primitive intersects more than one bounding box, store primitive in all intersected bounding boxes.

Updating Image Caches Shade, Lischinski, Salesin, DeRose, Snyder UpdateCaches(node, viewpoint) if node is outside the view frustum then node.status  CULL else if node.cache is valid for viewpoint then node.status  DRAW CACHE else if node is a leaf then UpdateNode(node, viewpoint) else UpdateCaches(node.back, viewpoint) UpdateCaches(node.front, viewpoint) UpdateNode(node, viewpoint)

Updating Image Caches A Visual Example

Updating Image Caches Shade, Lischinski, Salesin, DeRose, Snyder UpdateCaches(node, viewpoint) if node is outside the view frustum then node.status  CULL else if node.cache is valid for viewpoint then  DETERMINE VALIDITY? node.status  DRAW CACHE else if node is a leaf then UpdateNode(node, viewpoint) else UpdateCaches(node.back, viewpoint) UpdateCaches(node.front, viewpoint) UpdateNode(node, viewpoint)

Determining Impostor Validity Shade, Lischinski, Salesin, DeRose, Snyder Error metric In practice, only consider the eight corners of a node’s bounding box when computing error.

Determining Impostor Validity Error threshold Impostor can be used if… – –AND, imposter is not too close to the viewpoint Too close if tex >= screen (Schaufler, Sturzlinger) OR, too close if viewpoint is in the same node (Shade et al)

Updating Image Caches Shade, Lischinski, Salesin, DeRose, Snyder UpdateCaches(node, viewpoint) if node is outside the view frustum then node.status  CULL else if node.cache is valid for viewpoint then node.status  DRAW CACHE else if node is a leaf then UpdateNode(node, viewpoint) else UpdateCaches(node.back, viewpoint) UpdateCaches(node.front, viewpoint) UpdateNode(node, viewpoint)  CACHE NEW IMAGE OR NOT?

Updating Nodes Shade, Lischinski, Salesin, DeRose, Snyder UpdateNode(node, viewpoint) if viewpoint is inside node then if node is a leaf then node.status  DRAW GEOMETRY else node.status  RECURSE return k  EstimateCacheLifeSpan(node, viewpoint) amortizedCost  (cost to create cache) / k + (cost to draw cache) if (amortizedCost < (cost to draw contents) then CreateCache(node, viewpoint) node.status  DRAW CACHE node.drawingCost  (cost to draw cache) else if node is a leaf then node.status  DRAW GEOMETRY node.drawingCost  (cost to draw geometry) else node.status  RECURSE node.drawingCost  node.back.drawingCost + node.front.drawingCost

Updating Nodes Shade, Lischinski, Salesin, DeRose, Snyder UpdateNode(node, viewpoint) if viewpoint is inside node then if node is a leaf then node.status  DRAW GEOMETRY else node.status  RECURSE return k  EstimateCacheLifeSpan(node, viewpoint)  HOW TO ESTIMATE LIFE SPAN? amortizedCost  (cost to create cache) / k + (cost to draw cache) if (amortizedCost < (cost to draw contents) then CreateCache(node, viewpoint) node.status  DRAW CACHE node.drawingCost  (cost to draw cache) else if node is a leaf then node.status  DRAW GEOMETRY node.drawingCost  (cost to draw geometry) else node.status  RECURSE node.drawingCost  node.back.drawingCost + node.front.drawingCost

Estimating Impostor Life Span Shade, Lischinski, Salesin, DeRose, Snyder Approximate the safety zone around current viewpoint.

Estimating Impostor Life Span Shade, Lischinski, Salesin, DeRose, Snyder Approximating safety zone for a leaf node –Evaluate d for each corner of node’s bounding volume. –Let r be the smallest such d. –Set the safety zone to be the axis-aligned cube inscribed inside a sphere of radius r around current viewpoint.

Estimating Impostor Life Span Shade, Lischinski, Salesin, DeRose, Snyder Approximating safety zone for an interior node. –Compute safety zone Z using the bounding box of the node. –Intersect Z with the safety zones of the node’s children

Main Algorithm Differences Shade et al. –View frustum culling –Cost-benefit analysis for cache update Schaufler, Sturzlinger –No view frustum culling –No cost-benefit analysis for cache update Replace invalid impostors with new ones no matter what

Questions so far? Method Overview Partition scene into a tree (BSP-tree or k-d-tree) For each frame: –First traversal of tree Cull nodes outside of the view frustum (optional) Update the cached impostors of visible nodes –Second traversal of tree Render scene into framebuffer, back to front

Tests and Analysis of Results Shade, Lischinski, Salesin, DeRose, Snyder Silicon Graphics Indigo2 workstation –250 MHz R4400 processor –320 MB RAM –Maximum Impact graphics board with 4 MB texture memory Outdoor Scene (island forest) –1117 willow trees –40,599,982 triangles

Tests and Analysis of Results Shade, Lischinski, Salesin, DeRose, Snyder

BSP-tree Preprocessing –46 seconds construction time –No splitting of trees, only terrain is split –13 levels, 1072 leaf nodes –150 MB storage space Walkthroughs –640 x 480 frame resolution –2 pixel error threshold

Tests and Analysis of Results Shade, Lischinski, Salesin, DeRose, Snyder Actual geometry Image cache 2 pixel error threshold

Tests and Analysis of Results Shade, Lischinski, Salesin, DeRose, Snyder Actual geometry Image cache 8 pixel error threshold

Tests and Analysis of Results Shade, Lischinski, Salesin, DeRose, Snyder Actual geometry Image cache 2 pixel error threshold

Tests and Analysis of Results Shade, Lischinski, Salesin, DeRose, Snyder Actual geometry Image cache 8 pixel error threshold

Tests and Analysis of Results Shade, Lischinski, Salesin, DeRose, Snyder

Tests and Analysis of Results Schaufler, Sturzlinger INDIGO R3000 –39k triangles per second –No hardware support for texture mapping One cache update visualization Two performance tests

Tests and Analysis of Results Schaufler, Sturzlinger Cache update visualization Diagonal walkthrough of some complex scene k-d-tree preprocessing –341 bounding boxes 256 x 256 maximum texture resolution Impostor updates for every 6 th frame of 120 frames shown…

Tests and Analysis of Results Schaufler, Sturzlinger Cache update visualization

Tests and Analysis of Results Schaufler, Sturzlinger

Two performance tests –Make up for hardware insufficiencies 1 st test: Render at a small screen resolution 2 nd test: Replace each textured polygon with two gouraud shaded polygons –Sideward translation in front of forest (100 frames) –Zoom from one corner of forest towards its center (100 frames) –1 pixel threshold

Tests and Analysis of Results Schaufler, Sturzlinger Procedurally generated forest scene –100 trees –Approximately polygons k-d-tree preprocessing –Max depth of 4 –585 bounding boxes

Tests and Analysis of Results Schaufler, Sturzlinger

Comments on Papers Shade et al. –Relatively thorough explanation of proposed method –No justification of formulas for calculating cached image life expectancy. Schaufler, Sturzlinger –Insufficient explanation of testing procedure –Confusing graphs

Summary

Hierarchical image caching –Impostors are cheap mimics of complex 3d geometry. Take advantage of path coherence –Caching impostors hierarchically further saves rendering time Allows generation of impostors from children impostors –Scene partitions are disjoint and does not rely on object groupings Able to handle intersecting objects Automatic cache update control using error metric based on angular discrepancy Improved frame rates for large, complex static scenes Following work impacted by the insight that large and complex geometry can be efficiently rendered by hierarchically caching cheap representations

Summary Hierarchical image caching –Impostors are cheap mimics of complex 3d geometry. Take advantage of path coherence –Caching impostors hierarchically further saves rendering time Allows generation of impostors from children impostors –Scene partitions are disjoint and does not rely on object groupings Able to handle intersecting objects Automatic cache update control using error metric based on angular discrepancy Improved frame rates for large, complex static scenes Following work impacted by the insight that large and complex geometry can be efficiently rendered by hierarchically caching cheap representations

Summary Hierarchical image caching –Impostors are cheap mimics of complex 3d geometry. Take advantage of path coherence –Caching impostors hierarchically further saves rendering time Allows generation of impostors from children impostors –Scene partitions are disjoint and does not rely on object groupings Able to handle intersecting objects Automatic cache update control using error metric based on angular discrepancy Improved frame rates for large, complex static scenes Following work impacted by the insight that large and complex geometry can be efficiently rendered by hierarchically caching cheap representations

Summary Hierarchical image caching –Impostors are cheap mimics of complex 3d geometry. Take advantage of path coherence –Caching impostors hierarchically further saves rendering time Allows generation of impostors from children impostors –Scene partitions are disjoint and does not rely on object groupings Able to handle intersecting objects Automatic cache update control using error metric based on angular discrepancy Improved frame rates for large, complex static scenes Following work impacted by the insight that large and complex geometry can be efficiently rendered by hierarchically caching cheap representations

Summary Hierarchical image caching –Impostors are cheap mimics of complex 3d geometry. Take advantage of path coherence –Caching impostors hierarchically further saves rendering time Allows generation of impostors from children impostors –Scene partitions are disjoint and does not rely on object groupings Able to handle intersecting objects Automatic cache update control using error metric based on angular discrepancy Improved frame rates for large, complex static scenes Following work impacted by the insight that large and complex geometry can be efficiently rendered by hierarchically caching cheap representations

Summary Hierarchical image caching –Impostors are cheap mimics of complex 3d geometry. Take advantage of path coherence –Caching impostors hierarchically further saves rendering time Allows generation of impostors from children impostors –Scene partitions are disjoint and does not rely on object groupings Able to handle intersecting objects Automatic cache update control using error metric based on angular discrepancy Improved frame rates for large, complex static scenes Following work impacted by the insight that large and complex geometry can be efficiently rendered by hierarchically caching cheap representations

Summary Hierarchical image caching –Impostors are cheap mimics of complex 3d geometry. Take advantage of path coherence –Caching impostors hierarchically further saves rendering time Allows generation of impostors from children impostors –Scene partitions are disjoint and does not rely on object groupings Able to handle intersecting objects Automatic cache update control using error metric based on angular discrepancy Improved frame rates for large, complex static scenes Following work impacted by the insight that large and complex geometry can be efficiently rendered by hierarchically caching cheap representations

Questions?