Visibility Culling III: Image-Space Occlusion David Luebke Computer Science Department University of Virginia

D2 Review: General Occlusion Culling When cells and portals don’t work…When cells and portals don’t work… –Trees in a forest –A crowded train station Need general occlusion culling algorithms:Need general occlusion culling algorithms: –Aggregate occlusion –Dynamic scenes –Non-polygonal scenes

D3 Review: Hierarchical Z-Buffer A Z-pyramid represents depth buffer in hierarchical fashionA Z-pyramid represents depth buffer in hierarchical fashion –Depth at each pixel at a given level represents the max of the depth of the four corresponding pixels at next level –Rasterize triangles into buffer hierarchically with early termination

D4 Review: Hierarchical Z-Buffer Organize scene into an octree (a kind of spatial hierarchy)Organize scene into an octree (a kind of spatial hierarchy) –Before rendering the polygons associated with a node in the octree, test if the node’s bbox is visible –Do an occlusion query to “test render” the faces of the bounding box against the depth-buffer –If query fails, bbox would not be visible and we can skip the node’s geometry

D5 Review: Hierarchical Z-Buffer HZB algorithm is most effective when Z-pyramid is “mostly complete”HZB algorithm is most effective when Z-pyramid is “mostly complete” –Start by rendering nodes visible last frame directly into full-res Z-buffer –Build Z-pyramid from full-res Z-buffer –Traverse octree front-to-back, using Z-query to test nodes and render if needed Note: also test nodes already rendered to see if they have become invisible Note: also test nodes already rendered to see if they have become invisible

D6 Review: Hierarchical Z-Buffer HZB algorithm exploits:HZB algorithm exploits: – Image-space coherence : Polygons occluded in one pixel are probably occluded in nearby pixels Polygons occluded in one pixel are probably occluded in nearby pixels – Object-space coherence: Polygons near an occluded polygon are probably also occluded Polygons near an occluded polygon are probably also occluded – Temporal coherence: Polygons visible recently are probably still visible Polygons visible recently are probably still visible

D7 Hierarchical Occlusion Maps A more hardware-friendly general occlusion culling algorithmA more hardware-friendly general occlusion culling algorithm Two major differences from HZB:Two major differences from HZB: –Separates occluders from occludees –Decouples occlusion test into an depth test and a overlap test

D8 Hierarchical Occlusion Maps Occluders versus occludees:Occluders versus occludees: Blue parts: occluders Red parts: occludees

D9 Hierarchical Occlusion Maps Depth versus overlap:Depth versus overlap: View Point Z X Y Depth + Overlap = Occlusion Depth + Overlap = Occlusion

D10 Hierarchical Occlusion Maps Representation of projection for overlap test: occlusion mapRepresentation of projection for overlap test: occlusion map –Corresponds to a screen subdivision –Records average opacity per partition Generate by rendering occludersGenerate by rendering occluders –Record pixel opacities (i.e., coverage)

D11 Rendered Image Occlusion Map Occlusion Maps

D12 Occlusion Map Pyramid Analyzing cumulative projection:Analyzing cumulative projection: –A hierarchical occlusion map (HOM) –Generate by recursive averaging (once per frame) –Records average opacities for blocks of multiple pixels, representing occlusion at multiple resolutions –Construction can be accelerated by texture hardware

D13 Occlusion Map Pyramid 64 x 6432 x 3216 x 16

D14 Occlusion Map Pyramid

D15 Overlap Tests Query: is projection of occludee inside cumulative projection of occluders?Query: is projection of occludee inside cumulative projection of occluders? –Cumulative projection: occlusion pyramid –Occludee projection: expensive in general Overestimate occludee with 3-D bounding box Overestimate occludee with 3-D bounding box Overestimate projection of 3-D bounding box with 2-D bounding rectangle in screen-space Overestimate projection of 3-D bounding box with 2-D bounding rectangle in screen-space

D16 Overlap Tests Hierarchical structure enables some optimizations:Hierarchical structure enables some optimizations: –Predictive rejection Terminate test when it must fail later Terminate test when it must fail later –Conservative rejection The transparency threshold The transparency threshold –Aggressive Approximate Culling Ignore objects barely visible through holes Ignore objects barely visible through holes The opacity threshold The opacity threshold

D17 Aggressive Approximate Culling

D18 Hierarchical Occlusion Maps Not discussed here:Not discussed here: –Depth test Depth estimation buffer Depth estimation buffer Modified Z-buffer Modified Z-buffer –Selecting occluders For more details, see Hansong Zhang’s dissertation at UNCFor more details, see Hansong Zhang’s dissertation at UNC

D19 HOM: Discussion Provides a robust, general, hardware- friendly occlusion culling algorithmProvides a robust, general, hardware- friendly occlusion culling algorithm –Supports dynamic scenes –Supports non-polygonal geometry –Few hardware assumptions

D20 HOM: Discussion Not clear how relevant the actual algorithm is on modern hardwareNot clear how relevant the actual algorithm is on modern hardware Most interesting/important points:Most interesting/important points: –Separation of occluders and occludees –Factoring occlusion into overlap + depth –Aggressive approximate culling To think about:To think about: –How could we simplify the occluders to accelerate occlusion testing?

D21 Current Hardware Support Recent hardware supports Z-query operationRecent hardware supports Z-query operation –Allows systems to exploit: Object-space coherence (bounding boxes) Object-space coherence (bounding boxes) Temporal coherence (last-rendered list) Temporal coherence (last-rendered list) –Examples in OpenGL: HP_OCCLUSION_QUERY HP_OCCLUSION_QUERY NV_OCCLUSION_QUERY NV_OCCLUSION_QUERY Go to NVIDIA occlusion presentation… Go to NVIDIA occlusion presentation… –An aside: applies to cell-portal culling!

D22 Visibility Culling: Discussion When is visibility culling worthwhile? When is visibility culling worthwhile? –When scene has high depth complexity Examples: architectural walkthroughs, complex CAD assemblies, dense forest Examples: architectural walkthroughs, complex CAD assemblies, dense forest Non-examples: terrain (usually), single highly- tessellated object (e.g., bunny, a radiositized room) Non-examples: terrain (usually), single highly- tessellated object (e.g., bunny, a radiositized room)

D23 Visibility Culling: Discussion How does visibility culling compare to: How does visibility culling compare to: –Level-of-detail: Reduces geometry processing Reduces geometry processing Helps transform-bound apps Helps transform-bound apps –Visibility culling: Reduces geometry and pixel processing Reduces geometry and pixel processing Helps transform- and fill rate-bound apps Helps transform- and fill rate-bound apps –Texture / Image representations: Reduces geometry and pixel processing Reduces geometry and pixel processing Incurs texture/image processing costs Incurs texture/image processing costs

D24 Visibility Culling: Discussion How does visibility culling interact with level of detail? How does visibility culling interact with level of detail? –Fairly seamless integration; generally a win –One issue: visibility of simplified model may differ from original model; requires some care –LODs can speed up occluder selection and rendering

D25 Visibility Culling: Discussion How does visibility culling interact with texture and image-based representations? How does visibility culling interact with texture and image-based representations? –Texture/image reps generally replace far-field geometry Involves an implicit occlusion culling step Involves an implicit occlusion culling step Reduces scene depth complexity, decreasing the utility of visibility culling Reduces scene depth complexity, decreasing the utility of visibility culling If near-field geometry still includes complex heavily- occlusive assemblies, still a win If near-field geometry still includes complex heavily- occlusive assemblies, still a win

D26 Visibility Culling: Discussion How much culling effort is appropriate? How much culling effort is appropriate? –Cells and portals: relatively cheap, with large potential speedups –Hierarchical occlusion maps: relatively costly, carefully weigh potential gains –Multiple processors allow much more aggressive culling calculation Pipelining culling calculations, Performer-style, allows cull time = render time Pipelining culling calculations, Performer-style, allows cull time = render time Tradeoff: one frame increased latency Tradeoff: one frame increased latency

D27 Summary The basic, very powerful idea:The basic, very powerful idea: –Rapidly compute a potentially visible set –Let hardware handle the rest For many scenes, visibility culling is a simple way to get huge speedupsFor many scenes, visibility culling is a simple way to get huge speedups –View-frustum culling always a must –For scenes with high depth complexity, occlusion culling can be a big win

D28 Summary Architectural models: visibility is practically a solved problemArchitectural models: visibility is practically a solved problem –Cells and portals work well Cull-box portal culling: simple, fast Cull-box portal culling: simple, fast Line-stabbing: elegant, powerful Line-stabbing: elegant, powerful

D29 Summary Occlusion culling of general models: still a largely open problemOcclusion culling of general models: still a largely open problem –Important issues: Dynamic scenes Dynamic scenes Aggregate occlusion effects Aggregate occlusion effects –Image-based approaches seem most promising at this time

D30 Summary General occlusion culling algorithms:General occlusion culling algorithms: –Hierarchical Z-buffer: A simple, truly elegant algorithm A simple, truly elegant algorithm But doesn’t seem amenable to hardware But doesn’t seem amenable to hardware –Hierarchical occlusion maps: Separates occluders from occludees, overlap from depth Separates occluders from occludees, overlap from depth Lends itself well to aggressive culling Lends itself well to aggressive culling Fairly high overhead, only worthwhile with high depth complexity Fairly high overhead, only worthwhile with high depth complexity –Recent hardware supports (asynchronous) occlusion query