Visibility Sorting and Compositing without Splitting for Image Layer Decompositions John Snyder, Jed Lengyel

Layered Decomposition Problem Given set of moving geometric parts, Given set of moving geometric parts, and moving camera, and moving camera, find visibility ordering for every frame. find visibility ordering for every frame. Given set of moving geometric parts, Given set of moving geometric parts, and moving camera, and moving camera, find visibility ordering for every frame. find visibility ordering for every frame. parts sorted images composited result sort composite

Occlusion Cycles When no order exists, can split or group parts. split group

Non-Splitting Layered Decomposition Don’t split parts that form occlusion cycles: run-time splitting is slowrun-time splitting is slow splitting is often unnecessarysplitting is often unnecessary splitting destroys coherencesplitting destroys coherence Don’t split parts that form occlusion cycles: run-time splitting is slowrun-time splitting is slow splitting is often unnecessarysplitting is often unnecessary splitting destroys coherencesplitting destroys coherence Sorting requires no global separating plane.

Image-Based Rendering Acceleration [Lengyel97] sprites composited image

Other Applications fast special effects fast special effects animation playback with selective display animation playback with selective display incorporation of external image streams incorporation of external image streams image stream compression image stream compression fast hidden line rendering fast hidden line rendering z resolution targeting z resolution targeting fast special effects fast special effects animation playback with selective display animation playback with selective display incorporation of external image streams incorporation of external image streams image stream compression image stream compression fast hidden line rendering fast hidden line rendering z resolution targeting z resolution targeting

Previous Work invisibility culling [Greene93,Zhang97] invisibility culling [Greene93,Zhang97] analytic hidden surface removal [Mulmuley89] analytic hidden surface removal [Mulmuley89] global visibility [Teller93,Durand97] global visibility [Teller93,Durand97] image layer decompositions [Schumacker69,Newell72,Fuchs80] image layer decompositions [Schumacker69,Newell72,Fuchs80] dynamic visibility [Torres90,Sudarsky96] dynamic visibility [Torres90,Sudarsky96] depth sorting for special effects [Max85] depth sorting for special effects [Max85] invisibility culling [Greene93,Zhang97] invisibility culling [Greene93,Zhang97] analytic hidden surface removal [Mulmuley89] analytic hidden surface removal [Mulmuley89] global visibility [Teller93,Durand97] global visibility [Teller93,Durand97] image layer decompositions [Schumacker69,Newell72,Fuchs80] image layer decompositions [Schumacker69,Newell72,Fuchs80] dynamic visibility [Torres90,Sudarsky96] dynamic visibility [Torres90,Sudarsky96] depth sorting for special effects [Max85] depth sorting for special effects [Max85]

Occlusion Graphs pairwise occlusion relation: A  E Bpairwise occlusion relation: A  E B relations on parts forms directed graph: occlusion graphrelations on parts forms directed graph: occlusion graph pairwise occlusion relation: A  E Bpairwise occlusion relation: A  E B relations on parts forms directed graph: occlusion graphrelations on parts forms directed graph: occlusion graph A B view from E A A B B side view E E

Occlusion Graphs (Ex. 1) C B A AB C

Occlusion Graphs (Ex. 2) C B A AB C

Occlusion Graphs (Ex. 3) AB C Forms strongly connected component (SCC).

Resolving Non-Binary Cycles SCCs need not be rendered as an aggregate!

Occlusion Testing via Collision E E BA For convex hulls of objects A and B: B  E A  ch(A  E)  B   For convex hulls of objects A and B: B  E A  ch(A  E)  B  

Occlusion Testing via Collision: Occluding Example E E ch(A  E)  B    B  E A ch(A  E) B

Occlusion Testing via Collision: Non-Occluding Example E E ch(A  E) B // ch(A  E)  B    B  E A

Trivial Sorting Algorithm compute occlusion graph compute occlusion graph detect and topologically sort SCCs detect and topologically sort SCCs at least quadratic in number of parts! at least quadratic in number of parts! compute occlusion graph compute occlusion graph detect and topologically sort SCCs detect and topologically sort SCCs at least quadratic in number of parts! at least quadratic in number of parts! E E A B C D F

Improving the Trivial Algorithm E E A B C D F A  B,C,D,F but nothing occludes A. Process A first.

Improving the Trivial Algorithm E E B C D F Sorting Output: A A

Improving the Trivial Algorithm E E C D F Sorting Output: A, B B

Improving the Trivial Algorithm E E D F Sorting Output: A, B, C C

Improving the Trivial Algorithm E E Sorting Output: A, B, C, D, F D F

Incremental Visibility Sort (IVS) similar to Newell, Newell, & Sancha algorithm, 1972similar to Newell, Newell, & Sancha algorithm, 1972 sorts parts, not polygonssorts parts, not polygons detects SCCs, doesn’t splitdetects SCCs, doesn’t split uses order from last query, not depth orderuses order from last query, not depth order culls more efficiently using kd-treeculls more efficiently using kd-tree similar to Newell, Newell, & Sancha algorithm, 1972similar to Newell, Newell, & Sancha algorithm, 1972 sorts parts, not polygonssorts parts, not polygons detects SCCs, doesn’t splitdetects SCCs, doesn’t split uses order from last query, not depth orderuses order from last query, not depth order culls more efficiently using kd-treeculls more efficiently using kd-tree Depth order doesn’t indicate visibility order. E E A B Z Z

IVS Algorithm While L is nonempty, pop off top element A if A is unmarked if nothing remaining on L occludes A send A to output else mark and reinsert A into L else // A is marked check for occlusion cycle if found, group cycle objects and reinsert else reinsert A into L While L is nonempty, pop off top element A if A is unmarked if nothing remaining on L occludes A send A to output else mark and reinsert A into L else // A is marked check for occlusion cycle if found, group cycle objects and reinsert else reinsert A into L

Occlusion Culling Fundamental IVS query: Which parts remaining in L occlude A? use convex hulls around parts use convex hulls around parts bound hulls with spatial and angular extents bound hulls with spatial and angular extents occlusion cull reduces to 1D interval intersection occlusion cull reduces to 1D interval intersection bounded parts organized in dynamic kd-tree bounded parts organized in dynamic kd-tree Fundamental IVS query: Which parts remaining in L occlude A? use convex hulls around parts use convex hulls around parts bound hulls with spatial and angular extents bound hulls with spatial and angular extents occlusion cull reduces to 1D interval intersection occlusion cull reduces to 1D interval intersection bounded parts organized in dynamic kd-tree bounded parts organized in dynamic kd-tree object convex hull kd extent

Spatial Extents // B  A

Angular Extents // B  A

Kd-Tree Occlusion Culling rebalance kd-tree every frame rebalance kd-tree every frame kd-tree supports part deactivation kd-tree supports part deactivation rebalance kd-tree every frame rebalance kd-tree every frame kd-tree supports part deactivation kd-tree supports part deactivation B D C A x1x1x1x1 y1y1y1y1 y2y2y2y2 x1x1x1x1 y1y1y1y1 y2y2y2y2 A C B D y y x x

Exact Occlusion Testing Does B  A? Does B  A? uses hybrid Chung/Gilbert collision algorithm uses hybrid Chung/Gilbert collision algorithm requires extremal vertex query on A and B requires extremal vertex query on A and B no need to create ch(A  E) dynamically no need to create ch(A  E) dynamically exploits coherence in object motion exploits coherence in object motion Does B  A? Does B  A? uses hybrid Chung/Gilbert collision algorithm uses hybrid Chung/Gilbert collision algorithm requires extremal vertex query on A and B requires extremal vertex query on A and B no need to create ch(A  E) dynamically no need to create ch(A  E) dynamically exploits coherence in object motion exploits coherence in object motion D D extremal vertex minimizes Dv on A A v v

Results (Video)

IVS Complexity

Performance with Increasing Objects

Future Work animation previewer animation previewer continuous time queries continuous time queries adaptive splitting, especially for terrain adaptive splitting, especially for terrain faster, less conservative visibility tests faster, less conservative visibility tests automatic part decomposition automatic part decomposition visibility sorting with minimal splitting visibility sorting with minimal splitting animation previewer animation previewer continuous time queries continuous time queries adaptive splitting, especially for terrain adaptive splitting, especially for terrain faster, less conservative visibility tests faster, less conservative visibility tests automatic part decomposition automatic part decomposition visibility sorting with minimal splitting visibility sorting with minimal splitting

ConclusionsConclusions non-splitting layered decomposition useful non-splitting layered decomposition useful software visibility sorting is practical software visibility sorting is practical basic ideas of approach: basic ideas of approach: exploit temporal and spatial coherenceexploit temporal and spatial coherence exploit properties of convex objectsexploit properties of convex objects exploit incremental collision detection algorithmsexploit incremental collision detection algorithms non-splitting layered decomposition useful non-splitting layered decomposition useful software visibility sorting is practical software visibility sorting is practical basic ideas of approach: basic ideas of approach: exploit temporal and spatial coherenceexploit temporal and spatial coherence exploit properties of convex objectsexploit properties of convex objects exploit incremental collision detection algorithmsexploit incremental collision detection algorithms

Extent Tracking use “vertex descent” use “vertex descent” on convex polytope, local minimizer of extent is also global minimizer on convex polytope, local minimizer of extent is also global minimizer use “vertex descent” use “vertex descent” on convex polytope, local minimizer of extent is also global minimizer on convex polytope, local minimizer of extent is also global minimizer D D frame t 0 v0v0 v0v0 frame t 1 v1v1 v1v1 v0v0 v0v0

Results: Tumbling Toothpicks uniform scale (“us”): add more of same size uniform scale (“us”): add more of same size biases occlusion complexity superlinearlybiases occlusion complexity superlinearly uniform density (“ud”): add more of scaled size uniform density (“ud”): add more of scaled size occlusion complexity increases linearlyocclusion complexity increases linearly uniform scale (“us”): add more of same size uniform scale (“us”): add more of same size biases occlusion complexity superlinearlybiases occlusion complexity superlinearly uniform density (“ud”): add more of scaled size uniform density (“ud”): add more of scaled size occlusion complexity increases linearlyocclusion complexity increases linearly eccentric ellipsoids moving in cubical volume eccentric ellipsoids moving in cubical volume

Performance with Increasing Velocity

Performance with Increasing Objects Uniform Density & Uniform Scale

Kd-Tree Culling Performance

six aircraft flying through valley six aircraft flying through valley investigates rendering acceleration at various levels of terrain splitting investigates rendering acceleration at various levels of terrain splitting update rate assumptions: update rate assumptions: aircraft parts: 20%aircraft parts: 20% terrain parts: 70%terrain parts: 70% sky 40%sky 40% six aircraft flying through valley six aircraft flying through valley investigates rendering acceleration at various levels of terrain splitting investigates rendering acceleration at various levels of terrain splitting update rate assumptions: update rate assumptions: aircraft parts: 20%aircraft parts: 20% terrain parts: 70%terrain parts: 70% sky 40%sky 40% Results: Canyon Flyby aggregation penalty: render every frame aggregation penalty: render every frame

Results: Canyon Flyby cpu times are in ms per frame cpu times are in ms per frame update rate is poly-weighted update rate is poly-weighted cpu times are in ms per frame cpu times are in ms per frame update rate is poly-weighted update rate is poly-weighted

Newell, Newell, & Sancha (1972) traverse depth-sorted list of polygons traverse depth-sorted list of polygons if next polygon doesn’t overlap, send to output if next polygon doesn’t overlap, send to output otherwise use tests of increasing complexity: otherwise use tests of increasing complexity: screen bounding box, vertex/plane, 2d intersectionscreen bounding box, vertex/plane, 2d intersection if unoccluded, polygon is sent to output if unoccluded, polygon is sent to output if occluded, polygon is marked and reinserted if occluded, polygon is marked and reinserted marked polygons are split to remove cycle marked polygons are split to remove cycle traverse depth-sorted list of polygons traverse depth-sorted list of polygons if next polygon doesn’t overlap, send to output if next polygon doesn’t overlap, send to output otherwise use tests of increasing complexity: otherwise use tests of increasing complexity: screen bounding box, vertex/plane, 2d intersectionscreen bounding box, vertex/plane, 2d intersection if unoccluded, polygon is sent to output if unoccluded, polygon is sent to output if occluded, polygon is marked and reinserted if occluded, polygon is marked and reinserted marked polygons are split to remove cycle marked polygons are split to remove cycle E E A A Z Z C C B B F F H H G G