Fast Global-Illumination on Dynamic Height Fields Derek Nowrouzezahrai University of Toronto John Snyder Microsoft Research Derek from UofT Presenting my work  real-time GI for dynamic HF geometry Collaboration  John Snyder  MSR

Related Work shadow map filtering [Reeves87; …] light “bleeding” artifacts small light sources [Donnelly06]  WEALTH OF WORK IN SHADOW MAP FILTERING TO APPROXIMATE SOFT SHADOWS  AS WITH SHADOW MAPPING, ARBITRARILY COMPLEX GEOMETRY + REAL-TIME PERF.  SUFFER FROM LIGHT-BLEEDING + OVER BLURRING.  ARTIFACTS CAN BE MINIMIZED HEURISTICALLY. CON: ONLY HANDLE APPROX. SMALL LIGHT SOURCES ENV LIGHTING CAN BE APPROXIMATED WITH MANY LIGHT SOURCES

Related Work static geometry [Sloan02; Ng04; …] dynamic geometry [Bunnell05, Ren06, Sloan07, Ritschel08]  MOTIVATED BY GROWING LIT. IN PRT.  STATIC CASE: DECOUPLE + PRECOMPUTE DIRECTIONAL LIGHTING RESPONSE @ SHADING POINTS  CAN RELIGHT UNDER VARYING (TYPICALLY DISTANT) ILLUMINATION DYNAMIC CASE: REN ET AL. + SLOAN ET AL. APPROX. SCENE GEOM. AS SET OF SPHERES + ACCUMULATE OCCLUSION OVER SPHERES IN LOG SH SPACE RITSCHEL ET AL. USE MANY INCOHERENT SHADOW MAPS TO APPROX GI EFFECTS IN DYNAMIC SCENES  SCALES WITH THE NUMBER OF INDIRECT AND DIRECT LIGHTS.

Related Work screen-space shading [Shanmugam07;Ritschel09…] ignores view-occluded blockers [Dimitrov08] horizon mapping [Max88; …] precomputation for hard shadows on static geometry [Sloan&Cohen00]  RECENTLY: IMAGE-SPACE SHADING FOR STATIC + DYNAMIC GEOMETRY.  THESE TECHNIQUES USE: SCREEN-SPACE DEPTH, POSITION, NORMALS TO APPROX. THE A-&-D OCCLUSION VALUES  NAMELY: DIMITROV ET AL. & RITSCHEL ET AL.: SAMPLE MANY DEPTHS AROUND EACH PIXEL IN FRAME BUFFER TO APPROXIMATE AO AND DO.  NOT TEMPORALLY COHERENT: SINCE VIEW-OCCLUDED GEOMETRY DOES NOT CAST SHADOWS  CAN’T HANDLE DISTANT EFFECTS DUE TO LOCAL VISIBILITY SHAMPLING HORIZONG MAPPING: PRECOMPUTE MAX BLOCKING ANGLE @ MANY AZIMUTHAL DIRECTIONS. HARD SHADOWS FOR FIXED GEOMETRY

Related Work fast soft-shadowing on dynamic height fields [SN08] OUR PREVIOUS WORK: LAST YEAR @ EGSR PRESENTED MULTI-RESOLUTION SAMPLING SCHEME TABLE-BASED SH VISIBILITY SLICES

Goals all of [SN08] as well as dynamic indirect illumination glossy effects (direct and indirect) GOALS: ALL OF [SN08]: REAL-TIME SHADOWS FROM SMALL AREA + ENV ANIMATING LIGHTING DYNAMIC GEOMETRY [CLICK] NOW WITH GLOBAL ILLUMINATION GLOSSY

Goals HERE ILLUSTRATE THE DIFFERENCE B/W [SN08] & OUR RESULTS [CLICK x2] [SN08] Our results

Goals unified formulation for direct- and indirect-illumination diffuse and glossy bounces environmental + directional lighting dynamic geometry (not precomputed) real-time performance simple implementation limitation: geometry is a height field applications: terrain rendering (flight simulators, games, mapping/navigation) data visualization GOALS SUMMARY: UNIFIED FORMULATION DIRECTIONAL RESPONSE: CAST SHADOWS DIFFUSE + GLOSSY INDIRECT ENV + SMALL AREA SOURCES DYNAMIC GEOMETRY REAL-TIME SIMPLE CODE: 1 SHADER < 150 LINES [CLICK]  MAIN LIMITATION: ONLY HANDLES H.F. GEOMETRY POSSIBLE APPLICATIONS: TERRAIN RENDERING  FLIGHT SIM. + GAMES + MAPPING

Summary of Main Ideas approximate visibility & incident radiance w/ multi-resolution compute visibility and radiance at discrete azimuthal directions determine final spherical visibility and incident radiance create height and shading pyramids sample from pyramid levels pre-filter data  ROUGHLY DIVIDE INTO 3 MAIN IDEAS  FIRSTLY: MULTI-RES SAMPLING APPROACH FOR CALC. HORIZON MAP AND INCIDENT RADIANCE SECONDLY: ANALYTIC REPRESENTATION FOR VIS + INCIDENT RADIANCE PER AZIM DIR THIRDLY: CONVERTING FROM SAMPLED ANALYTIC REPRESENTATION TO SH  START WITH OUR MULTI-RES VIS & IR SAMPLING

Azimuthal Swaths [SN08] for smaller area (key) light sources: restrict azimuthal extent and use m = 3 get sharper shadows acts as a geometric mask only sample where necessary env lights and incident radiance: complete swath and use m = 32 AS IN [SN08] SAMPLING @ DISCRETE AZIM DIRECTIONS RECONSTRUCT CONTINUOUS SPHERICAL Fn’s B/W THESE SAMPLES [CLICK] FOR SMALLER KEY LIGHTS: RESTRICT AZIM. EXTENT GEOM MASK ENV + INDIRECT LIGHT COMES FROM ALL AZIM. DIRECTIONS

Definitions and Notation blocking angle: (t) angle p makes at t along  incident radiance: u(t) incident radiance at t towards p along  u(t) (t)  BASIC DEFINITIONS: H.F. = SCALAR FUNCTION OF 2 VARS, X + Y, DENOTE Z = F(X,Y)  REPRESENT H.F. AS TABULATED UNIFORM GRID IN X + Y 3D SURFACE POINT P BLOCKING ANGLE, \OMEGA(t) = ANGLE HORIZON MAKES ALONG AZIMUTHAL DIRECTION \PHI INCIDENT RADIANCE U(t) ALONG AZIMUTHAL DIRECTION \PHI REQUIRE: SAMPLE \OMEGA AND U ALONG \PHI TO DETERMINE VIS + IR Sample  and u at all points t along direction .

Calculating the Max Blocking Angle  NEED MAX BLOCKING ANGLE FOR VIS  TRADITIONAL WAY OF DETERMINING: EXHAUSTIVELY SAMPLE H.F. ALONG \PHI, CALC. ANGLE MADE B/W P & HEIGHT @ SAMPLE POINT  LIKE “WALKING” ALONG \PHI FOR ALL T  AFTER EXHAUSTIVELY SAMPLING: KEEP MAX HEIGHT DIFFERENCE == MAX ANGLE

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle [CLICK…]

Calculating the Max Blocking Angle HAVE THE MAX ALONG \PHI

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? FOR IR: NEED TO DETERMINE POINTS WHICH CONTRIBUTE INDIRECT LIGHT @ P

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK…]

Calculating the Incident Radiance  Which points on the height field contribute indirect radiance? [CLICK] ONLY POINTS W/ MONOTONICALLY INCREASING BLOCKING ANGLES CONTRIBUTE INDIRECT LIGHT CALLED CASTING SET T The set of points with monotonically increasing blocking angles. We call this the casting set.

Brute Force Sampling – Pitfall… PROBLEM WITH BRUTE FORCE SAMPLING: [CLICK] UNEVEN OR INSUFFICIENT SAMPLING LEADS TO ALIASING “W/O KNOWING ANYTHING SPECIFIC ABOUT THE HF, WE BASICALLY NEED A CONSTANT # OF SAMPLES PER UNIT DISTANCE” SOLVE BY: PRE-FILTERING DATA USE MULTI-SCALE SAMPLING Problem: aliasing – need many samples in t. Solution: prefilter data, apply multi-scale sampling.

Multi-Resolution Height Sampling height pyramid level i sampling distance for level i Sample coarser levels further from x. fi fi-1 fi-2 fi-3 τ i i-1 i-2 i-3 DEFINE A MULTI-RES BLOCKING ANGLE, \OMEGA_i  F_i == SUCCESSIVELY PRE-FILTERED VERSIONS OF ORIGINAL H.F. STORED IN PYRAMID == A STACK AFTER UPSAMPLING \TAU_i == MULTI-RES SAMPLING DISTANCE [CLICK]  AS WE MOVE ALONG LARGER DISTANCES IN THE DIRECTION \PHI, WE SAMPLE FROM COARSER LEVEL IN PYRAMID

Multi-Resolution Radiance Sampling multi-scale incident radiance samples radiance pyramid (for the previous bounce) blocking angle at Sample coarser levels further from x. ui ui-1 ui-2 ui-3 τ i i-1 i-2 i-3  U_i == SUCCESSIVELY PRE-FILTERED VERSIONS OF PREVIOUS BOUNCE’S SHADING STORED IN PYRAMID == A STACK AFTER UPSAMPLING [CLICK] AS WE MOVE ALONG LARGER DISTANCES IN THE DIRECTION \PHI, WE SAMPLE FROM COARSER LEVEL IN PYRAMID

Summary of Main Ideas approximate visibility & incident radiance w/ multi-resolution compute visibility and radiance at discrete azimuthal directions determine final spherical visibility and incident radiance analytic visibility and incident radiance use normalized Legendre polynomials (NLPs) NEXT: ANALYTIC REPRESENTATION FOR VIS + INCIDENT RADIANCE PER AZIM DIR

Analytic Occlusion Elevation Function we start with the binary occlusion function: σ 1 and represent it analytically in the Normalized Legendre Polynomial (NLP) basis: START W/: BINARY OCCLUSION Fn TRANSITION @ \OMEGA = \SIGMA AND REPRESENT IT IN THE N.L.P. BASIS W/ THE FOLLOWING ANALYTIC FORM [CLICK] MOTIVATION BEHIND N.L.P.: n-TH ORDER N.L.P. CAPTURES ALL ELEV. VARIATION OF n-TH ORDER SH

Analytic Visibility and IR can represent visibility and incident radiance in NLP with 1 max - = v () max 1 visibility  binary function with 1 transition from 0 to 1 @ max as  increases u () incident radiance  piece-wise constant, RGB function of elevation + = NOW THE N.L.P. PROJECTION OF VIS & IR CAN BE EASILY COMPUTED W/ OCCLUSION Fn’s [CLICK] [EXPLAIN]

Summary of Main Ideas approximate visibility & incident radiance w/ multi-resolution compute visibility and radiance at discrete azimuthal directions determine final spherical visibility and incident radiance NLPSH blending & projection fast shading pipeline LASTLY: CONVERTING FROM SAMPLED ANALYTIC REPRESENTATION TO SH

From Sampled NLP to Full SH given (2 x m) NLP vectors need full spherical functions (represented in SH) interpolate between azimuthal samples + project resulting spherical function into SH requires only 1 pre-computed matrix! matrix acts on NLP coefficients at edges of each swath rotate & sum across swaths for final SH GIVEN: N.L.P. COEFFS FOR VIS + IR @ EACH AZIM DIRECTION WANT: SH COEFFS OF FULL SPHERICAL FUNCTIONS [CLICK] INTERPOLATE B/W SAMPLES + PROJECT INTO SH 1 PRE-COMPUTED LINEAR OPERATOR MATRIX ACTS ON N.L.P. COEFFS AT EDGES OF A SWATH VERY EFFICIENT AND SIMPLE TO IMPLEMENT: SINGLE SHADER SEE SUPP MAT All operations performed in a single GPGPU shader. See supplemental material for full source code.

Global Illumination Shading with SH at each shading point: compute m azimuthal visibility + incident radiance NLP vectors interpolate & project into SH. Rotate & sum across directions BRDF: clamped cosine and/or Phong lobe ) ( x f N R or OUTLINE SHADING REQUIREMENTS: [CLICK] SAMPLE BLOCKING ANGLES ALONG EACH AZIM DIR. COMPUTE N.L.P. VIS + IR COEFFS [CLICK] APPLY INTERP/PROJECT OPERATOR YIELDING SH COEFFS OF VIS + IR [CLICK] COMPUTE VIEW-EVALUATED BRDF SH COEFFS [CLICK] QUERY EXTERNAL LIGHTING external lighting environment

Global Illumination Shading with SH Direct Illumination: BRDF x Visibility SH Product and take inner product with lighting Indirect Illumination: BRDF take inner product with Incident Radiance SHADING ALGORITHM: FOR DIRECT ILLUM: SH PRODUCT OF VIS & BRDF DOT WITH LIGHTING FOR INDIRECT ILLUM: DOT BRDF WITH IR 42

Comparison to Ground Truth ALGORITHM CONVERGENCE: 2 PARAMETERS AFFECT QUALITY { MULTI-RES STEPS, # AZIM DIRs. } ONLY SHOWING INDIRECT AS K INCREASES, WE CONVERGE  MORE FILTERING & MULTI-RES STEPS [CLICK] AS # OF AZIM. DIRECTIONS INCREASE, WE CONVERGE [CLICK] [EXPLAIN]

Memory Usage we typically sub-sample visibility & IR shade with full-resolution geometry & normals HF resolution pyramid levels memory requirements no sub-sampling 2x sub-sampling 256 x 256 (130k triangles) 33 17.4 MB 4.5 MB 512 x 512 (522k triangles) 37 76 MB 20 MB 1024 x 1024 (2.1M triangles) 41 336 MB 88 MB 2048 x 2048 (8.4M triangles) 45 1.4 GB 360 MB SUMMARIZE MEMORY REQUIREMENTS: #’S INCLUDE ALL INTERMEDIATE TEXTURES USED FOR STORAGE ON THE GPU TYPICALLY SUB-SAMPLE VIS + IR COMPUTATION & SHADE @ FULL-RES NORMALS/GEOMETRY W/O SUB-SAMPLING: MODEST UP TO 1024^2 (2.1 MILLION TRIANGLES) W/ SUB-SAMPLING: CAN HANDLE WELL OVER 8 MILLION TRIANGLES

Measured Performance HF resolution FPS (nVidia GTX 285) no ss 2x ss (130k triangles) 50 125 229 512 x 512 (522k triangles) 12 36 73 1024 x 1024 (2.1M triangles) 3 8 19 [EXPLAIN]

Image Results [ITERATE]

Results [ITERATE]

Results [ITERATE]

Conclusions multi-resolution sampling of: visibility incident radiance compact, analytic representation of: elevation-only functions SH interpolation and projection operators simple GPU implementation real-time up to 512x512 dynamic HFs can sub-sample visibility and incident radiance performance independent of geometric content CONCLUSION: MULTI-RES VIS + IR SAMPLING EFFICIENT + ANALYTIC REPRESENTATION OF ELEV-ONLY Fn’s N.L.P.-to-SH INTERPOLATION OPERATORS SIMPLE IMPLEMENTATION REAL-TIME IMPLEMENTATION FOR HIGH-RES HFs PERF. INDEPENDENT OF GEOM. CONTENT

Future Work combine with dynamic shadow casters via [Ren06;Sloan07] (sphere set blocker approximation) apply to image-space global illumination frameworks generalize geometry local height field displacements tiled height field representations FUTURE WORK: COMBINE WITH DYNAMIC SHADOW CASTERS: E.G. CHARACTER WALKING ON MOUNTAIN. IMAGE-SPACE GI FOR FAR-FIELD SHADOWING AND REFLECTION EFFECTS GENERALIZED GEOMETRY

We acknowledge the helpful suggestions of the anonymous reviewers. Thanks! Any questions? Thank you for attending my talk. I’d be more than happy to answer any of your question.