© Machiraju/Möller Rendering Equation Various Solutions cis782 Advanced Computer Graphics Raghu Machiraju

© Machiraju/Möller Reading Chapter 16.5 of “Physically Based Rendering” by Pharr&Humphreys Chapter 19, 20 in “Principles of Digital Image Synthesis,” by A. Glassner “Realistic Image Synthesis Using Photon Mapping,” by H. W. Jensen

© Machiraju/Möller Biased vs. Unbiased Algorithm unbiased algorithms: –Noise very apparent OR we need tons of rays –Variance (error) decreases in a predictable way Biased approaches –Irradiance caching + photon mapping –Smooth results –Cheap (only little more computing time than Whitted ray-tracing)

© Machiraju/Möller Biased vs. Unbiased Algorithm If we find little noise in the image: –For unbiased methods can be fairly sure that image is correct –For biased methods don’t have this assurance Increasing the sampling rate might not get rid of existing artifacts

© Machiraju/Möller Irradiance Caching Introduced by Greg Ward 1988 Implemented in RADIANCE –Public-domain software Exploits smoothness of irradiance –Cache and interpolate irradiance estimates

© Machiraju/Möller Irradiance Caching Indirect lighting changes rather slowly Compute indirect light at sparse set of samples Interpolate neighboring values from this set of samples

© Machiraju/Möller Irradiance Caching A.How to come up with such a (sparse) set of samples? B.How to store these samples? C.How to interpolate from neighbors?

© Machiraju/Möller A) Set of Samples Find them adaptively If there are no good nearby samples then compute a new irradiance sample Store irradiance (radiance is directionally dependent; i.e. expensive to store): we have: Assuming Lambertian-type surfaces (f=const.)

© Machiraju/Möller A) Set of Samples More glossy surfaces: hemispherical-directional reflectance

© Machiraju/Möller A) Set of Samples Irradiance chaching Path Tracing Same Computation Time

© Machiraju/Möller A) Set of Samples Irradiance caching Irradiance samples used

© Machiraju/Möller A) Set of Samples High error for specular surface Specular BSDF => Whitted integrator Diffuse BSDF => irradiance caching –Interpolation from known points –A path tracing step for new points –Use cos-weighted distribution:

© Machiraju/Möller B) Storing Samples Octree data structure –Each node stores samples that influence this node (each point has a radius of influence!) Radius of influence –determined by harmonic mean –d i is the distance that the i th ray traveled before intersecting an object –Computed during path tracing

© Machiraju/Möller C) Interpolation from neighbors Simply weighted sum: With weights: Skip samples –Normals too different –Too far away –If in front N N’ p p’

© Machiraju/Möller Images from “Radiance”

© Machiraju/Möller Images from “Radiance”

© Machiraju/Möller Caching Techniques Irradiance caching: –Compute irradiance at selected points and interpolate. Photon mapping: –Trace “photons” from the lights and store them in a photon map, that can be used during rendering.

© Machiraju/Möller Photon Mapping Basic idea : Density estimation with a discrete density of photons 2-step algorithm –Photon trace: Simulates the transport of individual photons Photons emitted from source Photons deposited on surfaces Photons reflected from surfaces to other surfaces –Rendering Photons collected by rendering

© Machiraju/Möller First Pass - Photon Trace For 100 photons emitted from 100W source, each photon initially carries 1W. Propagate this radiant flux through scene using MC methods.

© Machiraju/Möller Estimating incident flux At any patch of surface, we can estimate the incident flux: Just average the contributions of all the photons that hit the patch.

© Machiraju/Möller A Photon For each surface interaction, we store: struct photon { float x,y,z;// position char power[4];// power (RGBE) char phi, theta;// incident direction short flag; }

© Machiraju/Möller Photon Storage Store this information about surface interactions in photon map (kd-tree) Photon storage is decoupled from geometry

© Machiraju/Möller Second Pass - Rendering Estimate flux incident at a surface point based on nearby photons.

© Machiraju/Möller Radiance Estimate Expand ball until it contains some reasonable number of photons. Use intersection with plane to estimate area of surface patch.

© Machiraju/Möller Radiance Estimate Recall the reflected radiance equation Convert incident radiance into incident flux Reflected radiance in terms of incident flux Numerically  A =  r 2

© Machiraju/Möller Caustic from a Glass Sphere Photon Mapping: photons / 50 photons in radiance estimate

© Machiraju/Möller Caustic from a Glass Sphere Path Tracing: 1000 paths/pixel

© Machiraju/Möller Sphereflake Caustic

© Machiraju/Möller Reflection Inside A Metal Ring photons / 50 photons in radiance estimate

© Machiraju/Möller Caustics On Glossy Surfaces photons / 100 photons in radiance estimate

© Machiraju/Möller HDR environment illumination Using lightprobe from

© Machiraju/Möller Cognac Glass

© Machiraju/Möller Cube Caustic

© Machiraju/Möller Global Illumination photons / 50 photons in radiance estimate

© Machiraju/Möller Global Illumination photons / 500 photons in radiance estimate

© Machiraju/Möller Fast estimate 200 photons / 50 photons in radiance estimate

© Machiraju/Möller Indirect illumination photons / 500 photons in radiance estimate

© Machiraju/Möller Global Illumination global photon map vs. caustics photon map

© Machiraju/Möller Photon tracing Photon emission Photon scattering Photon storing

© Machiraju/Möller Photon emission Given  Watt lightbulb. Emit N photons. Each photon has the power  /N Watt Photon power depends on the number of emitted photons. Not on the number of photons in the photon map.

© Machiraju/Möller What is a photon? Flux (power) - not radiance! Collection of physical photons – A fraction of the light source power – Several wavelengths combined into one entity

© Machiraju/Möller Diffuse point light Generate random direction Emit photon in that direction // Find random direction do { x = 2.0*random()-1.0; y = 2.0*random()-1.0; z = 2.0*random()-1.0; } while ( (x*x + y*y + z*z) > 1.0 );

© Machiraju/Möller Example: Diffuse square light Generate random position p on square Generate diffuse direction d Emit photon from p in direction d // Generate diffuse direction u = random(); v = 2*  *random(); d = vector(cos(v),sin(v), );

© Machiraju/Möller Other Sources Random directions and importance sampling (e.g., diffuse) within volumes of emission (bounding volumes) Projection Maps – Cells in scene which indicate if photons map or not

© Machiraju/Möller Surface interactions The photon is –Stored (at diffuse surfaces) and –Absorbed (A) or –Reflected (R) or –Transmitted (T) A + R + T = 1.0

© Machiraju/Möller Photon scattering The simple way: –Given incoming photon with power  p –Reflect photon with the power R*  p –Transmit photon with the power T*  p

© Machiraju/Möller Photon scattering The simple way: –Given incoming photon with power  p –Reflect photon with the power R*  p –Transmit photon with the power T*  p Risk: Too many low-powered photons - wasteful! When do we stop (systematic bias)? Photons with similar power is a good thing.

© Machiraju/Möller Russian Roulette Probability of termination: q Terminate un-important photons and still get the correct result

© Machiraju/Möller Russian Roulette Example Surface reflectance: R = 0.5 Incoming photon:  p = 2W r = random(); if ( r < 0.5 ) reflect photon with power 2 W else photon is absorbed

© Machiraju/Möller Russian Roulette Intuition Surface reflectance: R = incoming photons with power:  p = 2W Reflect 100 photons with power 2W instead of 200 photons with power 1W

© Machiraju/Möller Russian Roulette Very important! Use to eliminate un-important photons Gives photons with similar power

© Machiraju/Möller Storing Photons data structure for storing and searching For uniform distribution – cubes are best For non-uniform – kd-trees, Voronoi etc. k-d-trees, perhaps best

© Machiraju/Möller Storing Photons Use kd-tree – a sequence of axis-aligned partitions –2-D partitions are lines –3-D partitions are planes Axis of partitions alternates with depth of the tree Average access time - O(log n) Worst case O(n) when tree is skewed Need to maintain a balanced tree Balancing O(n log n) k nearest neighbors in O(k + log n)

© Machiraju/Möller Balancing kd-tree

© Machiraju/Möller Multiple Photon Maps Interpolation from the map causes blurriness For specular interactions Whitted Ray-Tracing Otherwise divide photon map into –direct light map L i,d –Indirect light map L i,i –Caustic map L i,c

© Machiraju/Möller Photon Mapping: Rendering Classification of photons in Photon Map

© Machiraju/Möller Multiple Photon Maps Direct light map L i,d Indirect light map L i,i Caustic map L i,c

© Machiraju/Möller + = Direct Illumination Indirect Illumination + Specular Part = Another example

© Machiraju/Möller + = Caustics Another example (cont’d)

© Machiraju/Möller Features Photon tracing is unbiased –Radiance estimate is biased but consistent –The reconstruction error is local Illumination representation is decoupled from the geometry

© Machiraju/Möller Box global photons, caustic photons

© Machiraju/Möller Box: Global Photons global photons

© Machiraju/Möller Fractal Box global photons, caustic photons

© Machiraju/Möller Cornell Box

© Machiraju/Möller Indirect Illumination

© Machiraju/Möller Little Matterhorn

© Machiraju/Möller Mies house (swimmingpool)

© Machiraju/Möller Mies house (3pm)

© Machiraju/Möller Mies house (6pm)

© Machiraju/Möller Final Gathering

© Machiraju/Möller Recap Decouple representation of illumination from geometry Photon Map – – Illumination as points in a global data structure –Cache of light paths. Estimate illumination based on density – Error is of low frequency – Not high frequency noise Unbiased method! Average expected value is not correct ! Consistent – technique will converge as more points/photons are added Photon Mapping, Photon Tracing, Photon Map

© Machiraju/Möller Monte-Carlo Ray Tracing Advantages –L(D|S)*E – All paths of light –Arbitrary geometry Parametric, subdivisions, implicit Procedural models –Low Memory Find intersections on demand –Arbitrary BRDFs Disadvantages –Noise –Complexity Jensen

© Machiraju/Möller L lightsource Pixel contribution rays reflected ray refracted ray LS*[D]E paths Photon Tracing

© Machiraju/Möller Photon Tracing –Photon generation stage Emit photons on light sources Random walk (trace photons through scene) Store interactions (position x, power phi, …) Photon Map Data Structure (e.g. kd-tree.)

© Machiraju/Möller pp  p xpxp Photon? A photon p is a particle of light that carries flux  p (x p,  p ) –Power:  p – magnitude (in Watts) and color of flux it carries, stored as an RGB triple –Position: x p – location of the photon –Direction:  p – the incident direction  i used to compute irradiance Photons vs. Rays –Photons propagate flux –Rays gather radiance

© Machiraju/Möller Scatter use light path tracing packets of flux (photons) distributed photons stored in a spatial data structure (e.g. kd-tree.)

© Machiraju/Möller Sources Point source –Photons emitted uniformly in all directions –Or emission is controlled by geometry Power of source (W) distributed evenly among photons Flux of each photon equal to source power divided by total # of photons For example, a 60W light bulb would send out a total of 100K photons, each carrying a flux  of 0.6 mW Photons sent out once per simulation, not continuously as in radiosity

© Machiraju/Möller Point Diffuse Sources emit_photons_from_diffuse_point_light() { n e =0 While (not enough photons){ do {// rejection sampling x = 2  1 -1; y =2  2 – 1; z = 2  3 – 1; } while (x^2+Y^2+z2 > 1) d= [x, y, z] T p= light_source_position trace photon from p in direction d n e = n e + 1 } scale power of stored photons with 1/n e }

© Machiraju/Möller Photon Scatter – Ray Tracing Redux Shoot(ray, pow, d) IF d > dmax THEN RETURN (object, x) = FirstIntersect(ray) IF no intersection THEN RETURN IF k r > 0 THEN Shoot(k r * pow, reflected ray, d+1) IF k t > 0 THEN Shoot(k t * pow, refracted ray, d+1) IF kd > 0 THEN Shoot(kd* pow, random-direction, d+1) END

© Machiraju/Möller ? Russian Roulette Reflected flux only a fraction of incident flux After several reflections, can spend lot of time tracking very little flux Also ensure that photons have approximately same power Absorb some photons and reflect the rest at full power Spend time tracing fewer full power photons Probability of reflectance is the reflectance  Probability of absorption is 1 –  Variance --- It does increase it

© Machiraju/Möller Mixed Surfaces Surfaces have specular and diffuse components –  d – diffuse reflectance –  s – specular reflectance –  d +  s < 1 (conservation of energy) Let  be a uniform random value from 0 to 1 If  <  d then reflect diffuse Else if  <  d +  s then reflect specular Otherwise absorb How about RGB ? Use averages across channels

© Machiraju/Möller Photon Map Catch the bug in the photon Map

© Machiraju/Möller Photon Struct photon{ float x, y, z; char power[4];//four bytes or 3 floats char ,  ;//incident direction short flag;// splitting plane axis } Theta = 256 * atan(dx,dy)/(2*pi) Phi = 256 * acos(dx)/pi

© Machiraju/Möller Nearest Neighbor Seacrh

© Machiraju/Möller Photon Mapping – Stage 2 Gathering walk to the end of “all” photon traces Le1Le1 Le2Le2 Le3Le3

© Machiraju/Möller Rendering Modified distribution ray tracing Gather: for each point to be shaded find k-nearest photons in the hood using K-d tree Query these k nearest photons for energies estimate local radiance using light flux of photons Approximate radiance by density estimation Density estimation: radiance = sumOfEnergies/coveredArea

© Machiraju/Möller Rendering

© Machiraju/Möller Anomalies Use Convex hull of photons Flatten disk

© Machiraju/Möller Need for Filtering

© Machiraju/Möller Filtering Too few photons cause blurry results Simple averaging produces a box filtering of photons Photons nearer to the sample should be weighted more heavily Results in a cone filtering of photons

© Machiraju/Möller Gaussian Filtering  = 0.918,  =1.953

© Machiraju/Möller Photon Numbers 10,000 P 100 Radiance 500,000 P 500 Radiance Compromise: Direct lighting with feelers 10,000 P, 100 Radiance

© Machiraju/Möller Photon Numbers 200 P 50 Radiance Really fast

© Machiraju/Möller Images caustics dispersion

© Machiraju/Möller Images

© Machiraju/Möller 4 Components

© Machiraju/Möller Components (1) Radiance thro a pixel Direct Lighting Shadow Photons

© Machiraju/Möller Caustics & Indirect Diffuse Components (2)

© Machiraju/Möller Schlick Caustic  =1.0  =0.1  =0.5  =0.01

© Machiraju/Möller Combine All Maps Outgoing radiance at x Reflected light at x BRDF at x Incoming radiance at x Direct illlumIndirect- Caustics Indirect- Diffuse

© Machiraju/Möller Combine All Maps Direct illlum No photon Map Caustics MC ray tracing

© Machiraju/Möller Issues (1) Memory/Time Costs –The number of photons and the size of spatial data structure, (and hence memory footprint) increase proportionally with quality requirements of image. –K-nearest neighbour search is non-trivial: time- complexity is highly data-structure dependent.

© Machiraju/Möller Issues – Generation (2) Poor spectral-space sampling –Restricted to RGB. Original algorithm: Pure forward simulation  Visual importance not taken into account  Photon density is high (only) where illumination is high  Problematic whenever photon density doesn’t match importance distribution –High density (high cost) in unimportant regions –Low density (low quality) in important region

© Machiraju/Möller Bad Importance Distribution importance distribution photon distribution