Realtime Caustics using Distributed Photon Mapping Johannes Günther Ingo Wald * Philipp Slusallek Computer Graphics Group Saarland University ( * now at MPII Saarbrücken)

June 21, 2004EGSR 2004, Norrköping, Sweden2 Introduction What is wrong with this picture? What is wrong with this picture?

June 21, 2004EGSR 2004, Norrköping, Sweden3 Introduction With caustic illumination With caustic illumination

June 21, 2004EGSR 2004, Norrköping, Sweden4 Outline Introduction & Motivation Introduction & Motivation Previous Work Previous Work Analysis of Photon Mapping Analysis of Photon Mapping Faster Photon Mapping Faster Photon Mapping Parallelization Issues Parallelization Issues Results & Examples Results & Examples

June 21, 2004EGSR 2004, Norrköping, Sweden5 Previous Work Caustics (Bidirectional) path tracing [Lafortune, Veach] (Bidirectional) path tracing [Lafortune, Veach] –Need many samples  too slow / too noisy Density estimation [Shirley, Walter] Density estimation [Shirley, Walter] –Use of pre-computed, view-independent illumination representation  Not interactive Particles in hierarchical radiosity system [Granier] Particles in hierarchical radiosity system [Granier] –Particle tracing accelerated by exploiting HR structure –Still too slow for interactive use Hardware based projection [Wand] Hardware based projection [Wand] –Implemented via texture lookups on GFX –Limited to far lights and no occluders considered –Only reflective caustic generators

June 21, 2004EGSR 2004, Norrköping, Sweden6 Previous Work Photon Mapping Photon map [Jensen] Photon map [Jensen] –Preferred tool for caustics –Independent of scene geometry –But problematic for interactive use (later) Hashed photon mapping [Wald] Hashed photon mapping [Wald] –Approximation  lower quality On programmable graphics hardware [Purcell] On programmable graphics hardware [Purcell] –Also approximative –Limited performance (interactive only at low resolution) –No caustics during interaction –Simple scenes

June 21, 2004EGSR 2004, Norrköping, Sweden7 Outline Introduction & Motivation Introduction & Motivation Previous Work Previous Work Analysis of Photon Mapping Analysis of Photon Mapping Faster Photon Mapping Faster Photon Mapping Parallelization Issues Parallelization Issues Results & Examples Results & Examples

June 21, 2004EGSR 2004, Norrköping, Sweden8 Analysis – Overview Photon Map: Preprocessing Preprocessing –Photon generation –Construction of kd-tree Queries during rendering Queries during rendering –Search k-nearest photons  Density estimation

June 21, 2004EGSR 2004, Norrköping, Sweden9 Analysis Interactivity: now ‘pre’processing every frame Interactivity: now ‘pre’processing every frame High quality caustic: > 100K photons High quality caustic: > 100K photons No problem with realtime ray tracing (>4Mrays/s) ? No problem with realtime ray tracing (>4Mrays/s) ? Low caustic photon yield Low caustic photon yield –Several rays per photon –Only few photons contribute to caustics (in many cases below 10%) Slow photon shooting Slow photon shooting –Incoherent rays  slow ray tracing, no SSE –Costly light source and BRDF sampling (up to 60% of total shooting time)  4-15 times slower

June 21, 2004EGSR 2004, Norrköping, Sweden10 Analysis Building kd-tree – –Serious problem with many photons complexity O(n log n) – –Hard to parallelize – –Small changes in map  complete rebuild Queries are costly – –Traversal of kd-tree – –Density estimation  About 10 times as expensive as tracing a ray  Have to address all of these problems

June 21, 2004EGSR 2004, Norrköping, Sweden11 Outline Introduction & Motivation Introduction & Motivation Previous Work Previous Work Analysis of Photon Mapping Analysis of Photon Mapping Faster Photon Mapping Faster Photon Mapping Parallelization Issues Parallelization Issues Results & Examples Results & Examples

June 21, 2004EGSR 2004, Norrköping, Sweden12 Selective Photon Tracing [Dmitriev] Adaptive sampling in path space Adaptive sampling in path space Exploiting similarity property of Halton numbers Exploiting similarity property of Halton numbers Indirect caustics captured Indirect caustics captured Improve yield by factor 4 Improve yield by factor 4

June 21, 2004EGSR 2004, Norrköping, Sweden13 Improve kd-tree Balancing kd-tree requires costly median finding Balancing kd-tree requires costly median finding Instead: “split in middle” Instead: “split in middle” –Spatial center of current voxel –Now unbalanced  Construction times faster

June 21, 2004EGSR 2004, Norrköping, Sweden14 Improve kd-tree Balancing kd-tree requires costly median finding Balancing kd-tree requires costly median finding Instead: “split in middle” Instead: “split in middle” –Spatial center of current voxel –Now unbalanced  Construction times faster Not intuitive: also traversal faster [Wald ’04] Not intuitive: also traversal faster [Wald ’04] –Better adaptation to photon density  Query times faster

June 21, 2004EGSR 2004, Norrköping, Sweden15 Accumulation of Photons Performance still not enough for high quality Performance still not enough for high quality  Improve quality by accumulation across frames kd-tree with “split in middle” allows augmentation kd-tree with “split in middle” allows augmentation Accumulated result is view-independent! Accumulated result is view-independent!  High quality walkthrough

June 21, 2004EGSR 2004, Norrköping, Sweden16 Outline Introduction & Motivation Introduction & Motivation Previous Work Previous Work Analysis of Photon Mapping Analysis of Photon Mapping Faster Photon Mapping Faster Photon Mapping Parallelization Issues Parallelization Issues Results & Examples Results & Examples

June 21, 2004EGSR 2004, Norrköping, Sweden17 Interleaved Sampling & Filtering [Keller] 5x5 interleaving 27k photons filtered Quality comparable to 25 x 27k = 680k photons Quality comparable to 25 x 27k = 680k photons

June 21, 2004EGSR 2004, Norrköping, Sweden18 Filtering Caustic Illumination Originally: restrict filtering to smooth regions Originally: restrict filtering to smooth regions –Continuity tests on server limits to 5fps (IGI [Wald ’02]) But: Only filter caustic illumination Only filter caustic illumination –Potential artifacts less visible  No need for costly discontinuity checks New: Simple box filter  efficient implementation Simple box filter  efficient implementation –SSE, cache friendly, independent of filter size Performance: CPU 300 fps, network now ~22 fps Performance: CPU 300 fps, network now ~22 fps

June 21, 2004EGSR 2004, Norrköping, Sweden19 Outline Introduction & Motivation Introduction & Motivation Previous Work Previous Work Analysis of Photon Mapping Analysis of Photon Mapping Faster Photon Mapping Faster Photon Mapping Parallelization Issues Parallelization Issues Results & Examples Results & Examples

June 21, 2004EGSR 2004, Norrköping, Sweden20 Performance Scalability: roughly linear Scalability: roughly linear Upper limit: about 22 fps (network bounded) Upper limit: about 22 fps (network bounded)

June 21, 2004EGSR 2004, Norrköping, Sweden21 Comparison to IGI [Wald ’02] Same #clients (9) and frame rate (4.8 fps) Same #clients (9) and frame rate (4.8 fps) 2600 phs 2500 phs phs 8000 phs IGI system new system

June 21, 2004EGSR 2004, Norrköping, Sweden22 Examples Indirect caustics Indirect caustics No performance drop No performance drop

June 21, 2004EGSR 2004, Norrköping, Sweden23 Examples Headlight simulation Headlight simulation Complex model Complex model Many layers of glass Many layers of glass Pointlights inside bulbs Pointlights inside bulbs Using 18 Dual clients Using 18 Dual clients 250k phs / 3 fps 25M phs / 11 fps

June 21, 2004EGSR 2004, Norrköping, Sweden24 Examples Headlight simulation Headlight simulation Complex model Complex model Many layers of glass Many layers of glass Pointlights inside bulbs Pointlights inside bulbs Using 18 Dual clients Using 18 Dual clients 250k phs / 3 fps 25M phs / 11 fps photo

June 21, 2004EGSR 2004, Norrköping, Sweden25 Video

June 21, 2004EGSR 2004, Norrköping, Sweden26 Conclusion Improvements on all aspects of the photon map Improvements on all aspects of the photon map –Improved yield by Selective Photon Tracing –Optimized kd-tree (construction and traversal) Efficient parallelization Efficient parallelization –Interleaved Sampling –Faster filtering  Realtime caustic simulation  High quality walkthrough  Linear scalability, up to 22 fps

June 21, 2004EGSR 2004, Norrköping, Sweden27 Questions ?

June 21, 2004EGSR 2004, Norrköping, Sweden28

June 21, 2004EGSR 2004, Norrköping, Sweden29 No discontinuity buffer? 5x5 filter, blurring over edges hardly visible 5x5 filter, blurring over edges hardly visible

June 21, 2004EGSR 2004, Norrköping, Sweden30

June 21, 2004EGSR 2004, Norrköping, Sweden31 Photon Map Two pass approach: Preprocessing Preprocessing –Generation Emit photons into sceneEmit photons into scene Trace path until absorptionTrace path until absorption –Construction of kd-tree For fast retrieval of hit pointsFor fast retrieval of hit points During rendering (ray tracing) During rendering (ray tracing) –At each point to shade: do query Search k-nearest photonsSearch k-nearest photons Accumulate power & divide by occupied areaAccumulate power & divide by occupied area  Density estimation Concentrate on caustics

June 21, 2004EGSR 2004, Norrköping, Sweden32 Client-Server Approach High computational cost  need parallelization High computational cost  need parallelization –even more necessary with photon mapping Commodity PCs Commodity PCs –24 Dual-AthlonMP For ray tracingFor ray tracing “Thin” clients: 100 Mbit Ethernet, no graphics card,...“Thin” clients: 100 Mbit Ethernet, no graphics card,... –1 server node Runs application, controls clients, etc.Runs application, controls clients, etc. Fully switched 100MBit network Fully switched 100MBit network –Gigabit uplink to server

June 21, 2004EGSR 2004, Norrköping, Sweden33 Outlook Combination with Instant Radiosity / IGI2 to add indirect illumination Combination with Instant Radiosity / IGI2 to add indirect illumination Further improve sampling and caustic photon yield Further improve sampling and caustic photon yield Still need a cluster Still need a cluster  Special Hardware (e.g. SaarCOR)

June 21, 2004EGSR 2004, Norrköping, Sweden34 Goal Generate and visualize caustic light effects Generate and visualize caustic light effects Interactive frame rates Interactive frame rates –At video resolution 640x480 Support complex scenes Support complex scenes Preview quality acceptable during scene modifications Preview quality acceptable during scene modifications High quality walkthroughs High quality walkthroughs

June 21, 2004EGSR 2004, Norrköping, Sweden35 Improve Yield of Caustic Photons Cannot help on #rays per photon path Cannot help on #rays per photon path But: improve efficiency of sampling But: improve efficiency of sampling Jensen: Jensen: –Project caustic generators onto (hemisphere of) light sources, sample only occupied solid angle –Captures only direct caustics –Mesh dependent, not in realtime Wald: Wald: –Project only bounding volume –Geometry independent –Bounding volume often too coarse approximation –Requires manual user interaction / scene preparation

June 21, 2004EGSR 2004, Norrköping, Sweden36 Distribution Options Generate same photon map on each client Generate same photon map on each client –Inefficient, only rendering/queries in parallel –Not scalable Generate different parts of photon map on clients Generate different parts of photon map on clients –Need exchange of photons –But: network bandwidth too small –Rendering can only start after receiving all photons  latency Generate different photon maps on clients Generate different photon maps on clients –Combine illumination on server by interleaving pixels –Reduce noise by filtering in image space  Interleaved sampling & discontinuity buffering [Keller]