Interactive Ray Tracing: From bad joke to old news David Luebke University of Virginia

Besides Parallelization l Besides parallelizing the algorithm, what else can we do to accelerate ray tracing? –Amortize the cost of shooting rays –Use ray tracing selectively

Amortize the cost of rays l The Render Cache –Work by Bruce Walters, currently at Cornell; also by Reinhard et al (Utah) –Basic idea: n Cache ray “hits” as shaded 3D points n Reproject points for new viewpoint n Now many pixels already have color! n Shoot rays for newly uncovered pixels n Shoot rays to update stale pixels –Show demo(?) –Web page w/ good examples, source:

Amortize the cost: Tole et al. l Tole et al. extend these ideas to path tracing –Cache ray hits in object space as Gouraud- shaded vertices –Designed for very slow sampling schemes (full bidirectional path tracing) n Pick pixels to sample carefully n Use OpenGL hardware to display current solution as it is gradually updated –Show the Tole video

Amortize The Cost: Frameless Rendering l Eliminate frames altogether –If you can render 1/3 of the pixels in a vertical retrace period: n Double buffering displays a new frame after 3 vertical refreshes n Single buffering causes horizontal tearing artifacts n Frameless rendering updates pixels as soon as they are computed… …but computes them in a randomized order to avoid coherent tearing artifacts –Show the Utah video

Shoot Rays Selectively l Use ray tracing selectively to augment a traditional interactive pipeline –Ex: use rays for shadows only –Ex: Use ray tracing to calculate corrective textures where necessary (e.g., shiny objects)

Summary So Far l Interactive ray tracing is a reality –Parker et al (SGI supercomputer) –Wald et al (Cluster of PCs) l Why IRT? –Complex/realistic shading –Big data –Decoupled sampling

Summary So Far l How IRT? –Ray tracing is embarrassingly parallel n Field of VAX/Cray joke n But memory coherence is a problem –Brute force: shared-memory supercomputer –Slightly smarter: distributed cluster n Fan-in, latency, model sharing are issues –Amortize cost n Cache/reuse samples, frameless rendering –Use selectively n Shadows only, corrective textures

Moving to Hardware l Next topic: moving ray tracing to the GPU –Why do this? l Two papers: –Ray Engine (Carr et al., U. Illinois) –Ray Tracing On Programmable Graphics Hardware (Purcell et al., Stanford) n I stole most of the following slides from this talk

Related Work: The Ray Engine l Nathan Carr, Jesse Hall, John Hart (University of Illinois) l Basic idea: use the fragment hardware! –Ray intersection is a crossbar: n Intersect a bunch of rays with a bunch of triangles, keep closest hit on each ray –Triangle rasterization is a crossbar: n Intersect a bunch of pixels with a bunch of triangles, keep closest hit at each pixel

(a) Naive: intersect all rays w/ all polys (b) Acceleration structures break crossbar grid up into a sparse block structure, but blocks are still dense crossbars (c) Result: a series of points on the crossbar, max 1 per ray (closest wins)

(a) Each pixel potentially intersected with each poly (b) Modern hardware

Ray Engine l Map ray casting crossbar to rasterization crossbar –Distribute rays across pixels n Ray-orgins texture n Ray-directions texture –Broadcast a stream of triangles as the vertex data interpolated across screen-filling quads n Quad color  Triangle id n Quad multi-texture coords: n Triangle vertices a,b, normal n, edges ab, ac, bc –Output: n Color = triangle id, alpha = intersect, z = t value

Ray Engine l Bulk of ray tracing computation is intersection l CPU handles bounding volume traversal, recursion, etc l GPU does ray-intersection on bundles of rays and triangles handed to it by CPU –NV_FENCE to keep both humming l Sometimes the CPU should intersect rays!

Why Ray Tracing? l Global illumination l Good shadows! –Doom 3 will be using shadow volumes n Expensive! –Shadow maps are hard to use and prone to artifacts l Efficient ray tracing based shadows could be the next killer feature for GPUs Doom 3 [id Software]

Why Ray Tracing? l Output-sensitive algorithm –Sublinear in depth complexity l Selective sampling –Frameless rendering [Bishop et al. 1994] –Render Cache [Walter et al. 1995] –Shading Cache [Tole et al. 2002] l Interactive on clusters of PCs [Wald et al. 2001] and supercomputers [Parker et al ] Power Plant [Wald et al. 2001]

Beyond Moore’s Law Yearly growth well above Moore’s Law (1.5) SeasonProductMT/sYr rateMF/sYr rate 2H97Riva H98Riva ZX H98Riva TNT H99Riva TNT H99GeForce H00GeForce2 GTS H00GeForce2 Ultra H01GeForce H02GeForce Courtesy of Kurt Akeley NVIDIA Historicals

Graphics Pipeline Application Geometry Rasterization Texture Fragment Display Command Textures Fragment Program Registers Fragment Input Fragment Output Traditional PipelineProgrammable Fragment Pipeline

Contributions l Map complete ray tracer onto GPU –Ray tracing generally thought to be incompatible with the traditional graphics pipeline l Abstract programmable fragment processor as a stream processor l Map ray tracing to streaming computation l Show that streaming GPU-based ray tracer is competitive with CPU-based ray tracer

Assumptions l Static scenes l Triangle primitives only l Uniform grid acceleration structure

Stream Programming Model Programmable fragment processor is essentially a stream processor l Kernels and streams –Stream is a set of data records –Kernels operate on records –Streams connect kernels together –Kernels can read global memory kernelinputrecordstreamoutputrecordstream kernel globals globals

Streaming Ray Tracer (Simplified) Generate Eye Rays Traverse Acceleration Structure Intersect Triangles Shade Hits and Generate Shading Rays Camera Grid Triangles Materials rays ray-voxel pairs hits pixels

Eye Ray Generator CameraScreen Generate Eye Rays rays Camera Scene

Traverser CameraScreen Traverse Acceleration Structure Grid rays ray-voxel pairs Scene

Intersector CameraScreenScene Intersect Triangles Triangles hits ray-voxel pairs

Intersection Code float4 Intersect( float3 ro, float3 rd, int listpos, float4 h ) { float tri_id = texture( listpos, trilist ); float3 v0 = texture( tri_id, v0 ); float3 v1 = texture( tri_id, v1 ); float3 v2 = texture( tri_id, v2 ); float3 edge1 = v1 – v0; float3 edge2 = v2 – v0; float3 pvec = Cross( rd, edge2 ); float det = Dot( edge1, pvec ); float inv_det = 1/det; float3 tvec = ro – v0; float u = Dot( tvec, pvec ) * inv_det; float3 qvec = Cross( tvec, edge1 ); float v = Dot( rd, qvec ) * inv_det; float t = Dot( edge2, qvec ) * inv_det; // determine if valid hit by checking // u,v > 0 and u+v < 1 // set hit data into h based on valid hit return float4( {t,u,v,id} ); } Intersect Triangles Triangles hits ray-voxel pairs

Ray Tracing on a GPU l Store scene data in texture memory –Dependent texturing is key l Multipass rendering for flow control –Branching would eliminate this need

Scene in Texture Memory xyz … … … xyz … … Uniform Grid 3D Luminance Texture Triangle List 1D Luminance Texture Triangles 3x 1D RGB Textures vox0 vox1vox2vox3vox4vox5voxM vox0vox2 tri0 tri1tri2tri3tri4tri5triN v0 v1 v2

Texture As Memory l Currently limited in size - 128MB –About 3M 36 bytes per triangle l Uniform grid –Maps naturally to 3D textures –Requires 4 levels of dependent texture lookups l 1D textures limited in length –Emulate larger address space with 2D textures l Want integer addressing – not floating point –Efficient access without interpolation l Integer arithmetic

Streaming Flow Control Fragments (Input Stream) Fragment Program (Kernel) Fragment Program Output (Output Stream) Rasterization Texture (Globals) Application and Geometry Stages

Multiple Rendering Passes Pass 1 Generate Eye Rays Draw quad Rasterize

Multiple Rendering Passes Pass 1 Generate Eye Rays Run fragment program

Multiple Rendering Passes Pass 1 Generate Eye Rays Save to offscreen buffer (rays)

Multiple Rendering Passes Pass 2 Traverse Draw quad Rasterize

Multiple Rendering Passes Restore (rays) Pass 2 Traverse Run fragment program

Multiple Rendering Passes Pass 2 Traverse Save to offscreen buffer (ray voxel pr)

Streaming Ray Tracer Generate Eye Rays Traverse Acceleration Structure Intersect Triangles Shade Hits and Generate Shading Rays Camera Grid Triangles Materials

Multipass Optimization l Reduce the number of passes –Choose to traverse or intersect based on work to be done for each type of pass n Connection Machine ray tracer [Delany 1988] n Intersect once 20% of active rays need intersecting l Make each pass less expensive –Most passes involve only a few rays –Early fragment kill based on fragment mask n Saves compute and bandwidth

Scene Statistics v – average number of voxels a ray pierces t – average triangles a ray intersects s – average number of shading evaluations per ray P – number of rendering passes s t v C = R*(Cr + v*Cv + t*Ct + s*Cs) + R*P*Cmask P

Performance Estimates l Pentium III 800 MHz CPU implementation –20M intersections/s [Wald et al. 2001] l Simulated performance –2G instructions/s and 8GB/s bandwidth –Instruction limited n 56M intersections/s –Nearly bandwidth limited n 222M intersections/s l Streaming ray tracing is compute limited!

Demo Analysis l Prototype Performance (ATI R300) –500K – 1.4M raycast/s –94M intersections/s –Only three weeks of coding effort l ATI Radeon 8500 GPU (R200) –114M intersections/s [Carr et al. 2002] –Fixed point operations –Only ray-triangle intersection kernel

Summary l Programmable GPU is a stream processor l Ray tracing can be a streaming computation l Complete ray tracer can map onto the GPU –Ray tracing generally thought to be incompatible with the traditional graphics pipeline l Streaming GPU-based ray tracer is competitive with CPU-based ray tracer

Architectural Results l Fragment mask proposed for efficient multipass –Stream buffer eliminates this need l Stream data should not go through standard texture cache l Triangles cache well for primary rays, secondary less so l Branching architecture –More cache coherence than the multipass architecture for scene data –Reduces memory bandwidth for stream data –But has its own costs…

Final Thoughts l Ray tracing maps into current GPU architecture –Does not require fundamentally different hardware –Hybrid algorithms possible l What else can the GPU do? –Given you can do ray tracing, you can do anything –Fluid flow, molecular dynamics, etc. l GPU performance increase will continue to outpace CPU performance increase