Breaking the Frame David Luebke University of Virginia

Graphics Hardware 2005: Evolution or Revolution? Frameless Rendering Technique: [Bishop et al Implementation & Video[Parker et al. 1999]Video Codec: huffYUV [dll] [inf]dllinf

Graphics Hardware 2005: Evolution or Revolution? Overview: What We’re Doing Spatio-temporally adaptive frameless sampling –Prioritize sampling towards regions of greater change Spatial change: edges Temporal change: motion Reconstruction of resulting samples –A “deep buffer” stores samples in time & space –Reconstruct image at front edge of time: apply filter kernel with varying width in space and time

Graphics Hardware 2005: Evolution or Revolution? Static scenes/regions –Old samples useful, use them to sharpen/antialias –Temporal width should dominate Dynamic scenes/regions –New samples useful, old samples stale –Emphasize new samples even if image is less sharp –Spatial width should dominate Temporally Adaptive Reconstruction

“Traditional” frameless Adaptive frameless Adaptive Frameless Rendering [Dayal et al., EGSR05] Video Preview

Graphics Hardware 2005: Evolution or Revolution? Summary Better than traditional frameless rendering Better than traditional framed rendering! –Frameless = ungridded temporal sampling  lower latency –Samples when and where needed  better images at low sampling rates –Antialiases static regions by incorporating old samples  lower error even than 10x sampling rate Still in simulation

Discussion: Asynchronous Graphics What if reconstruction was part of display? –Imagine display as systolic array of pixels –Input: stream of samples, not sequence of images Enables asynchronous parallel graphics –Parallel graphics frameworks share common constraint: must ultimately combine all results into a single frame –Breaking the frame also breaks underlying assumption and constraint in parallel graphics! See SIGGRAPH Panel “The Ultimate Display” –Punchline: Refreshing every pixel every time = Bad Idea

The End Acknowledgements: OpenRT Interactive Raytracing Project BART ray tracing benchmark Stanford 3D Scanning Repository National Science Foundation awards , , and

Graphics Hardware 2005: Evolution or Revolution? SamplerReconstructor Controller Deep Buffer Ray Tracer Adaptive Filter Bank Deep Buffer samples variation, gradients image locations samples tiling, view, gradients SamplerReconstructor Controller Deep Buffer Ray Tracer Adaptive Filter Bank Deep Buffer samples variation, gradients image locations samples tiling, view, gradients System overview

static scenedynamic scene Temporally Adaptive Reconstruction

static scenedynamic scene Comparison: Traditional Frameless

Graphics Hardware 2005: Evolution or Revolution? Discussion: Coherence What about coherence? –Frameless rendering implicitly gives up spatial coherence, which is big win for fast ray tracers Partially ameliorate with tiled structure, gradient rays Might need to organize “random” samples around memory –But we gain temporal coherence! Fewer samples: needn’t resample everywhere every frame Can we design a parallel architecture around this temporal coherence?

Graphics Hardware 2005: Evolution or Revolution? Comparison: Render Cache Probably most closely related approach –Sampling based on (framed) priority image Biased toward old & undersampled regions –Killing off old samples also biases towards age –Semantic “hints” age some samples quicker (e.g. specular surfaces) –Temporal response by aging samples if new one detects variance Error diffusion dither to place samples within image –Image-space reconstruction via (non-adaptive) filtering 7x7 “prefilter” followed by 3x3 Gaussian Depth culling helps with occlusions –See [Walter et al 1999]

Graphics Hardware 2005: Evolution or Revolution? Evaluation: Mostly Dynamic

Graphics Hardware 2005: Evolution or Revolution? Evaluation: Mostly Static