GCAFE 28 Feb Real-time REYES Jeremy Sugerman

GCAFE 28 Feb This Talk  Still exploring future GPUs, Direct3D / GL  This time in a software context  Offline REYES, as RenderMan, is king of the rendering world  What would real-time REYES look like?  Where are current GPUs / Direct3D and REYES the same? Similar? Different?

GCAFE 28 Feb Background  “The Reyes Image Rendering Architecture” –Pixar: Rob Cook, Loren Carpenter, Ed Catmull –SIGGRAPH 1987, First film 1985, Oscar 1993  “An architecture optimized for fast high-quality rendering of complex animated scenes.”  Fast: feature-length film in a year  High-Quality: virtually indistinguishable from live action  Complex: as visually rich as real scenes.

GCAFE 28 Feb REYES Features  Primitives are smooth surfaces –Automatic, adaptive refinement  Programmable shading including displacement –On surface-space grids  Very high resolution geometry for visibility –After shading  Stochastic sampling in time, space, lens –Motion-blur, anti-aliasing, depth of field  Order-independent (depth sorted) blending  Wide pixel reconstruction filtering

GCAFE 28 Feb Basic REYES Pipeline Split Blend & Filter Sample Bust & Bound Shade Dice Bound Primitives Unshaded Grids Shaded Grids Micropolygons Visible Points Primitives Multiple Primitives Parallel scalability is added with screen-space bucketing computed during the Bounding and Splitting. Deferring Dicing also allows each bucket to be depth-sorted and occlusion culled.

GCAFE 28 Feb REYES on a GPU?  If you squint at it right… ‘Grids of micropolygons’ → ‘quads’ ‘Displace and shade’ → ‘shade vertices’(*) ‘Bust & bound, sample’ → ‘rasterize’(*) ‘Blend & filter’ → ‘blend & filter’(*)  Technically, OpenGL even includes implicit surface (NURBS) evaluators, but there is no serious support in (current) GPUs.

GCAFE 28 Feb Opaque GPU-REYES  Tesselate to grids in advance (CPU, cache, …)  Render grids as GL_QUADS  Displace, shade grids with a vertex shader  Cull, sample, and produce visible points with the rasterizer (MSAA / FSAA)  Reconstruction during ROP / FB  Flat fragment shading with micropolygons!  Current GPU lacking in tesselation, shading richness, stochastic sampling, blending

GCAFE 28 Feb GPU Mismatches  High level surfaces versus triangles  Surface vertex shading versus fragment shading –Including a constrained shading language  Stochastic sampling versus MSAA rasterization  Depth order versus primitive order blending  Note: REYES only tries to make ‘pictures’.  Bonus: Opaque REYES is totally unordered.

GCAFE 28 Feb Straightforward GPU Evolution  High level surfaces are already coming –Without ordering, performance is easy  Shading surface points versus pixels is fakeable –Less flexible GLSL/HLSL is fine –Unified shader cores help a lot –Adjust rast for small quad workloads  Stochastic sampling versus MSAA rasterization –Allow rast to time-sample linear motion

GCAFE 28 Feb Sticking Point: Blending  Primitive order blending generates poor images  Depth order risks unbounded storage needs –Samples must be buffered until all closer samples are blended. –Closest sample may not happen until the last grid of the last primitive.  Unbounded storage is anathema in GPU environments and real-time systems in general.

GCAFE 28 Feb Options for Depth-Order Blending  Classic REYES buckets grids into screen-space partitions. –Reduces, but still does not bound, footprint  Assume GPU REYES has fixed on-chip buffers  Three options when a bucket fills: –Rendering fails (awful) –Spill to memory (slow? when memory fills?) –Shrink bucket, discard samples newly outside (recomputation hard? too compute wasteful?)

GCAFE 28 Feb GPU REYES Summary  Unordered  Retained mode input (at least for blending)  Simplified shading language  Separate opaque and blended modes  Constrained reconstruction filters  Bound and split on CPU? Simplified adaptivity?  Mandatory displacement shader separation?  Constraints on stochastic sampling?  Blending policy– Primitive order? Spill? Shrink?

GCAFE 28 Feb Acknowledgements  Kayvon has provided explanation and discussion beyond a point where I can separate any ideas unique to myself.  Kurt and Pat provided excellent historical context and insight.  Gordon Stoll and Bill Mark repeated the word ‘REYES’ and insisted that its shading / anti- aliasing properties were valuable until I dug into it in self-defense.