iCheat: A Representation for Artistic Control of Cinematic Lighting Juraj ObertJaroslav Křivánek Daniel Sýkora Fabio Pellacini Sumanta Pattanaik University of Central Florida Czech Technical University in Prague Dartmouth College 2 3

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 2 Motivation Lighting crucial for film –Story telling, scene mood, emotions Simple control more important than physical accuracy –Direct illumination: easy to control –Indirect illumination: hard to control

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 3 Goal Simple artistic control of indirect illumination

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 4 Existing Approaches Relighting engines [Pellacini05, Ragan-Kelley07, Hasan06] Goal-based lighting design [Schoeneman93, Kawai93, Pellacini07] Procedural modifications of shader code [Christensen06] Platform-specific solutions [Tabellion04] – filter lights

Our Approach “Holistic approach” Direct control of the interaction between –direct lighting –materials –geometry Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 5

6 Lighting Design Pipeline Lighting design interface Scale/offset matrix Final rendering Real-time preview

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 7 Lighting design interface Lighting Design Interface [Obert07] [Obert07] 1. 4D selection a.caster b.receiver 2. Modification –hue / sat / intensity –falloff …

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 8 Lighting Design Pipeline Lighting design interface Scale/offset matrix Final rendering Real-time preview

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 9 Real-Time Preview Real-time preview Direct-to-Indirect Transfer [Hasan06] –Transfer matrix T –direct on gather samples -> indirect on view samples –Performance / flexibility modifications –Row-wise matrix multiply –Screen space sub-sampling and interpolation

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 10 Lighting Design Pipeline Lighting design interface Scale/offset matrix Final rendering Real-time preview

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 11 Encoding of Indirect Illumination Adjustments Scale/offset matrix Key idea: –Discrete sampling in 4D –Scale/offset modify the transport matrix T ),(),(),(),('gvogvTgvsgvT 

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 12 Example Edits: Indirect Brightness Sphere reflects more light into the environment

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 13 Example edits: Color bleeding Blue wall bleeds more color on the cone

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 14 Geometry Term Edits Modify the geometry term G(g,v) Linear w.r.t. direct illumination at gather samples Example effects: –Distance falloff modification, orientation-based modification, etc.

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 15 General Edits Non-linear w.r.t. direct illumination at gather samples –s / o matrices valid for current direct illumination Example effects –4D gamma correction, 4D HSV correction, etc.

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 16 Lighting Design Pipeline Lighting design interface Scale/offset matrix Final rendering Real-time preview

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 17 Final rendering Applicable to any GI algorithm Our implementation: –irradiance caching [Ward88] –radiance caching [Křivánek05] Radiance of gather rays modified by the s / o matrices –4D interpolation

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 18 Final Rendering Example Original Design 1 Design 2

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 19 Animation Edits designed per key-frame Interpolated for other frames –Possible artifacts when previously occluded large areas become suddenly visible

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 20 Interpolation Example iCheat keyframe iCheat KeyframeInterpolated frame Original

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 21 More Interpolation iCheat keyframe iCheat KeyframeInterpolated frame Original

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 22 Video

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 23 Conclusion Artistic control of indirect illumination Simple representation of edits –Scale / offset factors for discretely sampled 4D transfer operator –Comprehensive, renderer-independent, efficient Limitations –More bulky than shader edits –Possible artifacts due to discrete sampling

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 24 Acknowledgements Vlasta Havran –Golem ray tracer Universal Production Partners (upp.cz) –Orion scene Ministry of Education of Czech Republic –“Center for Computer Graphics” National Science Foundation –CNS , CCF

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 25 References CHRISTENSEN P., FONG J., LAUR D., BATALI D.: Ray tracing for the movie cars. In Proc. of IEEE Symposium on Interactive Ray Tracing (2006), pp. 1–6. HAŠAN M., PELLACINI F., BALA K.: Direct-to-indirect transfer for cinematic relighting. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3 (2006), 1089–1097. KŘIVÁNEK J., GAUTRON P., PATTANAIK S., BOUATOUCH K.: Radiance caching for efficient global illumination computation. IEEE Transactions on Visualization and Computer Graphics 11, 5 (September/October 2005). KAWAI J. K., PAINTER J. S., COHEN M. F.: Radioptimization: Goal based rendering. In SIGGRAPH’93 Proceedings (1993), pp. 147–154. OBERT J., KŘIVÁNEK J., SÝKORA D., PATTANAIK S.: Interactive light transport editing for flexible global illumination. ACM SIGGRAPH 2007 sketches

Obert et al. iCheat: A Representation for Artistic Control of Cinematic Lighting 26 References PELLACINI F., BATTAGLIA F., MORLEY R. K., FINKELSTEIN A.: Lighting with paint. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 2 (2007). PELLACINI F., VIDIMČE K., LEFOHN A., MOHR A., LEONE M., WARREN J.: Lpics: A hybrid hardware-accelerated relighting engine for computer cinematography. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3 (2005), 464– 470. RAGAN-KELLEY J., KILPATRICK C., SMITH B. W., EPPS D., GREEN P., HERY C., DURAND F.: The lightspeed automatic interactive lighting preview system. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (2007). TABELLION E., LAMORLETTE A.: An approximate global illumination system for computer generated films. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3 (2004), 469–476. WARD G. J., RUBINSTEIN F. M., CLEAR R. D.: A ray tracing solution for diffuse interreflection. In SIGGRAPH’88 Proceedings (1988), pp. 85–92.