Bridging the Gap to the Real Wojciech Matusik Adobe Systems, Inc.

Computer Graphics: Past, Present, Future Goal of this Talk –Look at Trends –Extrapolate –New Challenges Disclaimer: My personal (very biased) view I have not even taken a cg course. Time Milestones

Time Milestones Computer Graphics: Past, Present, Future Goal of this Talk –Look at Trends –Extrapolate –New Challenges Disclaimer: I will present my own biased view.

A Simulation to Generate Images or Videos Simulation Lights Shading Motion Geometry

Traditional Computer Graphics Simulation Lights Shading Motion Geometry Components –Hand Modeled –Procedurally Defined –Analytical Formulas Courtesy of Wikimedia

Traditional Computer Graphics: Geometry Simulation Lights Shading Motion Geometry axiom = F production rule F -> FF-[-F+F+F]+[+F-F-F] L-Systems Fractals Perlin Noise, Hypertextures Courtesy of Wikimedia Courtesy of Ken Perlin Courtesy of Wikimedia

Traditional Computer Graphics: Lights Simulation Lights Shading Motion Geometry Directional Point Spot Courtesy of Wikimedia

Traditional Computer Graphics: Shading Simulation Lights Shading Motion Geometry Cook’s Shade Trees Phong Blinn-Phong Cook-Torrance Courtesy of Rob Cook

Traditional Computer Graphics: Animation Simulation Lights Shading Motion Geometry Animation Controllers Keyframing Courtesy of Jessica Hodgins

Traditional Computer Graphics Real World Lights, Geometry, Shading, Motion, Simulations Editing and User Input Computer Graphics

Data-driven Computer Graphics: 90s - now Real World Lights, Geometry, Shading, Motion, Simulations Input Devices Progress in development of sensors Cheap storage and computation power for data processing

Data-driven Computer Graphics: Geometry Simulation Lights Shading Motion Geometry 3D Scanning Courtesy of Marc Levoy

Data-driven Computer Graphics: Lights Simulation Lights Shading Motion Geometry Environment Mapping HDR Imaging Courtesy of Paul DebevecCourtesy of Wikimedia

Data-driven Computer Graphics: Shading Simulation Lights Shading Motion Geometry Measured Appearance Courtesy of Kristin Dana Courtesy of Wikimedia

Data-driven Computer Graphics: Animation Simulation Lights Shading Motion Geometry Motion Capture Courtesy of Wikimedia

Data-driven Computer Graphics Real World Lights, Geometry, Shading, Motion, Simulations Input Devices

Data-driven Modeling and Editing Real World Lights, Geometry, Shading, Motion, Simulations Input Devices Editing and User Input

Data-driven Modeling and Editing: Geometry Simulation Lights Shading Motion Geometry Modeling by Example, 2004 Funkhouser et al.

Data-driven Modeling and Editing: Lights Simulation Lights Shading Motion Geometry Skyfinder : Attribute-based Sky Image Search, 2009 Tao et al.

Data-driven Modeling and Editing: Shading Simulation Lights Shading Motion Geometry Data-driven Reflectance Model, 2003 Matusik et al.

Data-driven Modeling and Editing: Animation Simulation Lights Shading Motion Geometry Motion Graphs, 2002 Kovar et al., Lee et al., Arikan & Forsyth

Data-driven Computer Graphics: 90s - now Real World Representations Abstractions Simulations Input Devices Editing and User Input

Computer Graphics: Future Real World Representations Abstractions Simulations Input Devices Editing and User Input

Shape Capture, Modeling, and Fabrication Real Object 3D Model 3D Scanning 3D Printing Output User Edits Courtesy of Wikimedia

Motion Capture, Modeling, and Robotics Human Motion Motion Model Mocap Controller Design Humanoid Robot User Edits Courtesy of Wikimedia

Light Field Capture, Modeling, and Display 3D Scene Light Field Camera Array Autostereoscopic Display Light Out User Edits Courtesy of M.Levoy Courtesy of F. Durand

Appearance Capture, Modeling, and Fabrication Real Material Data-driven Model Measurement Fabrication Output User Edits

Challenges: Output Mapping Data Abstraction Device Capabilities Device Capabilities Data Abstraction

Output Mapping Examples –Gamut Mapping –Tone Mapping –Image Retargeting Courtesy of Wikimedia Courtesy of Shai Avidan Courtesy of Wikimedia

Different Phenomena –Reflectance, Scattering, Deformations, Motion Rules –Linear vs. Perceptually-Based –Global vs. Local Generalized Output Mapping Algorithms

Complex aggregates from base materials with known properties Future Printers, Displays, and Output Devices

Future Output Devices Future Printers, Displays, and Output Devices Data-driven Representations and Abstractions Output Mapping

Computer Graphics: Closing the Loop Real World Representations Abstractions Simulations Input Devices Editing and User Input

My Vision Statement The future of Computer Graphics and Interactive Techniques lies in novel input and output devices that operate in real- world environments and are accessible to everyone.