Raskar, Camera Culture, MIT Media Lab Camera Culture Ramesh Raskar Camera Culture Associate Professor, MIT Media Lab

Capturing and Sharing the Experience Computational Cinematography and Display –Future cameras for Movies and News –6D displays Morphable Studios –Illumination and appearance on proxy geometry Programmable Movies –Enriching with Meta-data Performance Capture –‘On-set’ natural motion capture, ‘Second Skin’ Ramesh Raskar, MIT Media Lab

Captured Blurred Photo

Refocused on Person

6D = light sensitive 4D display One Pixel of a 6D Display = 4D Display

Desired Virtual Model © Andrei State Morphable Studios ShaderLamps and BeingThere with UNC Chapel Hill

Towards ‘on-set’ performance capture 500 Hz with Id for each Marker Tag Visually imperceptible tags + Natural lighting Unlimited Number of Tags Base station and tags only a few 10’s $ Traditional: High-speed IR Camera + Body markers Second Skin: High-speed LED emitters+ Photosensing Body markers

Imperceptible Tags under clothing, tracked under ambient light

Capturing and Sharing the Experience Computational Cinematography and Display –Future cameras for Movies and News –6D displays Morphable Studios –Illumination and appearance on proxy geometry Programmable Movies –Enriching with Meta-data Performance Capture –‘On-set’ natural motion capture, ‘Second Skin’ Ramesh Raskar, MIT Media Lab

Mitsubishi Electric Research LaboratoriesSpecial Effects in the Real WorldRaskar 2006 Inverse Optical Mo-Cap Device High Speed Projector + Photosensing Markers High Speed Camera + Reflecting/Emitting Markers Params Location, Orientation, IllumLocation Settings Natural Settings Ambient Light Outdoors, Stage lighting Imperceptible tags Hidden under wardrobe Controlled Lighting Visible, High contrast Markers #of Tags Unlimited Space Labeling Unique Id Limited No Unique Id Marker swapping Speed Virtually unlimited Optical comm comps Limited Special high fps camera Cost Low Open-loop projectors Current: Projector/Tag=$100 High High bandwidth camera Current Camera: $10K Traditional

Coded Aperture Camera The aperture of a 100 mm lens is modified Rest of the camera is unmodified Insert a coded mask with chosen binary pattern

Capturing and Sharing the Experience Morphable Studios –Illumination and appearance on proxy geometry –Separating content from physical proxy Programmable Movies –Cliplet aggregation –Storytelling cameras via meta-data –Making cameras and world intelligent –Long-distance bar-codes Performance Capture –‘On-set’ motion capture, ‘Second Skin’ –Natural environments –Lightweight technologies Computational Cinematography and Display –Future cameras for Movies and News –Universal software platform –4D and 6D displays