Raskar, Camera Culture, MIT Media Lab

Camera Culture

Ramesh Raskar

Camera CultureAssociate Professor, MIT Media Lab

Capturing and Sharing the Experience

• Computational Cinematography and Display– Future cameras for Movies and News– 6D displays

• Morphable Studios– Illumination and appearance on proxy geometry

• Programmable Movies– Enriching with Meta-data

• Performance Capture– ‘On-set’ natural motion capture, ‘Second Skin’

Ramesh Raskar, MIT Media Lab

Captured Blurred Photo

Refocused on Person

6D = light sensitive 4D display

One Pixel of a 6D Display = 4D Display

Desired Virtual Model

© Andrei State

Morphable Studios

ShaderLamps and BeingThere with UNC Chapel Hill

Towards ‘on-set’ performance capture

• 500 Hz with Id for each Marker Tag• Visually imperceptible tags + Natural lighting• Unlimited Number of Tags• Base station and tags only a few 10’s $

Traditional: High-speed IR Camera + Body markers

Second Skin: High-speed LED emitters+ Photosensing Body markers

Imperceptible Tags under clothing, tracked under ambient light

Capturing and Sharing the Experience

• Computational Cinematography and Display– Future cameras for Movies and News– 6D displays

• Morphable Studios– Illumination and appearance on proxy geometry

• Programmable Movies– Enriching with Meta-data

• Performance Capture– ‘On-set’ natural motion capture, ‘Second Skin’

Ramesh Raskar, MIT Media Lab

Mitsubishi Electric Research Laboratories Special Effects in the Real World Raskar 2006Inverse Optical Mo-Cap

DeviceHigh Speed Projector + Photosensing Markers

High Speed Camera + Reflecting/Emitting Markers

Params Location, Orientation, Illum Location

Settings

Natural SettingsAmbient Light Outdoors, Stage lighting

Imperceptible tags Hidden under wardrobe

Controlled LightingVisible, High contrast Markers

#of TagsUnlimited

Space LabelingUnique Id

LimitedNo Unique Id Marker swapping

SpeedVirtually unlimited Optical comm comps

LimitedSpecial high fps camera

CostLow Open-loop projectors Current: Projector/Tag=$100

High High bandwidth camera Current Camera: $10K

Traditional

Coded Aperture CameraCoded Aperture Camera

The aperture of a 100 mm lens is modified

Rest of the camera is unmodifiedInsert a coded mask with chosen binary pattern

Capturing and Sharing the Experience

• Morphable Studios– Illumination and appearance on proxy geometry– Separating content from physical proxy

• Programmable Movies– Cliplet aggregation– Storytelling cameras via meta-data– Making cameras and world intelligent– Long-distance bar-codes

• Performance Capture– ‘On-set’ motion capture, ‘Second Skin’– Natural environments– Lightweight technologies

• Computational Cinematography and Display– Future cameras for Movies and News– Universal software platform– 4D and 6D displays