MIT Media Lab | Camera Culture
Shoot Now, Relight LaterIllumination Multiplexing with Lock-In Time of Flight SensorsAchuta Kadambi, Ayush Bhandari, Refael Whyte, Adrian Dorrington, Ramesh Raskar
How Can We
Relight a
Photograph?In this paper we propose a fusion of two
popular contexts, time of flight range cameras
and illumination multiplexing. Time of flight
cameras are a low cost, consumer-oriented
technology capable of acquiring range maps
at 30 frames per second. Such cameras have
a natural connection to conventional
illumination multiplexing strategies as both
paradigms rely on the capture of multiple
shots and synchronized illumination. While
previous work on illumination multiplexing has
exploited coding at millisecond intervals, we
repurpose sensors that are ordinarily used in
time of flight imagstrategiesing to demultiplex
via nanosecond coding.
ICCP 2014
Illumination Multiplexing
Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007
Time of Flight 3D Sensors
Time of Flight 3D cameras are an emerging camera technology. Such technology forms the basis for the new Kinect.
Time of Flight Operating Principle
Operation of ToF range cameras. The time difference of arrival between the emitted and received code is encoded in the phase offset. Thus, phase encodes depth.
Nanosecond vs Millisecond Coding
Case 1: Conventional High Speed MultiplexingCase 2: Using a lock-in ToF sensor allows for nanosecond coding.Case 3: Optimizing the Nanosecond Codes is the goal.
Illumination Multiplexing
Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007
Demonstration: Relighting Post-Capture
Demonstration: Scene Relighting.
Demonstration: Multiplexing Wavelength
Demonstration: Color ToF camera by multiplexing RGB lights.
Hardware Prototype
We validate our technique with a hardware prototype. It consists of an FPGA, Lock-in Sensor, and Solid State Illumination sources.
www.media.mit.edu/~achoo/demux