Embed Size (px)
DESCRIPTION
We have built a camera that can look around corners and beyond the line of sight. The camera uses light that travels from the object to the camera indirectly, by reflecting off walls or other obstacles, to reconstruct a 3D shape.
Citation preview
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
MIT Media LabRamesh Raskar
Computational Light Transport:
CORNAR: Looking Around Corners using
Trillion FPS Imaging
http://raskar.info/cornar
What isaround the corner ?
Can you look around the corner ?
Can you look around the corner ?
2nd Bounce
Multi-path Analysis
1st Bounce
3rd Bounce
Bit Hacking
Phot
on H
acki
ng
Computer Vision
Optics
Sensors
Signal Processing
Displays
Machine Learning
Computational Light Transport
Computational PhotographyIllumination
Co-designing Optical and Digital Processing
2nd Bounce
Multi-path Analysis
1st Bounce
3rd Bounce
CORNAR: Femto-PhotographyFemtoFlash
Trillion FPS camera
Computational Optics
Serious Sync
•2012: 3D around a corner (NatureComm, Velten, et. al.)•2011: Material Sensing (Siggraph Asia, Naik, Zhao, Velten, Raskar, Bala)•2011: DARPA Young Faculty Award•2011: Motion Sensing (CVPR, Pandharkar, Velten, Bardagjy, Bawendi, Raskar)•2009: Hidden barcode (Kirmani, Hutchinson, Davis, Raskar, ICCV’2009)•2008: Indirect depth (Hirsch, Raskar)•2008: Transient Light Transport (Raskar, Davis, March 2008)
With M Bawendi,
MIT Chemistry
Inverting Light Transport
Multiple Scattering Direct/Global
Dual Photography
[Seitz , Kutulakos, Matsushita 2005] [Nayar, Raskar et al 2006]
[Sen et al 2005]
LIDAR
[Atcheson et al 2008][Kutulakos, Steger 2005]
Collision avoidance, robot navigation, …
…, bronchoscopies, …
L
xz
s
S
Streak-camera
Laser beam
Occluder
CB
Echoes of Light
L
xz
s
S
Streak-camera
Laser beam
Occluder
CB
Echoes of Light
Streak Photo
L
xz
s
S
Streak-camera
Laser beam
Occluder
CB
Echoes of Light
3rd bounce
R
Multi-Dimensional Light Transport
5-D Transport
s1
Streak-camera
Occluder
C
3rd bounce
Why Pico-second Resolution?
s2ToF Diff = 0.15 mm
p1
p21st bounce
1cm
Curse of Pythagoras
Streak Photo
L
xz
s
S
Streak-camera
Laser beam
Occluder
CB
Echoes of Light
3rd bounce
R
Trillion FPS
ToF Streak Tube = Inverse of CRT
1D camera: Single scan line stretched vertically in time~2 ps resolution, 480 lines ~= 1 ns
But for small samples in biochemistry
Very accurate sync
Time-Image
Time Profile for a single pixel
R
Third Bounce (First bounce not shown)
Space, 640 pixels
Tim
e, ~
2ns e
ach
row
Time Image of a single point
Forward
Wavefront Propagation
Reconstruction
Non-linear Inversion
Invertibility
Scene Priors
Analysis
Resolution and dimensions
3D shape result from synthetic data
Steady State 4D
Impulse Response, 5D
[Raskar and Davis, 2007]
[Kajiya, 1986] [Seitz.., 2005]
Scene with hidden elements
Raw Time profiles
t
Signal Proc.
Novel light transport models and inference
algorithms
Photo, geometry, reflectance
beyond line of sight
3D Time images
Ultra fast illumination and camera
5D Capture
Time Resolved Multi-path Imaging
Third Bounce (First bounce not shown)
Third Bounce (First bounce not shown)
Third Bounce (First bounce not shown)
Third Bounce (First bounce not shown)
Third Bounce (First bounce not shown)
Capture Setup
Photos from Streak Camera
Hidden Scene
Capture Setup
Photos from Streak Camera
Hidden Scene OverlayReconstruction
Hidden 3D Shape Space-time Photos
Motion beyond line of sight
Pandharkar, Velten, Bardagjy, Lawson, Bawendi, Raskar, CVPR 2011
BRDF (reflectance) from Single Viewpoint and Time Images
Naik, Zhao, Velten, Raskar, Bala, (SIGGRAPH Asia 2011)
…, cardioscopies, …
Participating Media
Space-time Photos
Trillion Frames Per Second Imaging
http://raskar.info/trillionfps
Each frame = ~2ps = 0.6 mm of Light Travel
Camera Time
Camera Time
raskar.info/trillionfps
World Time
Lorentz transformation: distances, velocities, ordering
raskar.info/trillionfps
Pixel Ray Wave Photons(ampli+phase)
Steady State TransientImpulse/Step•2012: 3D around corner (Nature Comm, Velten et al)•2011: Material Sensing (Siggraph Asia, Naik, Zhao, Velten, Raskar, Bala)•2011: DARPA Young Faculty Award•2011: Motion Sensing (CVPR, Pandharkar, Velten, Bardagjy, Bawendi, Raskar)•2009: Hidden barcode (Kirmani, Hutchinson, Davis, Raskar, ICCV’2009)•2008: Indirect depth (Hirsch, Raskar)•2008: Transient Light Transport (Raskar, Davis, March 2008)
FemtoFlash
Trillion FPS camera
Computational OpticsSerious Sync
• Collaborations Welcome• Dataset Available• Propose Configurations
raskar(at)media.mit.eduhttp://www.media.mit.edu/~raskar/cornar/
Femto-PhotographyLooking Around the Corner
BRDF Detection
Trillion FPS Movies
Space-time Transforms
http://raskar.info
