Slide 1

T. Georgiev, Adobe Systems A. Lumsdaine, Indiana UniversityThe Multi-Focus Plenoptic Camera

2010 Adobe Systems Incorporated. All Rights Reserved. Capturing Memorable Moments

2 2010 Adobe Systems Incorporated. All Rights Reserved. Whats Missing with this Picture?

3 2010 Adobe Systems Incorporated. All Rights Reserved. The Lippmann SensorCan we request that Photography renders the full variety offered by the direct observation of objects? Is it possible to create a photographic print in such a manner that it represents the exterior world framed, in appearance, between the boundaries of the print, as if those boundaries were that of a window opened on reality.

Gabriel Lippmann, 1908.

2010 Adobe Systems Incorporated. All Rights Reserved. Yes, through integral photography, because it captures the entire 4D radiance of a scene and can leverage that data to create Lipmanns window opened on reality. Lippmann first proposed technology (which he called integral photograph or reversible print) to do this -- back in 1908. 4Pixels and Cores: What do we do with that power?Moores Law: Megapixels keep growing7.2 MP = 8 by 10 at 300dpiAvailable on cell phones60MP sensors available nowLarger available soon (can a use be found?)Use pixels to capture richer information about a sceneComputationally process captured dataGPU power also riding Moores Law curve

5

2010 Adobe Systems Incorporated. All Rights Reserved. How do we take advantage of these trends?Plenoptic camera High resolution; computationally intensive renderingThe pixels are samples of the radiance in the 4D ray spaceOptical elements (lenses, space) transform the ray spaceWe take a picture by rendering (computationally)We adjust the picture by transforming the ray space (computationally)6

2010 Adobe Systems Incorporated. All Rights Reserved. The Full Variety: Focusing

2010 Adobe Systems Incorporated. All Rights Reserved. A photograph represents only one of many possible renderings of a scene. If you were at the scene, you could focus on whatever you decided and look at the scene from an infinite number of different perspectives.

7Focusing

8 2010 Adobe Systems Incorporated. All Rights Reserved. Focusing

9 2010 Adobe Systems Incorporated. All Rights Reserved. The Full Variety: Different Views10

2010 Adobe Systems Incorporated. All Rights Reserved. Different Views11

2010 Adobe Systems Incorporated. All Rights Reserved. Different Views12

2010 Adobe Systems Incorporated. All Rights Reserved. Radiance (Plenoptic Function, Lightfield)Instead of integrating rays coming from all directions into a given point (traditional camera), capture the rays individually (capture the radiance) using the Lippmann SensorRecord all the information about the scene13 2010 Adobe Systems Incorporated. All Rights Reserved. Plenoptic lens array outside the camera14

2010 Adobe Systems Incorporated. All Rights Reserved. Microlenses inside the camera15

2010 Adobe Systems Incorporated. All Rights Reserved. Problem - SolutionProblem: Limited depth of fieldSolution: Microlenses of different focal lengths(This is similar to HDR where we use microlenses of different apertures)16

2010 Adobe Systems Incorporated. All Rights Reserved. MultifocusLytro and Raytrix already selling plenoptic camerasVariable focal lengths: Better focusing. Our work on that started 2009.Raytrix actually use microlenses of different focal lengths in their camera.Lytro use single focal length (Plenoptic 1.0)17 2010 Adobe Systems Incorporated. All Rights Reserved. Variable Focal Lengths Microlenses18

2010 Adobe Systems Incorporated. All Rights Reserved. 19

Variable Focal Lengths Microlenses 2010 Adobe Systems Incorporated. All Rights Reserved. 20

Variable Focal Lengths Microlenses 2010 Adobe Systems Incorporated. All Rights Reserved. 21

Variable Focal Lengths Microlenses 2010 Adobe Systems Incorporated. All Rights Reserved. 2122

Variable Focal Lengths Microlenses 2010 Adobe Systems Incorporated. All Rights Reserved. 23

Variable Focal Lengths Microlenses: Compare Focus Quality 2010 Adobe Systems Incorporated. All Rights Reserved. 24Phase Space analysis

2010 Adobe Systems Incorporated. All Rights Reserved. 25Phase Space analysis

2010 Adobe Systems Incorporated. All Rights Reserved. 26Phase Space analysis: Miocrolenses of different focal lengths

2010 Adobe Systems Incorporated. All Rights Reserved. To Find Out MoreThis work was first presented at Asilomar2010/

27 2010 Adobe Systems Incorporated. All Rights Reserved. Live Demonstrations28 2010 Adobe Systems Incorporated. All Rights Reserved. 2010 Adobe Systems Incorporated. All Rights Reserved. 29

ab