Axial Light Field for Curved Mirrors: Reflect Your Perspective, Widen Your View

Mitsubishi Electric Research Labs (MERL) Axial Light Field Taguchi, Agrawal, Ramalingam & Veeraraghavan

Axial Light Field for Curved Mirrors:Axial Light Field for Curved Mirrors:Reflect Your Perspective, Widen Your ViewReflect Your Perspective, Widen Your View

Yuichi TaguchiAmit Agrawal

Srikumar RamalingamAshok Veeraraghavan

Mitsubishi Electric Research Labs (MERL)


Non-Single Viewpoint Image(Spherical Mirror + Perspective Camera)


Single-Viewpoint Image (Cube Map, FOV 140)


No Approximation, No Knowledge of Scene Geometry


Single/Non-Single ViewpointSingle/Non-Single Viewpoint

Perspective Camera Catadioptric System (Mirror + Camera)

Single Viewpoint Single Viewpoint Non-Single Viewpoint

Virtual Viewpoint

Hyperbolic Mirror Spherical

MirrorLocus of

Viewpoint (Caustic)

Perspective Camera

Perspective Camera


Single-Viewpoint Catadioptric SystemsSingle-Viewpoint Catadioptric Systems

• Only a few single-viewpoint configurations

• Other common configurations lead to non-single viewpoint– Spherical mirror– Parabolic mirror with perspective camera– Hyperbolic/elliptic mirror with perspective camera not on foci

[Baker & Nayar 99]

Hyperbola/Ellipse Parabola

Foci

Orthographic CameraPerspective Camera

Virtual Viewpoint


Generating Single-Viewpoint Image fromGenerating Single-Viewpoint Image fromNon-Single Viewpoint ImageNon-Single Viewpoint Image

• Distortion Correction Approaches– Use one image– Single-viewpoint approximation – Use scene prior [Swaminathan et al. 03]

– Generating exact perspective views is impossible without knowing scene geometry

• Our Approach– Use multiple images– Capture all the rays required to generate an exact single-

viewpoint image– Does not require scene prior and 3D reconstruction

Scene Prior (e.g. Plane)


• If we capture all the rays that pass through the virtual viewpoint, we can generate a single-viewpoint image– What is the best possible sampling?

Light FieldLight Field

Mirror Surface

Virtual Viewpoint

Light Field Plane

[Levoy & Hanrahan 96, Gortler et al. 96]


Input Images (Axial Light Field)


Input Output

Copy Circles


Single-Viewpoint Image (Cube Map, FOV 140)


Key IdeaKey Idea

• Rotationally symmetric mirrors• Capture axial light field

– Move the camera along the mirror axis– Exact single-viewpoint image generation

without scene prior and 3D reconstruction– Better sampling than typical planar light field

Planar Light Field

AxialLight Field

Symmetry Axis


Geometric InterpretationGeometric Interpretation

A cone of rays in virtual camera

(Angle )

A cone of raysin real camera

(Distance d, Angle )

d

Axial camera [Ramalingam et al. 06](All the rays pass through the axis)

Virtual Viewpoint

Rotationally Symmetric

Mirror


Light Fields for Symmetric Mirrors in 3DLight Fields for Symmetric Mirrors in 3D

x

y

u

v

x-u-v Slice

x

u

v



x

Virtual Viewpoint

Spherical Mirror

y

Planar Light Field Sampling

x-u-v Slice

x

u

v

y=0



x

Virtual Viewpoint

Spherical Mirror

y

Planar Light Field Sampling

x

u

v

x-u-v Slice y0



x

Virtual Viewpoint

Spherical Mirror

y

x

u

v

Axial Light Field Sampling

x-u-v Slice (x,y)=(uz,vz)



x

Virtual Viewpoint

Spherical Mirror

y

u

v

Axial Light Field Sampling

Image Plane

u-v Slice (x,y)=(uz,vz)


Planar LF vs. Axial LFPlanar LF vs. Axial LF

• Planar Light Field– Sample a 4D subset of rays– The camera captures a cone of rays

only when placed on the mirror axis• Other cameras capture only a few rays

• Axial Light Field– Sample a 3D subset of rays– Every camera captures a cone of rays

• Light rays required to generate a single-viewpoint image are concentrated in this 3D subset



(Angle )


(Distance d, Angle )

d

Virtual Camera

Real Camera

Axial LF Sampling ParametersAxial LF Sampling Parameters


Axial LF Sampling ParametersAxial LF Sampling Parameters

Ray angle in virtual camera () []

Location of real camera (d)

Ray angle in virtual camera () []

Ray angle in real camera () []

Real Camera FOV

Cone

Concave Sphere

Parabola

SphereMirror Shapes

Virtual camera

Resolution Reduction


Single-Viewpoint Image GenerationSingle-Viewpoint Image Generation

Input Images

Virtual Perspective Image

Copy Resized Circles


(Angle )


(Angle , Distance d)


Simulation Results for Different Mirror ShapesSimulation Results for Different Mirror Shapes

Input Far

Input Near

Output

ConeConcave SphereParabolaSphere

Limited Input FOV

InvalidZ positionFOV 24

FOV 140


Changing Resolution PropertyChanging Resolution Property

• Change focal length (zoom) for each capture position– Change resolution property without changing mirror shapes

0 50 100 150 200 250 3000

50

100

150

200

250

300

Circle radius in virtual image [pixels]

Circle radius in real image [pixels]

0 50 100 150

1

2

3

4

5

Mag. factor of focal length

Location of real camera

Spherical Mirror

Constant focal length

Variable focal length

To achieve resolution propertysimilar to a perspective camera


ComparisonComparison

Planar Light Field Axial Light Field

Same Number (25) of Input Images (FOV 32 x 24)

Setup

Mirror Ball

Robot Arm


Output Cube Maps (FOV 140)

Aliasing /Ghosting

Planar Light Field Axial Light Field

ComparisonComparison


AdvantagesAdvantages

• Axial light field based catadioptric imaging

– Exact single-viewpoint image generation in wide angle• Without any scene prior, 3D reconstruction

– For any rotationally symmetric mirror

– Offer changing resolution property• Without changing mirror shape

– Virtual viewpoints can be varied on the mirror axis


LimitationsLimitations

• Static scene

• Precise camera motion along the mirror axis

• Implementation using multiple cameras would be difficult


SummarySummary

• Analysis of light rays for rotationally symmetric mirrors• Axial light field sampling• Exact single-viewpoint image generation

Documents

Axial Light Field for Curved Mirrors: Reflect Your Perspective, Widen Your View