A Multicamera Setup for Generating Stereo Panoramic Video

Tzavidas, S. , Katsaggelos, A.K. Multimedia , IEEE Transactions on

Volume: 7 , Issue:5 Publication Year: 2005

Professor: Jar - Ferr YangPresenter: Kuan - Hsien - Liu

Outline

• Introduction• Description of Proposed Stereo Panoramic

Video Setup• Analysis of Proposed Stereo Panoramic

Video Setup(tradeoff)• Disparity Variation• Experimental Results• Conclusion

Introduction• We present a new multi-camera setup that is capable of creating the desired views (360-

degree FOV with horizontal disparity) at video rates.

• Multiple viewpoint– Different viewpoint for different viewing direction

Circular Projections for Stereo Panorama

Circular projections: world points P1 and P2 are projected onto the image surface by using the rays tangent to the viewing circle.

Every world point P is pictured through two different viewpoints for the left and right panoramic images. Viewpoint R is used for the construction of the right panoramic image and viewpoint L for the construction of the left.

Circular Projection Circular + Stereo

• Rotating the regular camera and collecting small strips from the left part of the images produced by the regular camera , we construct the panoramic image for the right eye , and the lefteye is Similarly.

• The width of the strips is a function of the number of the cameras in the rig. In a practical scenario, we only have a limited number of images available, which results in a relatively large width of the strips (tens or hundreds of pixels). Project image to cylindrical surface before taking the strip.

Description of Proposed Stereo Panoramic Video Setup

• Consist of multiple cameras mounted on a rig with fixed geometry.

• We employ circular projections for the construction ofeach frame in the stereo panoramic video sequence.

• Taking the appropriate strips from the left and right part of the images and pasting them together.

• Moving part

ANALYSIS of PROPOSED STEREO PANORAMIC VIDEO SETUP

A. Size of the Rig and Perspective Correction• The circumference number of cameras• We assume that the rig is a regular polygon , each camera occupies one side of this polygon. W: width of physical camera N: number of cameras in the rig R: radius of the rig

,

N

2

)tan()2

tan(2

2)2

tan(

N

WWR

R

W

• This setup’s key point is ( N , W )

B. “Blind” Areas• Limited number of camerasSome parts of the environment are not imaged

by any of the cameras in the rig.

• Direction of view is the ray tangent to the viewing circle is defined by the angle

)arcsin(R

r

•How far the strip away from the central line of image?

Before taking the strips, each image is warped by performing a cylindrical projection , We obtain the strip by taking the part of the projected on the cylinder image that corresponds to angle Ω .

Line represents the leftmost boundary of the part of the environment that is projected onto the strip by camera1any world points lying to the left of this line will not be projected on the strip by camera 1.

11IC

Line represents the rightmost boundary of the part of the environment that is projected onto the strip by camera 2any world points lying to the right of this line will not be projected on the strip by camera 2.

22IC

22

)()22

(11111

ICCCCI

)sin()sin(|||| 11 RCCCT

22

)()22

(22222

ICCCCI

)sin()sin(|||| 22 RCCCT

左邊界

右邊界

• Referring to above analysis , there are three possible values for the angle Ω

11IC 22ICΩ < Ө/2: lines and intersect at a point in the lower half of the plane , the two lines move further apart thus leaving a growing part of the scene uncovered

11IC 22ICΩ = Ө/2: lines and become parallel , their slopes become equal( ) , so there are no world points that projected on both strips. The two successive strips have no overlapping regions.

21

Ω > Ө/2: lines and intersect at a point in the upper half of the plane.Therefore , the two successive strips overlap. The best? Not really!

11IC 22IC

We are interested in creating panoramic images at video rates and we want to avoid computationally expensive steps.So we use Ω = Ө/2

Because Ω = Ө/2 , so there is a blind area between strips. Therefore , the distance between two parallel lines:

))(cos(arcsinsin2

cos2

sin221

R

r

NR

RCTCT

))2

sin()2

(sin(

)2

sin()2

sin(21

R

RRCTCT

“Blind” Areas

A setup is valid or not?

222max

FOVFOV

222max

FOVFOV)tan()

2tan(2

N

WWR )arcsin(

R

rIn the previous: ,

• The increase in the number of cameras does not reduce the width of blind strips.

• For smaller cameras , the resulting blind strip width is smaller.

N=21

C. Disparity Variation • In actual Setup , the symmetrical construction is destroyed due to the limited cameras.The main consequence of such lack of symmetry is a change in the disparity of objects in the resulting stereo panorama

100

100

)(

rityIdealDispa

arityActualDisprityIdealDispa

ZariatonDisparityV

Ideal case Actual case

)arcsin(

)sin(

Z

rZ

r

)))2

cos((

)2

sin(arctan(

RZ

R

• Keep the maximum disparity variation as a representative value that describes the performance with respect to disparity of this particular setup.

• This change in disparity between ideal and actual case is defined as disparity variation

• Adding more cameras, beyond this critical number, does not improve the performance of the rig

• For a given number of cameras , there is a clear gain in going to cameras with smaller physical dimensions the maximum disparity variation is actually decreasing for smaller M

• when N is constant. We also observe in Fig. 13 that the maximum disparity variation even in the best case is quite significant (20% in the best case, almost 50% in the worst).

• Is the Disparity variation likely to produce a panorama that would uncomfortable for a human viewer?

It is known that disparity is a cue that is used by the human eye to judge relative depths only. when viewing a certain scene the eyes of the observer fixate at a certain depth, and use disparities to judge relative depths only for small depth differences with respect to the fixation point. We saw earlier that the disparity variation is not the same for different values of the depth. However, for a small range of depths we can assume that all corresponding disparities change approximately the same with respect to the ideal. In other words, their relative sizes remain approximately the same.

Disparity Variation 1002

22),(

0

0

offsetZ

The disparity at the same depth Z but at different directions

The angular disparity of P’= 21 ''2 COPCOP

We calculate the disparity 2ω and the percentage change with respect to the disparity for offset = 0 , which we call 2ω0(ω0 is given by (12))

• Smaller values of W tend to produce smaller disparity variation. • An increasing N tends to reduce the percent disparity variation. • AS Z increases , the distance between the different curves becomes smaller.• The minimum number of cameras needed is 21 , and the resulting maximum disparity variation is less than 2%.the lack of symmetry of an actual setup produces some variation in disparity but this variation is not likely to produce any visible effects in the resulting panoramic image.

Experimental Results

Virtual environment

One frame in the panoramic video

N=21 W=42(mm)

N=32 W=42(mm)

N=43 W=16(mm)

Conclusion

• The Important parameters defining a setup– Number of cameras & physical width of camera

• The minimum number of camera is the optimal choice(given a type of camera , ex: FOV & physical dimensions)

• The size of camera becomes smaller– Small rig– Reduces the blind area– Smaller disparity variation

• This setup will produced lots of data– New compression tech need to be developed for storage

and transmission.

　　　　

Thanks for your attention!

附錄 : Circular projection• For stereo panorama we develop a special type

of multiple viewpoint projections, where both the left-eye image share the same cylindrical surface. To enable stereo perception, the left-eye and the right-eye are located on viewing circle.

• The viewing direction is on the line tangent to the viewing circle.