Upload
jennifer-bell
View
212
Download
0
Tags:
Embed Size (px)
Citation preview
Visualization of Urban Scenes from Video Footages
Siyuan Fang
Supervised by Dr. Neill Campbell 29/10/2007
Overview
Generate panorama (or image mosaicing) from a video of street scenes
Potential applications: Virtual environment Robot navigation On-line tourist guide
Combine different source frames (or images) together
Outline
• Background • Research Proposal • Achievements • Future Plan
Mosaicing form Single-Viewpoint Images
Source images are captured using a camera with pure rotation movement around a fixed viewpoint, (the intrinsic parameter can also change)
Align (or register) source images and stitch them together, e.g., the general alignment (registration) model: homography for a pair of images.
The homography can be estimated by points (pixels or features) correspondence, robust estimations (e.g., RANSAC) are used to remove the outliers of the point correspondence
Further refinements: pixel value blending bundle adjustment
The limitation of the single-viewpoint image mosaicing--cannot put viewpoint far away enough
Mosaicing form Multi-Viewpoint Images : Perspective Mosaicing
• The difficulty: there isn’t a single uniform alignment --only planar scenes have the same homography if viewpoint is changing
• Traditional approaches: view synthesis and image based rendering e.g., [GORT96], [SHUM99]--picture surface alignment + residual pixel movement [KUMA95]
• User-interactive design: picture surface alignment + user refinement for the off-plane scenes [AGAR06]
• Limitations of these approaches
Strip Panorama: Parallel Projection Mosaicing
A simulation of the push-broom camera. A common setup: sideway translating camcorder and vertical trip scanning. [ZHEN03]
Parallel projection along the horizontal direction, hence, remain the perspective along the vertical direction, cause the aspect ratio distortion:
Further scenes get expanded Closer scenes get squashed
Strip Panorama: Parallel Projection Mosaicing Cont.
Two kinds of distortion, the same reason:
Aspect ratio distortion Blurred-truncated
distortion
A Better Strip Panorama: Crossed-Slit Mosaicing
The Crossed-Slit Cameral Model, and its special case: POX-slit model.
Strict sideway translation camcorder can be used to simulate POX-slit model: the cameral path is presumably considered as the horizontal slit and the vertical slit is determined by the camera path and the width of the image plane [ZOME03]
The Crossed-Slit Mosaicing cont.
Comparison between the Crossed-Slit Mosaicing and the Parallel projection Mosaicing
The Limitations of the Crossed-Slit Mosaicing (Distortions)
Aims and Objectives
Aims and basic requirements: . Use moving pinhole camcorder (normal) Capture real long street scenes Scalable Least distortion Automaticity
The Proposed Research Approach (provisional)
The proposed approach, 3 main tasks:I. Synthesis novel view based on the crossed-slit mosaic, but fully perspective
— remove two kinds of distortion based on geometry information
II. Handle extremely long scenes,
III. * Handle moving objects
Summary of Achievements
The literature survey on image mosaicing Single-viewpoint mosaicing View synthesis and imaged based rendering Approaches of strip mosaicng
A (provisional) research proposal
Some design and implementation work A single-viewpoint mosaic tool A part of the task 1
Future Plan
Further refine (modify) the research proposal
Complete the task 1 A formal math model and an
approximation algorithm Curved camera path and corner
effect Writing up
Start the design work of task 2 The main problem User interactive may be needed The result of task 1 can be
served as the building block
Reference
[AGAR06] A.Agarwala, M.Agrawala, M.Cohen and D.Salesin, R.Szeliski, Photographing long scenes with multi-viewpoint panoramas, ACM Transactions on Graphics, 25(3), 853 - 861, 2006.
[GORT96] S.J.Gortler, R.Grzeszczuk, R. Szeliski and M.F. Cohen, The Lumigraph, The Journal of Computer Graphics, 30, 43--54, 1996.
[KUMA95] R.Kumar, P.Anandan, M.Irani, J.Bergen and K.Hanna, Representation of scenes from collections of images, Proceedings of IEEE Workshop on Representation of Visual Scenes, 10-17 ,1995
[PELE00] S.Peleg, B.Rousso, A.Rav-Acha and A.Zomet, Mosaicing on Adaptive Manifold, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1144-1154, 2000.
[SHUM99] H.Y.Shum and L.W.He, Rendering with concentric mosaics, The Journal of Computer Graphics, 33, 299--306, 1999.
[ZHEN03] J.Y.Zheng, Digital route panoramas, IEEE Multimedia, 10(3), 57- 67, 2003.
[ZOME03] A.Zomet, D.Feldman, S.Peleg and D.Weinshall, Mosaicing New Views: The Crossed-Slits Projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 741- 754, 2003.