Upload
dyani
View
46
Download
4
Tags:
Embed Size (px)
DESCRIPTION
清华信息科学与技术国家实验室 普适计算研究部. 交互多视点视频 ( Interactive Multiview Video ). Supported by NSFC, 863,CNGI. Introduction. 交互多视点视频( Interactive Multiview Video ): - PowerPoint PPT Presentation
Citation preview
Details: Multiview video synthesis Motion vector based video objects segmentation and tracking Feature based incomplete 3D structure video synthesizing Layered depth images based multiview video rendering
Semi-immersive MultiView display environment High efficiency Multiview Video Coding Disparity based multiview video coding View dependent multiview video predicting codec
Interactive multiview video streaming View switch
Time frozen moment Time continuous sweep
交互多视点视频 (Interactive Multiview Video )
Introduction
交互多视点视频( Interactive Multiview Video ): The next generation multimedia applications will be interactive and realistic based on network. In one hand, people want to be more active and participant to meet the requirement of personality, in the other hand, they hope to get realistic experience while viewing the scene or interacting with other people. This project is proposed to resolve the represent, interaction, codec and transmission of 3D interactive video, and implement the interactive network multimedia application in the near future with the stereo video and multi-view video technology putting into practicability. Objects: Viewpoint selecting and multi-view 3D image synthesizing High efficiency multi-view video codec algorithms Streaming technology for Multi-view video A prototype of multi-view video streaming system
清华信息科学与技术国家实验室 清华信息科学与技术国家实验室 普适计算研究部普适计算研究部
feature point matching triangulation
virtual video synthesizing by interpolation
feature point matching triangulation
virtual video synthesizing by interpolation
Left - imageLeft - image
Right - imageRight - image
DisparityMap
DisparityMap MPEG - 4
Encoder
MPEG - 4Encoder
MPEG - 4Decoder
MPEG - 4Decoder
ReconstructedRight - image
ReconstructedRight - image
PSNR
+
Residual Texture
Residual Texture
-
ReconstructedResidual Texture
ReconstructedResidual Texture
DisparityEstimation
DisparityEstimation
Local -MPEG - 4Decoder
Local -MPEG - 4DecoderReconstructed
left- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Reconstructedleft- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Encoder side
Decoder side
Coded left image
Coded Disparity map
Coded right imageLeft - imageLeft - image
Right - imageRight - image
DisparityMap
DisparityMap MPEG - 4
Encoder
MPEG - 4Encoder
MPEG - 4Decoder
MPEG - 4Decoder
ReconstructedRight - image
ReconstructedRight - image
PSNR
+
Residual Texture
Residual Texture
-
ReconstructedResidual Texture
ReconstructedResidual Texture
DisparityEstimation
DisparityEstimation
Local -MPEG - 4Decoder
Local -MPEG - 4DecoderReconstructed
left- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Reconstructedleft- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Encoder side
Decoder side
Coded left image
Coded Disparity map
Coded right image
Rate AllocationRate Allocation
Region Segmentation Region Segmentation
Multiview Video Coding with Region based Rate Allocation
Left - imageLeft - image
Right - imageRight - image
DisparityMap
DisparityMap MPEG - 4
Encoder
MPEG - 4Encoder
MPEG - 4Decoder
MPEG - 4Decoder
ReconstructedRight - image
ReconstructedRight - image
PSNR
+
Residual Texture
Residual Texture
-
ReconstructedResidual Texture
ReconstructedResidual Texture
DisparityEstimation
DisparityEstimation
Local -MPEG - 4Decoder
Local -MPEG - 4DecoderReconstructed
left- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Reconstructedleft- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Encoder side
Decoder side
Coded left image
Coded Disparity map
Coded right imageLeft - imageLeft - image
Right - imageRight - image
DisparityMap
DisparityMap MPEG - 4
Encoder
MPEG - 4Encoder
MPEG - 4Decoder
MPEG - 4Decoder
ReconstructedRight - image
ReconstructedRight - image
PSNR
+
Residual Texture
Residual Texture
-
ReconstructedResidual Texture
ReconstructedResidual Texture
DisparityEstimation
DisparityEstimation
Local -MPEG - 4Decoder
Local -MPEG - 4DecoderReconstructed
left- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Reconstructedleft- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Encoder side
Decoder side
Coded left image
Coded Disparity map
Coded right image
Rate AllocationRate Allocation
Region Segmentation Region Segmentation
Left - imageLeft - image
Right - imageRight - image
DisparityMap
DisparityMap MPEG - 4
Encoder
MPEG - 4Encoder
MPEG - 4Decoder
MPEG - 4Decoder
ReconstructedRight - image
ReconstructedRight - image
PSNR
+
Residual Texture
Residual Texture
-
ReconstructedResidual Texture
ReconstructedResidual Texture
DisparityEstimation
DisparityEstimation
Local -MPEG - 4Decoder
Local -MPEG - 4DecoderReconstructed
left- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Reconstructedleft- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Encoder side
Decoder side
Coded left image
Coded Disparity map
Coded right imageLeft - imageLeft - image
Right - imageRight - image
DisparityMap
DisparityMap MPEG - 4
Encoder
MPEG - 4Encoder
MPEG - 4Decoder
MPEG - 4Decoder
ReconstructedRight - image
ReconstructedRight - image
PSNR
+
Residual Texture
Residual Texture
-
ReconstructedResidual Texture
ReconstructedResidual Texture
DisparityEstimation
DisparityEstimation
Local -MPEG - 4Decoder
Local -MPEG - 4DecoderReconstructed
left- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Reconstructedleft- image
Reconstructedleft- image
ReconstructedDisparity Map
ReconstructedDisparity Map
CompensatedRight - image
CompensatedRight - image
+
Encoder side
Decoder side
Coded left image
Coded Disparity map
Coded right image
Rate AllocationRate Allocation
Region Segmentation Region Segmentation
Multiview Video Coding with Region based Rate Allocation
video stream 1
video stream n
video stream 2
video stream 3
View switching position
View
Time
View Group 1
PP P PP
PP P PP
PP I PP
…
…
…
…
…
PP P PP
…
…
…
…
…
Supported by NSFC, 863,CNGI