View
223
Download
1
Embed Size (px)
Citation preview
Seamless Switching Seamless Switching of Scalable Video Biof Scalable Video Bitstreams for Efficietstreams for Efficie
nt Streamingnt StreamingXiaoyan Sun, Feng Wu, ShipeXiaoyan Sun, Feng Wu, Shipeng Li, Wen, Gao, and Ya-Qin Zng Li, Wen, Gao, and Ya-Qin Z
hanghang
OutlineOutline• Introduction• Seamless Switching Scheme Among Scalable
Bitstreams• Switching Down Between Two Bitstreams• Switching Up Between Two Bitstreams• Experimental Results• Discussions
IntroductionIntroduction• Why are scalable bitstreams needed? Switchin
g between non-scalable bitstreams– Drifting error
– Key frames
– Large storage requirements• Multiple non-scalable bitstreams• Key frames
Non-scalable bitstream1
Non-scalable bitstream2
drifting error
Non-scalable bitstream1
Non-scalable bitstream2
key frame
IntroductionIntroduction• SP-frames scheme (proposed in H.264)
– Extra bitstreams are used.– The size of S12 is similar to that of I-frame.– When switching down, S12 may deteriorate the net
works.– http://vc.cs.nthu.edu.tw/ezLMS/show.php?id=249
P S2 P
P S1 P
S12
bitstream1
bitstream2
SP frame
extra bitstream
IntroductionIntroduction• Why are multiple scalable bitstreams ne
eded? Single scalable bitstream– FGS
– Low coding efficiency in FGS bitstreams.• Motion prediction is based on the lowest quality
base layer.– Using multiple scalable bitstreams, coding e
fficiency is higher.
Base layer
Enhancement layer 1Enhancement layer 2
Enhancement layer 3
…
Base layer
Enhancement layer 1Enhancement layer 2
Enhancement layer 3
…
Frame n Frame n+1
Seamless Switching Scheme ASeamless Switching Scheme Among Scalable Bitstreamsmong Scalable Bitstreams
• Seamless switching (Definition)– The quality in each scalable bitstream is sm
ooth.– The switching among scalable bitstreams is
drifting-free.– Immediately switching from the current scal
able bitstream to one operated at lower rates without any delay.
Seamless Switching Scheme ASeamless Switching Scheme Among Scalable Bitstreamsmong Scalable Bitstreams
ScalableBitstream 1
ScalableBitstream 2
ScalableBitstream 1
ScalableBitstream 3
Bandwidth
Time
ScalableBitstream 1
ScalableBitstream 2
ScalableBitstream 3
Seamless Switching Scheme ASeamless Switching Scheme Among Scalable Bitstreamsmong Scalable Bitstreams
switching
SF switching frame (switching-up point)
High-bit-rate scalable bitstream
Low-bit-rate scalable bitstream
Switching Down Between Two Switching Down Between Two Bitstreams Bitstreams
• MVs are estimated in SB-H and are applied to both SB-H and SB-L.
• The quantization information of SB-L is coded in SB-H bitstream.– Overhead bits: 3*MB_number+5
• Instead of original frames, base layer frames in SB-H are used to encode base layer frames in SB-L.
First QPRange of difference
☆Switching down at everywhere and with no overhead
Switching Down Between Two Switching Down Between Two BitstreamsBitstreams
• SF Decoder
VLD Q-1 IDCT
MC
MC
DCTQL-1 QLIDCT
ref SB-L refBase
SB-HBase layer
SB-LBase layer
Video
MVs
Switching Down Between Two Switching Down Between Two BitstreamsBitstreams
• Quality loss of SB-L base layer– The reconstructed frames from SB-H base la
yer instead of the original video are used as the input of the SB-L base layer.
– The same set of MVs for SB-H base layer is used to encode the SB-L base layer.
Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams
n-1 n n+1
Switchingpoint
SB-LBase layer
SB-HBase layer
+
=Extra bitstream
☆Considerable amount of overhead bits
Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams
• SF Encoder
IDCT
Qs-1
DCT
Qs
BP VLC
DCT Qs
SB-HBase layer
SB-LBase layer
Extra bitstream
SF framen-1 n+1
n
☆n’ is used instead of n to avoid drifting error
= SB-L Base + Extra bitstream
n’n
Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams
• SF Decoder
VLD Q-1 IDCT
MC
refBase
SB-HBase layer
SB-LBase layer
VideoMVs
DCTQs-1 QsIDCT
BPVLD
Extrabitstream
Switching Up Between Two BitSwitching Up Between Two Bitstreamsstreams
• The SF frame is simpler than the SP frame.
• The SF frame gets better quality than the SP frame.
Experimental ResultsExperimental Results• SF scheme is applied on MBPFGS
– MBPFGS is modified from PFGS• Frame base -> Macroblock base•
–
– 10fps• An I-frame is inserted every 1’s in a non-scalable bit
stream • An SF frame is inserted every 1’s in a SF bitstream
32kbps64kbps80kbps112kbps
http://vc.cs.nthu.edu.tw/home/paper/codfiles/hschen/200305141532/pfgs.PPT
MBFGS-H bitstream
MBFGS-L bitstream
Experimental ResultsExperimental Results• Comparisons of extra bitstream
–
– Average PSNR loss caused by SF frames < 0.1 dB
Sequence SF (Qs=3) Lossless
News 32959.11 147878.1
Foreman 34918.33 173569.3
Coastguard 39824.78 158157.7 (Bits)
Experimental ResultsExperimental Results• Rate-distortion (static)
– Non-scalable bitstream (JVT H.26L codec)• Two bitstream with different quality• Switching at I-frame
– FGS and MPFGS• Only one bitstream
Non-scalable
FGS
MPFGS
SF
Inaccurate MVs and input video
Experimental ResultsExperimental Results• PSNR comparisons (dynamic
channel)– Foreman
– Coastguard
72 kbps
152 kbps
72 kbps
152 kbps
DiscussionsDiscussions• The SF scheme can be extended to the case with multi
ple scalable bitstreams.– (N-1) extra bitstreams are required.
• N*(N-1) extra bitstreams are required in SP frames.– High complexity of the decoder in decoding the high-bit-rate
scalable bitstream.• Three times of the MPEG-4 decoder.• Complexity scalability.
– Large size of the overhead bits inside the high bit-rate bitstream.
• The extra bitstream for switching up is large.– if the bitstreams switch up and down once for every second, t
he extra bitstream will cost 30-40 kbps.– An intelligent server should be able to control switching time
s.