8/12/2019 05157039

1/4

The 13th International Symposium on Consumer Electronics ISCE2009)

Efficienct Rate-control Algorithm for Multi-ViewVideo CodingSeanae Park

Department of Computer EngineeringKwangwoon UniversitySeoul, KoreapseaII18@kw.ac.kr

Donggyu SimDepartment ofComputer EngineeringKwangwoon UniversitySeoul, Koreadgsim@kw.ac.kr

IDlE

Figure 1 A referencing structure ofMV

II. MVC CODING STURCTURE

jI Stofedpirnue

view video. In this paper, we propose a view-level rate controlalgorithm to control rates of all the views in total. In theproposed algorithm, bitrate of each view can be regulated andtotal bitrate of all the views at the same time can be kept atarget bitrate by accounting for bitrates from bi-directionalframes.

In Section 2 we introduce the multi-view video coding andits referencing structure. The proposed rate-control algorithmfor multi-view video is presented in Section 3 Experimentalresults and conclusion are given in Section 4 and 5respectively

Multi-view video can offer users 3D effect with multiplevideos. It can lead us to feel realism. Multi-view video codingtechnology was developed to compress the multiple videos. InJanuary 2006, the MVC framework was decided in theBangkok MPEG meeting. Finally, the basic framework wasadopted as the International Standard as the amendment ofH.264/AVC. In prior MVC research works, improvement ofcoding efficiency has been focused. Figure 1 shows areferencing structure of the MVC. It employs hierarchical Bpictures in time and view axes. However, we need to transmitcompressed video streams over channels which have limitedbandwidth. Especially, MVC requires a large amount ofbandwidth in proportion to the numberofviews [4].

Keywords component; Rate control algorithm multi view videocodingI INTRODUCTION

A rate control in video coding is one of the most importantparts because of the limitation of channel bandwidth inpractical applications. Practical video coders have severalconstraints such as limited buffer size in a decoder and limitedchannel bandwidth in transmission channels. For single viewvideo, many rate control algorithms have been proposed. Ingeneral, they consist of GOP-, picture-, and MB-Ievel controlstages [1][2]. However, multi-view videos have multipleframes at the same scene. In MVC international standard, MVCmakes use of cross referencing between adjacent views forcoding efficiency [3]. So, if we want to apply a rate-controlalgorithm to MVC, we could control bitrate of each viewor/and total bitrate for all the views. Note that MVC hasmultiple B views which are composed of only B frames andother views also have many B frames. We need to use QPvalues of B frames in computing the initial QP of a GOP forMVC rate control, while existing rate-control methods forsingle-view videos do not use statistics of coding parameters ofB frames. It enables us to accurately control the rate of multi-

bstract n this paper , we propose an efficient rate-controlalgorithm which considers characteristics of multi-view videocoding MVC . A rate control algorithm is one of the mostimportant elements of video coding. There are many existingalgorithms to control the bit rate but they are designed for singleview video coding. The rate control algorithm for MVC should bealso conducted at view level. Conventional rate-controlalgorithms for single view videos do not use QP values of Bframes in computing a GOP initial QP. However, we have to takeinto account QP values of B frames because the MVC has manyB views which are composed of only B frames. n addition, otherviews also have many B frames because the MVC employ thehierarchical B for coding efficiency. n the proposed algorithm,initial QP of GOP should be computed with QP values of Bframes. n our research, the proposed algorithm can allocateproper bitrates for B views which are composed of only B framesand QP values of B frames can affect the initial QP ofGOP. Theproposed method can control total bitrate of multiple views to beacquired and displayed at the same time. n our experimentation,we found that the proposed method can control bitrates for allthe views according to target bitrates

978-1-4244-2976-9/09/ 25.00 2009 115

8/12/2019 05157039

2/4

III. PROPOSED RATECONTROL ALGORITHM

In this paper, we propose a rate control algorithm which isan extension of H.264/AVC rate-control algorithm inJMI2.0.[I]. The proposed rate-control algorithm is developedby considering view-referencing structure in the MVC. It isalso based on the second-order modeling between an amount ofgenerated bits and a quantization parameter which are used forconventional rate control algorithms. The proposed algorithmalso consists of three steps. The first step is view level ratecontrol, the second is GOP-level rate control, the third is framelevel rate control, and the last is MB- level rate control. All thesteps should be conducted for each view.

A. View level rate con trolIn the view-level, a target bit for each view can becomputed with the number of view and weight of each view. InMVC, each view could have different priorities. Weight factorscan be set by users, depending on application or priority ofview. In the proposed method, the target bit for one view iscomputed by

average QP is computed in GOP. As shown in figure I, MVCreferencing structure has views which consist of onlyhierarchical B pictures and other views also have a largenumber of hierarchical B pictures. We cannot help accountingfor bits from B pictures. In this paper, QP values ofB picturesare used for computing an average QP in the pre-GOP. If aGOP is the first one in video sequence, we cannot compute anaverage QP in the pre-GOP stage.

3

where j is a picture index in GOP, f is a frame rate, k is theview index indicating a view among multiple views. N is thenumber of pictures in a GOP, R represents bitrate, and V isremaining bit in an encoding buffer. A computed target bit byequation 3 is used in the picture level rate control.

Picture level rate controlAs shown in figure I, hierarchical B pictures which areregarded as stored pictures are exploited for MVC. In thehierarchical B picture structure, quality of pictures in the higherlevels in hierarchical levels depends on that of pictures in thelower levels. Therefore, the picture-level QP values arecomputed based on the hierarchy level in the hierarchical

structure. The picture-level QP is computedby

1t ~ t a l _ R x w l = 1R l = vlew_N Itatal R . - xw l -{R l-l -R l-l } 1=2,3,...,Nvlew_N

where, is the coding order index, and total_R is total bits forGroup ofGOP GGOP that is composed of multiple Gaps formulti-views. The view N is the number of views and w isweighting factor for view. R / is the target bit for t h viewand / is the using bit for t h view encoding. k { p ~ e v if Border =0Q B ~ r k QPr v Hlevel - L Border otherwise 6

B. GOP level rate controlIn this GOP-level, the total number of bits allocated to eachGOP is computed and the initial quantization parameter QP

of each GOP is set. In the proposed method, an initial QP iscomputed with average QP in pre-GOP, it is computed by

Qpk =max{QPok _ 2 min{Qpk 1 2 S;BQP i -1 min 2, N l 5 ~ }} I I I I i 1

2

If the level of a B picture is 0, we select a large value betweenQP values of I and P pictures in the pre-level.

D. Macro-block level rate controlA MB-Ievel rate control has two steps. First, a target bit iscomputed for encoding a macro-block, and MB-Ievel QP is setfor target bit. For target bit, we account for a bit for predictionerror and header data. A bit for error data is computed based oncomplexity of current macro-block. The complexity iscalculated by

where, i is an index of GOP and k is an index of view. TheSPBQP i-l is QP of P and B pictures in the i_Iyh GOP, andNpB i-l is the number of Band P pictures in the i_Iyh GOP.Conventional rate control algorithm of H.264/AVC for singleview does not account for the number of B pictures when the

116

6

8/12/2019 05157039

3/4

where 5 ; j is a mean of absolute difference between currentblock and its corresponding block. The bit for a header data iscalculated by using the average bit of a pre-encoding macroblock. At the last step, we use a RD model and computed targetbit, we can compute a suitable QP value.

result, they even have coding gain for fixed QP at some points.But, subjective quality for proposed algorithm is not good. So,we need to study more to improve subjective quality andcoding gain for other points.

V. CONCLUSIONIn this paper, we propose for rate control algorithm formulti-view video coding MVC). The proposed method hastwo main issues. The first one is view level rate-controlalgorithm. The MVC has multi-view and these can becontrolled in each view and also all view for application.Because, some application need good quality for some specialview and some application need equalized quality for all views.So, our proposed algorithm has not conventional rate controllevel but also view level. Second issue is an influence of Bpicture QP. Conventional rate-control algorithms for singleview videos do not use QP values of B frames in computing a

GOP initial QP. However, we have to take into account QPvalues of B frames because the MVC has many B views whichare composed of only B frames. The proposed method cancontrol total bitrate of multiple views to be acquired anddisplayed at the same time. In our experimentation, we foundthat the proposed method can control bitrates for all the viewsaccording to target bitrates. We found that a certain amount ofcoding gain have been achieved for a few sequences. However,we need to study more to improve coding gain and subjectivequality in 3D.

d dABLE II Target It an generate mean ItTarget bit Target bit Target bitenerated enerated eneratedaverage bits average bits average bits256kbps 384kbps 512kbpsBallroom 270.5kbps 393.5kbps 550.9kbps192 kbps 256 kbps 384 kbpsExit 205.2kbps 264.5kbps 393.8kbps768 kbps 1536 kbps 2048 kbpsUli 789.6kbps 1581.9kbps 2059.8kbps384 kbps 512 kbps 768 kbpsRace1 392.lkbps 520.lkbps 773.3 kbps256 kbps 384 kbps 512 kbpsFlamenco2 261.2 kbps 390.5 kbps 517.8 kbps256 kbps 512 kbps 1024 kbpsBreakdancers 266.0 kbps 570.7 kbps 1 31 kbps128 kbps 256 kbps 512 kbpsRena 132.3 kbps 259.9 kbps 519.2 kbps192 kbps 384 kbps 768 kbps

kko Kayo 198.8 kbps 392 kbps 773.9 kbps

TABLE I Initial QP and target bitTarget bit Target bit Target bitnitial nitial nitial 256kbps 384kbps 512kbpsBallroom 37 32 27192 kbps 256 kbps 384 kbpsExit 31 29 26768 kbps 1536 kbps 2048 kbpsUli 36 30 28384 kbps 512 kbps 768 kbpsRace1 28 26 24256 kbps 384 kbps 512 kbpsFlamenco2 34 30 28256 kbps 512 kbps 1024 kbpsBreakdancers 31 26 22128 kbps 256 kbps 512 kbpsRena 33 28 23192 kbps 384 kbps 768 kbpsAkko Kayo 36 29 24

IV. EXPERIMENTAL RESULTIn this paper, we implemented the proposed method inJMVM 3.0[5]. The rate control implementation in thiscontribution is based on the rate control scheme that wasadopted within the MPEG-4 AVC Joint Model 12.2JMI2.2)[6] reference software. Because the rate controlimplemented in JM12.2 supports the hierarchical B pictures,we applied it to the multi-view coding for each view. Wemodified several steps to determine initial QP and QPparameters for hierarchical B pictures.We followed all the conditions described in the coreexperiment document for MVC. In this experiment, we settarget bitrates to the same value for all views for a sequence.Table 1 shows the target bitrate and init ial QP values for eachsequence [7].

Table 2 shows a target bit and generated mean bit forseveral QP values. As shown in Table 1 the generated bits areclose to the target bits.In the figure 2, we show RD-curves of the rate-controlledJMVM and fixed-QP JMVM. We found that a certain amount

of coding gain have been achieved for a few sequences. In our

117

8/12/2019 05157039

4/4

Rena47.0

45.0

_43110:2ci 4 1

IlL

C 39.0

37.0

15 0 250 350 450 550Avg Rate kbps)

65 0 75 0

(d)Figure 2 R-D performance graph for proposed algorithm (a) Ballroom (b)Akko Kayo (c) Breakdancers (d) Rena

References

AcknowledgmentThis work was supported by the Seoul R BD Program of theSeoul City Government through the Seoul DevelopmentInstitute.

[I] Gary J Sullivan, Pankaj Topiwala, and Ajay Luthra, The H.264/AVCadvanced video coding standard: overview and introduction to the fidelityrange extension, Applications of Digital Image Processing XXVII 5558 454-474 (2004)[2] Join t Video Team NT ) ISO/IEC MPEG ITU-T VCEG, Adaptivebasic unit layer rate control for NT, NT O12, Mar. 2003.[3] Joint Video Team NT ) ISO/IEC MPEG ITU-T VCEG,, Joint VideoTeam ofISO/IEC MPEG and ITU-T VCEG, JVT-K049, Mar. 2004.[4] Joint Video Team NT ) ISO/IEC MPEG ITU-T VCEG, Rate controlreorganization in the JM (Joint Model) reference software, Joint Video TeamofISO/IEC MPEG and ITU-T VCEG, NT-W042, Apr. 2007. ] NT MVC Reference Software version JMVM 3.0.[6]JVT reference software version JM12.2,[7] ISO/IEC JTCl/SC29/WGII Requirements on multi-view video coding,N6501 (2004)

120000 0

Breakdancers

400 600 800Avg Rate kbps)

20 0

41.0

40.0

39 0: t38 .0a::z It

~ 3 7>36.0

35.0

34.00

c)