Optimum Bit Optimum Bit Allocation and Allocation and

Rate Control for Rate Control for H.264/AVCH.264/AVC

Wu Yuan, Shouxun Lin, YongdWu Yuan, Shouxun Lin, Yongdong Zhang, Wen Yuan, and Haong Zhang, Wen Yuan, and Haiyong Luoiyong LuoCSVT 2006

OutlineOutline• Introduction• Rate-Distortion Modeling• Rate Distortion Optimization• Macroblock-Layer Rate Control• Experimental Results

– Comparisons with JVT-G012

Introduction (1)Introduction (1)• H.264/AVC Rate Controller

http://www.pixeltools.com/rate_control_paper.html#bas

Intra/Interprediction

Residual

Introduction (2)Introduction (2)• The Chicken and Egg Dilemma

Rate Control

QP

RDO

MAD

Coding

(Intra/Inter mode selection)

!?

(Residual calculation)

Solution: Guess!MAD = MADprev +

MADprev

MAD

Rate-Distortion Modeling (1)Rate-Distortion Modeling (1)• Overhead Bit-Rate Prediction

– Overhead bits: QP, MV, MB mode, …– Using history

• Coding Complexity Prediction– Complexity: Residual (MAD)– Using history

• R-D Behavior Prediction– Using overhead bits, residual, and initial QP

• Distortion Prediction– Using history

Residual(org - pred)

Overhead(MV, QP, …)

Encoded frame

Entropycoding

Rate-Distortion Modeling (2)Rate-Distortion Modeling (2)• Overhead Bit-Rate Prediction

– JVT-G012: Hi = Hiave

– Proposed: Hi = Hiprev

n-1 n

Rate-Distortion Modeling (3)Rate-Distortion Modeling (3)• Coding Complexity Prediction

– MAD = MADprev + • Data points are first selected by the spatial and temporal distance, and then removing outliers.

%100MAD

|MAD-MAD|

pred

realpred

JVT-G012

n-1 n

RPE < 3%QP JVT-G012 Proposed32 20.58% 23.05%34 47.98% 58.12%

Rate-Distortion Modeling (4)Rate-Distortion Modeling (4)• R-D Behavior Prediction

– Assumption: DCT coefficients of residual can be approximated by

– Distortion

xexP

x

where,2

)(

||

(Laplace distribution)||),( xxxxD

DD

DR 0 where,)ln()(

* A. Viterbi and J. Omura, Principles of Digital Comuunicatin and Coding. New York: McGraw-Hill Electrical Engineering Series,1979

*

)()1(21)1()( 3

2 QstepRQstepQstep

QstepR

2

2MADMAD)(Qstepb

QstepaQstepR

Taylor expansion

Residual rate

Let /D = 1+x

Rate-Distortion Modeling (5)Rate-Distortion Modeling (5)• R-D Behavior Prediction

– Taylor expansion of R(Qstep) at Qstepave

– – –

• Rate of the ith MB:

QPmaxQPaveQPmin)()(

2)("

)()(')()(~

ave32

aveave

aveaveave

QstepQstepRQstepQstepQstepRQstepQstepQstepRQstepRQstepR

CQstepBQstepAQstepR 2)(

previiiiiii HCQstepBQstepAR 2

2)(" aveQstepRA

aveaveave )(")(' QstepQstepRQstepRB

ave2aveave)( QstepBQstepAQstepRC

(Rate of residual)

Rate of residual Rate of overhead

Rate-Distortion Modeling (6)Rate-Distortion Modeling (6)• Distortion Prediction

– Assumption 1: Distortion of DCT coefficients is uniformly distributed• D = 2 = |x-y|2/12 (Qstep2/12)

– Assumption 2: Qstepi Qstepiprev

• Qstepi (Diprev/Qstepi

prev)

• Di Qstepi2 (Qstepi Di

prev/Qstepiprev)

Di = iQstepi, where i = Diprev/Qstepi

prev

Set as 1

pred

real

DD

(For scalability)

Rate Distortion OptimizationRate Distortion Optimization• Consider Di = iQstepi for the ith MB

– subject to

N

iiiQstepQstep

QstepNN 1

,...,

1min1

TTN

ii

1

][1min11

,,...,1

N

ii

N

iiiQstepQstep

TTQstepNN

])([1min1

prev2

1,,...,1

N

iiiiiii

N

iiiQstepQstep

THCQstepBQstepAQstepNN

N

ki

k

N

k

N

k kk

kN

k k

i

i

i

ii

A

CA

BHT

AABQstep

1

2

1 1

2

1prev

* 42

(Lagrange theory)

Rate of the ith MB

Macroblock-Layer Rate Macroblock-Layer Rate Control (1)Control (1)

Macroblock-Layer Rate ControMacroblock-Layer Rate Control (2)l (2)

Macroblock-Layer Rate ControMacroblock-Layer Rate Control (3)l (3)• Rate Controller

1. Initialization: QPave = QPstart, QPmin = max(QPave-3, 0), and QPmax = min(QPave+3, 51). Let i = 0

2. Optimum Bit Allocation for ith MB:1) k = i.2) R-D modeling:

3) Optimum Computations:

4) k = k+1. If k N, jump to 2).5) Compute optimal QPi

*:

prevMADMAD kk

2max

2

max

kmax

MADMAD)(Qstepb

QstepaQstepR kkk

kH

N

ki

k

N

k

N

k kk

kN

k k

i

i

i

ii

A

CA

BHT

AABQstep

1

2

1 1

2

1prev

* 42

2)(" aveQstepRAk aveaveave )(")(' QstepQstepRQstepRBk

ave2aveave)( QstepBQstepAQstepRC kkk kkk CQstepBQstepAQstepR 2)(

(overhead bits)

(check if > T when adding with Hk)

2max

2

maxmax

MADMAD)(Qstepb

QstepaQstepR kkkk

(by linear regression)

(by linear regression)

Macroblock-Layer Rate ControMacroblock-Layer Rate Control (4)l (4)• Rate Controller

3. Adjust QPi*: QPi

* = max{QPi-1*-1, min{QPi

*,QPi-1*+1}}. Then QPi

* = max{1,QPave-3, min{51,QPave+3,QPi

*}}4. Encoding5. Update

(Reducing blocking effect)

(For smoothness)

Experimental Results (1)Experimental Results (1)CIF: Mobile, Paris

%100rateTarget

rate RealRate Prediction Error Ratio:

Experimental Results (2)Experimental Results (2)

Experimental Results (3)Experimental Results (3)