Tammam Tillo, Macro Grangetto, and Gabriella Olmo

CSVT, Jan 2008

1

Introduction◦ Multiple description coding◦ Redundant slice

Proposed Algorithm Optimal Allocation Problem Algorithm Implementation Experimental Results

2

Multiple Description Coding (MDC)◦ Single source Multiple Descriptions

◦ D0 < D1, D2

◦ Rsingle_description = R1 + R2 – Rredundancy

Encoder 1

Encoder 2

Central decoder

Side decoder 1

Side decoder 2

Channel I

Channel 2

R1

R2

D1

D2

D0S

S’1

S’2

S’3

Introduced by MD coding 3

I I2i+1

2i

Example

Redundant slice in H.264◦ A slice can be encoded and transmitted twice Called primary slice and redundant slice

◦ QPp < QPr ◦ The redundant slice is used at the decoder side if

the primary slice is lost

QPr

QPp Primary slice

redundant slice

Encoder Decoder

4

MDC with redundant slices

◦ Simple post and pre-processing + H.264 decoder

I P P I

Primary Slice

Redundant Slice

Source

I P P I Description 1

Description 2

5

Redundant slice◦ QPr > QPp

◦ Not for reference◦ Interlaced with primary slices◦ Drift problem◦ QPr drift redundancy◦ QPr drift (under loss rate p) Dtotal = (1-p)dprimary_slice + p(1-p)(dredundant_slice+dpropagation) + p2dall_loss Rtotal = Rprimary_slice(QPp) + Rredundant_slice(QPr)

6

Impact of loss of a primary slice k on the total distortion of the current GOP with size N ◦

N

ijkmkrkt jddd

1.,, ][Distortion caused by

loss of primary slice k

Distortion of redundant slice k at frame i

Propagated distortion cause by redundant slice k after frame i

...

i i+1

Loss occurs

k

N

7

Decay of propagated distortion [22]◦

dm,k = dr,k -dp,k dr,k when p,k << r,k

◦ f[n]: Power transfer function Representation of distortion attenuation Reasons

Spatial filter such as de-blocking filter Spatial interpolation such as fractional pel ME Intra blocks such as random intra replacement

iN

nkm

N

ijkm nfdjd

1,

1. ][][

]1[,, idd kmkm

...

i i+1

krd , kmd , ]1[, fd km ][, iNfd km

[22] N. Farber, K. Stuhlmuller, and B. Girod, “Analysis of error propagation in hybrid video coding with application to error resilience,” in ICIP, 1999. 8

Evaluation of dm,k ◦

◦ Aligned quantizer

◦

deefeeEd km )(}{ 22,

e: Difference between the primary and redundant reconstruction (drift)f(e): pdf of e

n

jkpkp

kr

kp

kr

kp jejeeef1

,,,

,

,

, )]()([)()(

: Delta function

r

pe

Primary reconstruction

Redundant reconstruction

Source x

Source x

e

f(e)

)( prp je

p

)( prp je e = 0,p, 2p,…f(e) = p/r at p|e

9

Evaluation of dm,k ◦

◦ p,k << r,k dm,k dr,k

2

,

,2

,

1

2

,

3,

1,,

2

,

,2,

112

)]()([)(

kr

kpkr

n

jkr

kp

n

jkpkp

kr

kpkm

jc

dejejeedeefed

Assumption:r,k/p,k = 2n+1dr,k = r,k

2/12

10

Evaluation of dm,k ◦ Not aligned quantizer

◦

r

p e

Reconstructed level

Reconstructed level

Source x

Source x

e

f(e)

r

r1

12

1)(

2,

2/

2/ ,

22,

,

,

kr

krkm

kr

kr

deedeefed

e can be any numberf(e) = 1/r

11

Accuracy of evaluated dm,k

Real distortion – evaluated dm,kReal distortion

12

Total GOP distortion dt,k ◦

Expected GOP distortion caused by loss of slice k at loss rate p◦

Expected GOP distortion caused by random loss of a slice◦

ikr

iN

nkr

iN

nkrkr

iN

nkmkrkt dnfdnfddnfddd .

0.

1.,

1.,, ][][][

ktkp

kktkpk

dppdp

dpdppdpd

,,

,02

,,

)1()1(

)1()1(

N

i

N

iiirip

kk DpDpdD

1 1,,)1(

f[0] = 1

iN

ni nf

0

][

low loss rate

Loss of primary sliceUsing primary slice Loss of primary and redundant slice

ik

krir dD ,,

ik

kpip dD ,,

Expected frame distortion

13

Optimization problem◦

◦ Lagrangian approach: min J = D + R ◦

Dmin

GOP1

,, )( subject to RRRN

iirip

0)1(

0)1(

,

,

.

,

,

.

ir

iri

ir

ip

ip

ip

R

Dpp

R

J

R

Dp

R

J

ir

iri

ip

ip

R

Dp

R

D

,

,

,

,

1,

1,1

,

,

1,

1,

1,

1,

r

r

iir

ir

r

ri

p

p

p

p

R

D

R

D

R

Dp

R

D

R

D

Since Dp,i/ Rp,i is independent of i

14

Optimal conditions◦

Redundant data < primary data i > i+1

1,

1,1

,

,

1,

1,

r

r

iir

ir

r

ri

p

p

R

D

R

D

R

Dp

R

D

15

Encoder

Decoder

QPp QPr,i

Post processing

Channel 1

Channel 2

Channel 1

Channel 2

Preprocessing

Merge

H.264 Standard decoding

Redundant slice

16

Determination of RD parameters◦

◦ H.264 reference software: QPp = QPr,1 + 3log(p1)

QPr,i = QPr,1 + 3log(1/i)

◦ In [22], assuming , deducing

Taking

3

12

285.0QP

RD1,

1,1

,

,

1,

1,

r

r

iir

ir

r

ri

p

p

R

D

R

D

R

Dp

R

D

-

nenf ][

yxyxuuyxtv ddHt ),(),(4

1][

2

22

22

1

1][ uv t

t

Variance of drift v[x,y,t]

Power spectral density of u[x,

y]= v[x,y,0]

Linear system (decoder)

)1()1( )1( ee iNi

Small for serious packet loss17

(QPi) = QPr,i – QPp ◦ i QPr,i◦ p QPr,i◦ QPr,i

18

Parameters◦ H.264 reference software JM9.4◦ P-slice with 5 reference pictures◦ 5 slices for a CIF picture and 3 slices for a QCIF

picture◦ Bernoulli model for PLR = p ◦ foreman and coastguard at CIF and QCIF format

are used

19

Selection of ◦ Not sensitive to

20

RD comparisons under different p and N (=0.4)◦ Shorter GOP size for larger PLR

p=0.01p=0.05

p=0.1

N=11

N=21N=45

21

Redundancy (=0.4)◦

i irip

i ir

RR

R

)( ,,

,

p

p

R

D

1,

1,

r

r

R

D

22

Comparisons with other MDC◦ Four descriptions by subsampling CIF to QCIF◦ Two descriptions by separating odd and even

rows Other MDC schemes:N = 21 with P and B picturesExtra computation is needed at the decoder side

23

Central decoder versus side decoder◦ p redundancy (central decoder) and (side

decoder)

No loss

24

Central decoder versus side decoder (p=0.05)

25