Multimedia Over IP Networks -- III

CS335 Principles of Multimedia Systems

Multimedia Over IP Networks -- III

Hao Jiang

Computer Science Department

Boston College

Nov. 13, 2007


Outline

We will discuss different issues related to robustness of multimedia data over IP network.

Topics include:

Error Correction

Error Concealment

Congestion Control


Error Correction for Bit Errors

Schemes to make encoded media stream resistant to bit errors:– Adding error correction codes.– Insertion of re-synchronization bits.– Bi-directional entropy code.– Self-synchronization entropy code.– Error resilient entropy coding (EREC).


Error Resilient Entropy Coding

Basic Idea: Convert variable length code to some kind of fixed length code.

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10


EREC Encoding

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Phase1 Phase2

Phase3 Final


EREC Encoding Scheme

1. The original code to be trans-coded is {Ci}, i = 0,…,M-1

2. Select slot width {Si} =0,…,M-1. Sum( Si ) = Sum ( length(Ci) ). Select search sequence {D(n)}, n =0,.., M-1. Set Phase = 0;

3. For i = 0 to M-1

If Si >= length(Ci)

Put the code in slot Si; Update left space;

else

Search slot mod( i+D(phase), M );

If there is space left, put the code in;

4. If all the codes fit into the slot

Terminate;

else phase++; Goto 3


EREC Decoding Scheme1. Get the header information about slotlength and slotwidth and M.

2. Set Phase = 0; Set {codelen(i) = 0, i = 0, …, M-1}, Set {cflag(i) =0, I=0,1, …, M-1}

3. For i = 0 to M-1

If codeword in slot i is fully decodable. Store codelen(i) = code length;

else search slot mod( i+D(phase), M ); Get bits from the position of codelen(mod( i+D(phase), M )); If fully decodable, update codelen(i) and cflag(i). Else get all the bits left in the searching slot.

4. If no cflag(i) == 0 or some no change since last phase, stop

else phase++; Goto 3


EREC for JPEG Coding

10111101100110101011101101011010010010010101011001101111100011111100100101010100000100001111001101010101001010101011001001110101101001101010110100010100101101001110101001011010011101010010110100101101001011010010110100010100010101000011001001010001011010001111110110101010111011110101101010111011100010101011100101010111001010110111011101001110100110110010101110110011101110001010110100011101011111101101010010011100010111000010111101101110101101010001001010011110001010011001010011011010010010101010010100111110101001111111011001011101110010010000111000011001100011110101001101010111101010101111100000011100110101101000100111100010101001111011110011111111010101110101100111101001110101101010101001010101010101001010011000001010001010

42 475101111011001101001010010110110101011000010101110110100111011010011010010111001011000110100100100101011100011000000101001000101101100110111110001111110010010101100010010101000001000011110011010111101111110100000101010010101010110101011001011111010010010101011001001110101101001111100111000101000011010101101010000101101111101110110010001001010011101101101010101111011101101010111010110100100101010100110010001010100111101011101010001100111100011000101001101100010010110101011111100101010101010011110000001011101010010111011101100010011010111111101010110101010101011000010101111101101110110010110101010111111010100010101110110001111010110100111101001011101110111011100000111010110101101001010100111111110110110000001001010010010001110011010010111011101010010001010001101110110101001110101001101001000100001100100101000110110010000011111111010001011010001111110110101011000110111000101101110111101011010010100101000111011100110101110111000101010111011110101011111111111101110010100101100011011111001110110001111101110010100100101010000100000111000111010110111011101001110100110110010101110110011110100011101011111101101010010011100010111100010010100111100010100000101111011011101011001010011010101110001010100100101101110011011010011010110010101110111011100011110010010101011010011010001111111111010101111101001010011111010100111111101100101110111101111010100010010000111000011001100011110101111100000011100110101010011010011111111110100010011110001010110011110001010010110100111101111001111111101010111010101000101110011110100111010110100100101100011010110101010010100010100000011111111101111001111101010101001010011011101101010100110111110011000001010001111111110111100101101011100001010110000011101100100011111111100111010001010011010101011111101110100110101011101

EREC Code

Decode

Header Information

10111101100110101011101101011010010010010101011001101111100011111100100101010100000100001111001101010101001010101011001001110101101001101010110100010100101101001110101001011010011101010010110100101101001011010010110100010100010101000011001001010001011010001111110110101010111011110101101010111011100010101011100101010111001010110111011101001110100110110010101110110011101110001010110100011101011111101101010010011100010111000010111101101110101101010001001010011110001010011001010011011010010010101010010100111110101001111111011001011101110010010000111000011001100011110101001101010111101010101111100000011100110101101000100111100010101001111011110011111111010101110101100111101001110101101010101001010101010101001010011000001010001010

Original code

Recovered code


JPEG Coding in Noisy Channels

Original image JPEG coding of Lena. Q=50.



JPEG coding of Lena. Q=50. BER=0.01 JPEG coding of Lena. Q=50. BER=0.001



JPEG coding of Lena. Q=50. BER=0.0001 JPEG coding of Lena. Q=50. BER=0.00001


EREC-JPEG in Noisy Channels

EREC-JPEG coding of Lena. Q=50. BER=0.01

EREC-JPEG coding of Lena. Q=50. BER=0.001


EREC-JPEG in Noisy Channels

EREC-JPEG coding of Lena Q=50. BER=0.001

EREC-JPEG coding of Lena Q=50. BER=0.0001


Another Example on Color Image Coding

Original image JPEG coding picture, Q=50


JPEG Coding in Noisy Channel

JPEG coding in with BER = 0.01 JPEG coding in with BER = 0.001


EREC Coding Results

Error resilient image coding inBER = 0.001.

Error resilient image coding inBER = 0.01.


Comparison Results

0

5

10

15

20

25

30

35

1.00E-02 1.00E-03 1.00E-04 1.00E-05 BER

PSNR(dB)

J PEG

EREC

Performance comparison JPEG and EREC. Lena. Q = 50. BPP = 0.6478. Random error.


Comparison Results

0

5

10

15

20

25

30

35

1.00E-02 1.00E-03 1.00E-04 1.00E-05 BER

PSNR(dB)

JPEG

EREC

Figure 5. Performance comparison JPEG and EREC. Gold Hill. Q = 50. BPP = 0.7567. Random Error.


Error Correction for Packet Loss

An error correction method

P1 P2 P3 C=P1 xor P2 xor P3

Reconstructedpacket

P1 P1 xor P3 xor C P3 C


Another Error Correction Scheme

D(n) R(n-1) D(n+1) R(n) D(n+2) R(n+1) D(n+3) R(n+2)

P(n) P(n+2) P(n+3)

R(n) is usually a “rougher” coding of D(n)

Recovered packet n+1


Dealing with Burst Packet Loss

Transform burst loss into random loss -- interlacing.

1 2 3 4 5

3 5 1 4 2Burst packet lost

2 3 5

Received packet loss pattern is random.

Originalpackets

Scrambledpackets

Receivedpackets


Error Concealment

Error correction usually needs additional error correction bits.

In fact, it is possible to recover or at least cover the

lost data using only the correctly received data – error concealment.

Error concealment can be used for images, videos and audios.


Images with Partial Data Missing

25% blocks lost 50% blocks lost


Reconstructed Images

Reconstructed images for cases of 25% and 50% block loss respectively.


Directional Adaptive Error Concealment

Color continuous Orientationcontinuous

Continuousin the desiredorientation

Missing imageblock


Error Concealment for Videos

Previous I or P P frame

v

For MPEG video, we can simply recover the lost blocks in P frames by replacing the lost blocks with the corresponding blocks in the previous I or P frame.


Congestion Control

The need for congestion control– Internet is a network that does not regulate the traffic of each

source strictly.– Aggressive sources may obtain more bandwidth than other

information sources.– When throughput in a network exceeds some threshold,

congestion collapse will happen.

Packet sending rate

Packetdeliveringrate

Normaloperation

Congestioncollapse


TCP Congestion Control

TCP uses a slow linear increase and exponential decrease scheme for congestion control.

Slow start

Fast decrease Time

PacketRate


TCP Congestion Control (cont)

Packet 1

ACK

packet2

ACK

Send-and-wait protocol. Maximum packet rate is 1/round_trip_time.


Sliding Window Scheme

Packet 1

ACK

Packet 2Packet 3Packet 4Packet 5Packet 6

If there is no bandwidth limitation, we can send data as fastas we can.


TCP Congestion Control

TCP protocol controls the bandwidth usage by adjusting the sliding window size (the number of packets sent out in a burst).

TCP detects congestion by timeout or the number of duplicate ACK.

Packet 1Packet 2Packet 3 x

Ack1

Ack1

Packet 1Packet 2Packet 3

Ack1

Ack1

Ack3


Multimedia and Congestion Control

Should multimedia traffic follow the same rules like TCP?

It depends. But some kind of scheme to avoid the congestion collapse of the whole network is worth to study.

Such congestion control schemes generally follow the same pattern like TCP congestion control schemes:

• Slowly increase the bandwidth usage to fully use the network resources.

• Reduce the usage of bandwidth if congestion is likely to happen.


TCP Friendly Protocols

For applications such as video conferencing, the video source can adapt its bit rate by changing the quantization factor.

Video source detects congestion condition by the parameters such as round trip time, packet loss ratio etc. using RTCP.

The bit rate of the video source can increase linearly in each round trip time if no congestion happens.

If congestion occurs, the video source should decrease the rate (as much as possible like a TCP source).

Documents

Multimedia Over IP Networks -- III