Error Control in Multimedia Streaming

What’s the difference

Traditional data: Text Packet loss is intolerable while delay is

acceptable

Multimedia data: Real-time video Some visual quality degradation is often

acceptable while delay must be bounded This feature introduces many new error

control mechanisms applicable to video application, but not applicable to traditional data.

Categories

FECRetransmissionError-resilient encodingError concealment

FEC

Forward Error Correction Add redundant information so that

original information can be reconstructed in the presence of packet loss.

More effective when the average number of consecutively lost packets is small.

Retransmission

May miss the play-out time, if the retransmission needs too much time which exceeds the allowable delaySo we need to know if the retransmission will be available before the deadline, then decide retransmit or not.

Error-resilient

Objective Enhance robustness of compressed video to

packet loss

Executed by the source to enhance robustnessDifference with FEC prevent error propagation or limit the scope

of the damage(caused by packet losses)

Error-resilient: example

MDC—multiple description coding Compressed into multiple streams (descriptions) Each description provides acceptable visual

quality More combined descriptions provide a better

visual quality

Advantage Robustness to loss, enhanced quality

Disadvantage Trade off between compression efficiency and

the reconstruction quality from one description

Error concealment

Performed by the receiver when packet loss has already occurredThe receiver conceal the lost data and make the presentation less displeasing to human eyesExampleError resilient and conceal are more about compression and the nature of the original data.

An architecture for error control mechanisms

FEC--categories

Channel codingSource codingJoint source/channel coding

FEC—Channel coding

The channel encoder places the k packets into a group and then create additional packets from them so that the total number of packets in the group becomes n.A user must receive K (K>=k) packets.These K packets will be enough for user to reconstruct all the original k packets.

FEC--Channel coding(contd.)

FEC--Channel Coding(contd.)

Pro Can scale to arbitrary number of receivers in a large

multicast group. Allows the network and receivers to discard some of

the packets which cannot be handled due to limited source.

Con Increase the transmission rate Increase delay Not adaptive to varying loss characteristics (can use

feedback to improve it)

FEC—channel coding(contd.)

EEP(equal error protection) All the bits of the compressed video stream are

treated equally. Not good for compressed video stream--MPEG

UEP(unequal error protection) Example: Priority Encoding Transmission Allow user to set different levels of error protection

for different segments of the video stream. Transport MPEG video which has an inherent priority

hierarchy (I-,P-,and B-frames), give more redundancy to I-frame, but less to B-frames.

Source coding

Source coding-based FEC(SFEC) The nth packet contains both nth GOB (group of

blocks) and redundant information about the (n-1)th GOB ( a much more compressed version)

The we can reconstruct (n-1)th GOB if it is lost and we receive nth GOB. (will has coarser quality)

Channel coding can perfectly recover, while SFEC can only recover an acceptable one.

Lower delay (needn’t to accumulate k blocks)

FEC—joint source/channel coding

Shannon’s Separation Principle Both source coding and channel coding can be done

independently. But it is only valid upon the assumption of allowing

infinitely long codewords (infinitely long delay)

FEC—JSCC(contd.)channel capacity-limited video commu. system:

spending more bits on the source means not enough channel protection, which leads to channel errors, received video quality is bad;

spending more bits on the channel means enough protection and no transmission errors, but then you have overcompressed the source material and received video quality is again bad.

There is a trade-off and balance point where the channel capacity is optimally allocated between source and channel to achieve the best received video quality.

FEC—JSCC(contd.)

Source coding Joint source/channel coding

FEC—JSCC(contd.)

Task 1: finding an optimal rate allocation between source coding and channel coding for a given channel loss characteristicTask 2: designing a source coding scheme to achieve its target rateTask 3: designing channel codes to match the channel loss characteristic and achieve the required robustness

Delay-constrained retransmission

An approach in Transport layerThe conventional retransmission scheme When packets are lost, the receiver sends

feedback to notify the source, the the source retransmits the lost packets

So the retransmitted packet arrives at least 3 one-way trip time after the transmission of original packet.

It only works if the one-way trip time is short with respect to the maximum allowable delay.

Categories

Unicast Receiver-based Sender-based Hybrid control

Multicast

Unicast—receiver-based

Receiver-based control Objective: minimize the requests of

retransmission that will not arrive timely for display

AlgorithmWhen receiver detects the loss of packet N: ))(( NTDRTTTif dsc

sDcT

)(NTd

Ask sender to retransmit packet N

Current time

Slack term, could include tolerance of error in estimating RTT.

Scheduled display time for packet N

Receiver-based(contd.)

Unicast—sender-based

Sender-based control Objective: suppress retransmission of

packets that will miss their display time at the receiver.

Algorithm:When the sender receives request of retrans. for

packet N:))(( '

0 NTDTTif dsc

0T

)(' NTd

retransmit packet N to receiver

Estimated one-way trip time

Estimate of Td(N)

Sender-based(contd.)

Unicast—hybrid control

Receiver-based control Objective: both of the objectives of receiver-

based and sender-based Receiver makes decisions on whether to

send retransmission requests while the sender makes decisions on whether to discard retransmission requests.

Higher performance at the cost of higher complexity.

Multicast

Retransmission has to be restricted within closely located multicast members. Because one-way trip times between these members tend to be small, and the neighbor members have the packet which is lost.Basic idea: this closely located multicast member will be the designated receiver (DR) in a tree-based hierarchical framework.

Logical tree with DR

Local recovery for multicast

LVMR

Layered Video Multicast with RetransmissionUse a statically configured logical treeImprove efficiency by not asking for retransmissions of those packets which cannot be recovered in time.Improve response time by sending immediate NAKs and multicasting retransmissions.Use buffers to not only absorb the jitter but also to increase the likelihood of getting retransmitted packets before deadline.

Adaptive Playback Points with Extended Control time

playback point of frame i is

In LVMR, playback point p0 is extended by an additional amount which is referred to as control time.

We have , is the maximum jitter in the network.

So there is at least a budget of time units after the arrival of a frame to recover any missing fragments.

Trade-off between latency and reliability.

LVMR—contd.

Rippi /0 P0:the playback point of frame 0

R: frame rate (frame/second)

00 tp

STORM

STructure-Oriented Resilient MulticastUse a dynamically changing logical tree

When a receiver first joins a multicast group, it uses an expanding ring search to look for parent nodes(DR).

Selects its parent based on the packet loss rate of the candidate parent nodes

Receiver periodically computes the ratio of the number of successful repairs from a parent to the number of NACKs sent to the parent. If this ratio falls below a threshold, start to find a new parent.

Let the receivers decide on their own the trade-off between latency and reliability.

Conclusion

FEC,retransmission, error-resilience are performed at both the source and the receiver side, while error concealment is carried out only at the receiver side. So it is used at the last stage.

They are combined to get a better performance.

Transport perspective: Retransmission, FEC

Compression perspective: Error-resilience, error concealment, FEC

Use feedback to improve the performance

Main ReferenceA. Albanese, J.Blomer, J.Edmonds, M. Luby, and M. Sudna, “Priority encoding transmission,” IEEE trans. On information Theory, vol.42,no.6,pp.1737-1744, Nov.1996J-C. Bolot and T.Turletti, “Adaptive error control for packet video in the Internet,” in Proc. IEEE Int. Conf. On Image Processing, pp.25-28, Lausane, Switzerland, Sept. 1996Dapeng Wu,Yiwei Thomas Hou,etc, Streaming Video over the Internet: Approaches and Directions IEEE Transaction on Circuits and System for Video Technology, Vol11,No1,February 2001Dapeng Wu, Yiwei Thomas Hou,etc, Transporting real-time video over the Internet:challenges and approaches, Proceedings of the IEEE, vl.88,no. 12, Dec.2000X. Li, S. Paul, P.Pancha, and M.H. Ammar, “Layered video multicast with retransmissions (LVMR): evaluation of error recovery schemes, “ in Proc. IEEE Int. Workshop on Network and Operating System Support for Digital Audio and Video, pp.161-172, May 1997.