09a video compstream_intro_trd_23-nov-2005v0_2

Video Compression and Streaming

Brussels, November 23rd, 2005G. Briand et al

MediaNet WorkshopBrussels – November 23rd, 2005

Video Compression and Streaming2

Foreword (G. Briand – Thomson R&D France)

High end video encoding (H. Denieul – Nextream)

Versatile video decoding (JM Moutain – STM)



Outline

Foreword Context, standardization Video compression and streaming in Medianet

Rational and objectives Main achievements

Conclusion



Introduction (1/2 )

Tremendous explosion in R&D in the video compression field in the last 30 years

Video compression is now everywhere : TV broadcasting over cable, satellite or terrestrial networks, CD-ROM, DVD, PC video storage, Videophone and teleconferencing, Internet streaming (VoD, IPTV), Video over mobiles.



Introduction (2/2)

Some figures Uncompressed video -> big amount of data

- Color picture 800x320 pix, 24 bits/pix -> 6.3 Mbits- SDTV 720x480, 30Hz, 16 bits/pix -> 166 Mbit/s- HDTV 1920x1080, 30Hz, 16 bits/pix -> 1Gbit/s

Communication and storage capacities limits- Cable or satellite bandwidth : 38 Mbit/s- ADSL : 1 to 8 Mbit/s- DVD capacity : 5 to 8 GB



H.264/SVC

SMPTE/VC1

2000’s1990’s1980’s1970’s1960’s

Standardization (1/2)Video codecs

Transform Coding 65/80

MC Prediction 72/89

Entropy Coding 49/76

H.261

MPEG-1

H.262/MPEG62

MPEG4 ASP

H.263

H.264/AVC

DVCPRO

1950’s

DPCM 52/80

Tec

hn

olog

ies

Sta

nd

ard

s

Videophone 56Kb/s – 2Mb/s

CD-ROM 1-1.5Mb/s

Digital TV, DVD 4 to 80 Mb/s

Camcorder, VTR 25 to 50 Mb/s

Videophone 30 Kb/s

Video streaming & post-prod 30 Kb/s to 600Mb/s

Convergence of all video applications, digital cinema 30 Kb/s to 600Mb/s

Wavelet 85/--



Standardization (2/2)DVB Transport

20011995 1997 1999 2003 2005 2007 2009

DVB-S

DVB-C

DVB-S2

DVB-T

DVB-H

DVB-IPI in progress

Satellite TV

Cable

Mobile TV

Terrestrial TV

IPTV



Bit rate evolution

Mbi

t/s

2001

0

1

2

3

4

5

6

1995 1997 1999 2003 2005 2007

MPEG-2

1st generation encoders

1st generationencoders

2nd generationencoders

MPEG-4/H.264 AVC

MPEG-4 ASP

C. Ratio from 4:2:2

166

28

Bit rate evolution for SDTV Broadcast

3rd generation encoders(advanced Pre-processing)

2nd generation encoders(Stat-Mux + Rate control improvements)

2009



R&D process

A virtuous cycleSilicon technology

evolution(ICs, DSPs)

New applications Video compression research

Lower bit rates



Video compression and streaming in MedianetRational

Video is still a cumbersome media especially for new broadband access networks and with emerging HDTV Need for more compression efficiency

Diversification of video formats, video coding standards, video quality requirements Need for more flexibility in codecs

Heterogeneity of networks in terms of QoS, applications and services Need for better adaptation of video bitstreams



Video compression and streaming in Medianet Objectives (1/2) Propose versatile platform architectures for video encoding and decoding

New architecture models for codec platforms Supporting a diversity of video coding standards and particularly the

new MPEG-4 AVC/H.264 Supporting multiple video formats Targeting a diversity of applications

Develop corresponding algorithms for efficient video compression and video enhancement Algorithm studies covering new tools offered by MPEG-4 AVC/H.264 Appropriate algorithms for error concealment and post-processing



Video compression and streaming in Medianet Objectives (2/2)

Propose new network adaptation concepts and algorithm for efficient video streaming over heterogeneous IP based networks For Unicast and Multicast applications, Fully compliant with the new MPEG-4 AVC/H.264 video compression

standard Introduce new concepts of Video Playout in case of low QoS

Propose standards evolution for improved video streaming Focusing on scalability of the bitstreams While preserving compression efficiency



Video compression and streaming in MedianetTasks/Partners over the streaming chain

Off line MPEG-4 AVC/H.264 encoding

StreamingServer for MPEG-4 AVC/H.264

MPEG-4AVC/H.264 decoding

IP Network

- Extension of H.264 for improved streaming

Fraunhofer HHI

- MP4 file format- Adaptive streaming- Unicast, Multicast Fraunhofer HHI, Thomson R&D

Real time encoder

- Advanced encoderarchitectures- MPEG-4 AVC/H.264 compression algorithm

Nextream, Thomson R&D

- Advanced decoderarchitectures- Post-processing- Erreur concealment

Integrated receiver

STMicroelectronicsUniv. Bristol

Adaptive Playout

Politecnico Di Milano



Video compression and streaming in MedianetOutcomes (1/4) Advanced architectures for video

encoding and decoding (STMicroelectronics, Nextream, Thomson R&D) Common requirements between STM

and Nextream Particularly targeting next generation

of MPEG-4 AVC/H.264 codecs Flexible and reconfigurable

architectures

PE PE PE PE

PE PE PE PE

PE PE PE PE

PE PE PE PE

Data Mover Unit

Array Control Unit

RAM RAM RAM RAM

RAM RAM RAM RAM

RAM RAM RAM RAM

RAM RAM RAM RAM

Instruction bus

Data bus

InstructionFIFO

SIMD program memory

32 bits RISC

Datacache

Instr.cache

DMA out

Timer

Interruptcontroller

Debug interface

DMA in

Start codedetector

Barrelshifter

Compa-rators

Look UpTables

OutputFIFO

BitstreamFIFO

Readcontrol

Micro-Sequencer

Num_Table

Code

Read

halfempty

Bitstreamdata

Alignedbitstream

data

LUT adress

GetBits(N)ShowBits(N)

Decode_BlockRead

Shift Messaginginterface

Load

Load

ControllerController

Standard function

RISC core

Memories

Dedicated hardware

Sharedcommunication

buffer

Slave access

SDRAM

Slave access

SDRAM adressprocessor

SDRAMcontroller

Master access Master access

32 bits RISC

Datacache

Instr.cache

ControllerController

Master access

Timer

Interruptcontroller

Debug interface

Messaginginterface

External accesshandler

Internal accesshandler

Slave access Master access

Inst

ruct

ion

FIF

O

- New concepts of SoC architecture introduced : mix of RISC and VLIW processors, new bus infrastructures

- Highly programmable, open to firmware evolution to cope multi-standard, multi-format applications- Scalable generic architectures targeting many applications from studio production equipments to

light platforms for mobile services



Algorithms for efficient video compression and video enhancement (Thomson R&D, STMicroelectronics, Bristol University) Trade-offs between performance and complexity, studied in conjunction with architecture

teams

Video encoding - Based on MPEG-4 AVC/H.264 : 6 to10 times more complex than MPEG-2 standard- HVS pre-analysis tools for Region of Interest based encoding

Mbi

t/s

20010

1

2

3

4

2003 2005 2007

MPEG-2

2nd generationencoders

MPEG-4/H.264 AVC

3rd generation encoders

1st generationencoders

2009

Video compression and streaming in MedianetOutcomes (2/4)

Post-processing- New solutions for artefacts reduction and

removal in the context of H.264 Error-concealment

- Particularly focused on new H.264 entropy coding solutions : CAVLC, CABAC



Efficient video streaming over heterogeneous IP based networks (Fraunhofer HHI, Thomson R&D, Politecnico di Milano)

Covering Unicast & Multicast applications Unicast : full H.264 AVC streaming system with dynamic data rate adaptation for best

effort networks – temporal scalability and bit stream switching – adaptive video playout Multicast : using H.264 AVC transrating for adaptation to local network characteristics

Video stream

Networkvarying delay, packet loss,

bandwidth limitationClient 1

Video decoder Adaptive Playout

Adaptive StreamingServer

Trans-rating

Client 2

Video decoder Adaptive Playout

Feedback

Temporalscalability

data rate 1

data rate 2

data rate 3 Bitstreamswitching

Video compression and streaming in MedianetOutcomes (3/4)



Video coding evolution for improved streaming ( Fraunhofer HHI) Propose scalability tools upon H.264

AVC Re-use of most H.264 tools Temporal and SNR (quality)

scalability preserving coding efficiency

Successful submission to MPEG-21 SVC (Scalable Video Coding).

H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0

I B P B P B

H20 H1

0 L20 H1

0 H20 H1

0

Spatial upsampling

H21 H1

1 L21 H1

1 H21 H1

1

H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1

{MP}1,2

{MP}0

Layer 0: QCIF, 7.5 Hz, 64 kbit/s

Layer 1: QCIF, 15 Hz, 128 kbit/s

Layer 2: CIF, 15 Hz, 256 kbit/s




H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0H22 H0

0 H12 H0

0 L22 H0

0 H12 H0

0 H22 H0

0 H12 H0

0

I B P B P BI B P B P B

H20 H1

0 L20 H1

0 H20 H1

0H20 H1

0 L20 H1

0 H20 H1

0

Spatial upsampling

H21 H1

1 L21 H1

1 H21 H1

1H21 H1

1 L21 H1

1 H21 H1

1

H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1H23 H0

1 H13 H0

1 L23 H0

1 H13 H0

1 H23 H0

1 H13 H0

1

{MP}1,2{MP}1,2

{MP}0{MP}0

Layer 0: QCIF, 7.5 Hz, 64 kbit/s

Layer 1: QCIF, 15 Hz, 128 kbit/s





Fraunhofer/HHI algorithm taken as a reference for further core experiments within JVT/H.264 SVC consortium.

Promising for future streaming system: simple adaptation to different bit-rates on a bit-stream level

Video compression and streaming in MedianetOutcomes (4/4 )



Conclusion

Outcomes summary Advanced architectures for next generation of encoding-decoding IC’s, 2006-2007 Improve video processing algorithms differentiating our offering - compression

performance, add-ons for IC decoders Fully compliant MPEG-4 AVC/H.264 end to end streaming chain coping with a

large range of QoS – Unicast and Multicast SVC coding scheme with efficient scalability tools opening-up to new concepts

and strategies for future video streaming applications

Dissemination Participation to Navshp in CG-4 ~15 participations to conferences, papers, trade fairs Contribution to standardisation : focus on H.264/SVC and DVB-IPI

Documents

09a video compstream_intro_trd_23-nov-2005v0_2