SMPTE PDA Now-AVC PDF.ppt - Onstream Mediasas-origin.onstreammedia.com/origin/smpte0109/SMPTE PDA Now/2… · Subset of High Profile t hat uses only I picture High 444 Intra These

SMPTE PDA Now: 3D Stereoscopic Display Technology

Copyright © 2008 Society of Motion Picture and Television Engineers. All rights reserved.1

AVC CompressionChoosing the right tools from a larger toolbox

Welcome and Happy New Year!January 8, 2009

Thank you to our SMPTE PDA Now Premium sponsors for their generous support



Your Host

Joel E. WelchDirector of Professional [email protected]

Copyright © 2009 Society of Motion Picture and Television Engineers. All rights reserved.

SMPTE PDA Now

• Series of monthly 1-hour online, interactive webcasts covering a variety of technical topics

• 2nd Thursday of each month • Free professional development benefit for

SMPTE members

• Sessions are recorded for member viewing


convenience.



Housekeeping

• Please use the text chat function to submit questions to the speaker

• Please feel free to add to the discussion or offer additional information via text chat

• In the rare instance that the slides seem to not progress, try closing your browser and siging in again


again

Today’s Speaker

Michael BergeronMichael BergeronStrategic Technical Liaison




What is AVC?

• AVC stands for Advanced Video Coding• It has been discussed in JVT that ITU-T and ISO/MPEG jointly

established (JVT received an “Emmy” for this work)• ITU T was standardized it as “H 264” and ISO standardized as• ITU-T was standardized it as H.264 , and ISO standardized as

“ISO/IEC 14496-10 MPEG-4 part10 AVC”• AVC is a core technology for all types of encoding applications• AVC has been enhanced for contribution quality recording as High

Profiles (which were called Fidelity Range Extension: FRExt) and amended with intra frame profiles

• AVCHD is an AVC implementation provided by Panasonic and S


Sony• AVC-Intra is a subset of AVC high profiles using Intra frame coding

The goal of the JVT was to double the compression efficiency of then existing techniques

Compression Overview

• The goal of compression is to take advantage of the inherent redundancy of data to transform a file into a smaller file

• If the original file can be reconstructed bit for bit, this can be considered lossless compression

• For significant compression of video data (more than ~1:3*) lossy compression is necessary

• The goal in improving compression techniques is to increase the possible compression ratio for a given


p p gsubjective quality loss*

• Compression improvements usually require increased complexity*

*Video compression quality is content and application dependent



Compression Overview

• When video data in compressed frame-by-frame this is intra-frame compression

• Adjacent video frames are quite similar and these redundancies create opportunity for further compression Inter framecreate opportunity for further compression. Inter frame compression takes advantage of this

• Inter-frame compression is usualy long GOP, where GOP indicates “group of pictures” because the video stream is compressed in groups of several pictures and intermediate pictures are derived in part from the pictures at the ends of the GOP

GO


• All else being equal long GOP can achieve higher compression ratios

• The performance of long GOP compression is highly content and application dependent

AVC Profiles & Applications

High Intra ProfilesSubset of High Profile that uses only I picture

High 444 Intra

These are suitable for video acquisition, post production and

High ProfilesHigh Profile is supported by Blu-ray Disc and AVCHD

High 4:2:2 Intra Profile (Hi422IP) e.g., AVC-Intra100

High 10 Intra Profile (Hi10IP) e.g., AVC-Intra 50

Extended Profilesbasic tools + high efficiency tool (excluding CABAC) + streaming tool

archiving


Baseline Profilebasic tools + error robustness

Main Profilesbasic tools + high efficiency tool (including CABAC)



AVC Profile ImplementationMost Initial Data

High 444 Intra

High Intra ProfilesSubset of High Profile that uses only I picture

High ProfilesHigh Profile is supported by Blu-ray Disc and AVCHD

Extended Profilesbasic tools + high efficiency tool (excluding CABAC) + streaming tool

Reasonable Compression Ratios

High 4:2:2 Intra Profile (Hi422IP) e.g., AVC-Intra100

High 10 Intra Profile (Hi10IP) e.g., AVC-Intra 50


Baseline Profilebasic tools + error robustness

Main Profilesbasic tools + high efficiency tool (including CABAC)

Lowest Data Rates

AVC New Tools

• Integer Transform & Quantization • Variable block sizes, 2x2, 4x4 and 8x8• Intra-Frame prediction• Intra-Frame prediction• Improved Inter-frame prediction (long GOP)• Finer prediction accuracy to minimize errors• Flexible image slicing optimizes multi-node

processing• Powerful CAVLC and CABAC entropy encoding


• Powerful CAVLC and CABAC entropy encoding• De-blocking Filters to hide errors



Variable Size Blocks

Varying block sizes are chosen to bestare chosen to best fit to image data structure

Generates optimal coefficient data sets.


Creates more efficient coding.

Pixel Block Error Image

Intra frame Prediction


Some possible prediction modes

Best approximation, repeat right adjacent pixels

Errors



Improved INTER frame Compression

More complex (weighted) referencingMore accurate motion vectors


¼ pixel & 1/8 pixel motion vector accuracy and a variety of block sizes

Motion PREDICTION

Advanced Entropy Coding

• Context Adaptive Variable Length Coding (CAVLC)changes the code table to suit the localized image contentcontent

~ 25%* improvement over non-adaptive coding (e.g., huffman coding)

• Context Adaptive Binary Arithmatic Coding (CABAC)looks at the entire pictures probability distribution & can provide a 30% compression gain10%* i t CAVLC


* estimated average, real results are content dependent

~ 10%* improvement over CAVLC



Earlier techniques

No Change in Video Quality

CAVLC

CABACRelative compressed file sizes


Significant differences in coding complexities!

Single Slice

Flexible slice implementation example

Software Encode / Decode

Single CPU Model Multi CPU Model

Mode

Total 4 CPUs

* Use 1 GOP = 6 frames for Explanation


CPU #0

CPU #0

CPU #1

CPU #2

CPU #3

GOP 0

GOP 0 GOP 1 GOP 2 GOP 1

GOP 2

GOP 3

CPU Idling Time



(Long GOP Multi-Slice Implementation)Multi Slice ModeCPU #0

CPU #1

CPU #2

A

B

C

Flexible slice implementation example

Single CPU Model Multi CPU ModelTotal 4 CPUs

CPU #2

CPU #3D

Software Encode / Decode * Use 1 GOP = 6 frames for Explanation


Single CPU ModelCPU #0

Multi CPU ModelCPU #0

CPU #1

CPU #2

CPU #3

GOP 0 GOP 1 GOP 2 A

B

C

D

A B C D

H.264/AVC standard(2005 edition FRExt)

HighProfile

(4:2:0, 8bit)

High 10Profile

(4:2:0, 10bit)

High 4:2:2Profile

(4:2:2, 10bit)

High 4:4:4Profile

(4:4:4, 14bit)

Example 1: AVCHD

AVC applied to a video acquisition format

MPEG-2 Transport StreamDolby Digital or Linear PCM

Specify Audio and Streaming

See Tablewww.AVCHD-Info.org

AVCHD

Select specific Implementations and bit rates


AVCHD Compliant Modes

HE mode 6Mbps

PH Mode 22 MbpS

Vendor selects media, e.g., SD card. “Memory Stick”, 8 cm DVD, Internal memory

AVCHD Media

And others…



Like Mini-DV, AVCHD is Not Vendor Specific

• www.AVCHD-info.orgAdobe Systems Inc. Apple Inc. ATEME S. A. A S ft I

InterVideo, Inc. I-O DATA DEVICE, INC. MAGIX AG MICRO STAR INT'L

PLANNET ASSOCIATE CO., LTD. Renesas Technology Corp. SAMSUNG ELECTRONICSArcSoft, Inc.

Avid Technology, Inc. B.H.A Corporation BUFFALO Inc. CANON INC. CyberLinkCorporation DigiOn, Inc.

&

MICRO-STAR INT'L MainConcept AG Nero AG NEC Electronics Corporation Pegasys Inc. Pinnacle SystemsPIONEER CO O O

SAMSUNG ELECTRONICS CO., LTD. SHARP CORPORATION Sonic Solutions Techno Mathematical Co., Ltd. tram, Inc. Ulead Systems, Inc.

C

Founding Members


D&M Holdings Inc. Focus Enhancements, Inc. FUNAI ELECTRIC CO., LTD.

CORPORATION PIXELA CORPORATION

VideACE, Inc.List as of 6/4/08

MPEG2 1440X1080 25Mpbs(HDV)

AVCHD 1440X108013Mbps

Long GOP AVCHD for lower data rates

32GB SD Card

HF Mode13Mbps 5 Hours

HN Mode9Mbps 8 Hours

HE Mode6Mb 12 Hours


6Mbps 12 Hours



AVC-Intra

AVC applied to video acquisition format

• AVC-Intra: is an Implementation of 2nd Amendment FRExt AVC

Example 2:

FRExt AVC

• AVC–Intra: Does NOT utilize long GOP encoding and is true INTRA [independent] frame compression

• AVC-Intra: Has been implemented 50 Mbps or 100 Mbps

• AVC Intra: Always records an 10bit quantization and


• AVC-Intra: Always records an 10bit quantization and class 100 implementation has No pre-filtering or sub-sampling

H.264/AVC standard(2005 edition FRExt)

HighProfile

(4:2:0, 8bit)

High 10Profile

(4:2:0, 10bit)

High 4:2:2Profile

(4:2:2, 10bit)

High 4:4:4Profile

(4:4:4, 14bit)

AVC applied to video acquisition formatAVC-IntraExample 2:

High 4:4:4Intra Profile

High 4:2:2Intra Profile

High 10Intra Profile

Limited to Intra-only

H.264/AVC standard(Amendment 2 to 2005 edition)

Include basic parameters & stream structure


SMPTE RP2027 can reside on NLE

Include basic parameters & stream structure

Class 50 Class 100

SMPTE RP2008 (MXF Wrapper) Recording Media Formats



Positioning

• RP 2027 Sets data rate and limits tools

• RP 2008 Establishes AVC-Intra in MXF

Intra Frame

DCT Based H.264 AVCDouble efficiency

Motion AVC Intra

DV

HDCAM

wrapper • Intended for applications currently using motion

JPEG


Intra Frame

Long GOP

Motion JPEG

MPEG 2 AVCHD

AVC-IntraDVCPROHD

D5

100 Mb Full Raster

4:4:4

Contrast AVC-Intra With AVCHD

• Spending a Larger AVC Bit Budget

100 Mbps

50 Mbps

25 Mbps

Intra Frame

10 Bit

4:2:2

Full Raster


12 Mbps



10 Bit Quantization Advantage

10 bit vector scope trace 8 bit vector scope trace


Inverse Bit Penalty

75

10 Bit Quantization Cost?

50

55

60

65

70

PSN

R-Y

(L

umin

ance

)

12bit

10 bit

8 bit

At Higher Bit Rates, AVC encodes images so accurately, reducing the bit depth, reduces the compression efficiency!


45

Bit Rate Mb/s 50 100 150 200 250 300 350 400 450 500

Intra Frame CODEC Test



Why is Intra Frame Compression Desirable?

• Consistent encode quality with different content

• Consistent encode quality over generations

• More Efficient use of processing power

• Easier edit and access


p g p

Long GOPLong GOPI Frame OnlyI Frame Only

Consistent Initial Image Quality

PS

NR

Time Time


I B B P B B I BBI I I I I I I II

GOP



Long GOPLong GOP11stst generationgeneration 11stst generationgeneration

I Frame OnlyI Frame Only

Consistent Generational Quality

I B B P B B I B BB BI I I I I I I I II I

11 generationgeneration

22ndnd generationgeneration

11 generationgeneration

GOP 0 GOP 1 GOP 2

Same picture typeSame picture type Picture type changesPicture type changes

22ndnd generationgeneration


B B I B B P B B IB PI I I I I I I I II I

GOP 0 GOP 1 GOP 2

Long GOPLong GOPI1 I4 I7 I2 B3 B4 B6 B7 B9 B10B0 B1 I8P5I8I5 I6I2 I3I0

I Frame OnlyI Frame Only

Efficient Random Frame Access

0

1

2

3

1 4 7 2 3 4 6 7 9 100 1 8585 62 30

(1) (2) (3)(4)(1)


(1) (2) (3)(4)If we need IIf we need I66 picturepicture

Decode IDecode I66 picturepicture

(1)If we need BIf we need B66 picture,picture,

May need to decode 4 picturesMay need to decode 4 pictures



Long GOPLong GOP

0

I I I I B B B B B BB B IPII II II

I I Frame OnlyFrame Only

Enhanced Use of Multi-Slicing

0

1

2

3

GOP 0 GOP 1 GOP 2


Slice 0

Slice 1

Slice 2

Slice 3

CPU #0CPU #1CPU #2CPU #3

GOP 0

GOP 1

GOP 2

CPU #0CPU #1CPU #2CPU #3 idle

CPU Activity

CPU Activity

• Integer Transform

• Intra frame prediction

AVC-Intra Narrowed AVC Compression Tool Selection

• No inter-frame prediction (with new advancements)

• No deblocking filters

• Advanced entropy coding

• Multi slice encoding


Only very high bit rate efficient tools are chosen

For maximize quality and processing efficiency

• No Variable block sizes



Bit-rate Saving using CABAC

25

[ % ]

CABAC makes sense at these data rates

Selecting The Appropriate Entropy Coding

10

15

20CAVLC Reduces complexity here


CODEC bit-rate [Mbps]

5

00 50 100 150 200

AVC-HD

Original Intra Prediction

Integer Transform Quantization

Lossless

Lossless

Lossy

Streamlined AVC-Intra Compression process

frame

Intra prediction error

Intra Prediction data = 21Mb

Integer T f Quantization

Advanced Entropy coding


40 Mb Total

Prediction Error = 12Mb

Transform Quantization



AVC-Intra Class100 Result• 100 Mbps Data Rate

• Full Resolution 10 bit 4:2:2 RecordingFull Resolution 10 bit 4:2:2 Recording

• Encoding and decoding capable of partial restore and optimized for multi core processing

AVC-Intra Class 50 Result

• 50 Mbps Data Rate


• 10 bit 4:2:0 Recording with Pre-filter

• Encoding and decoding capable of partial restore and optimized for multi core processing

Questions

Mi h l BMichael BergeronPanasonic

Joel E. Welch



Thank you to our SMPTE PDA Now Premium sponsors for their generous support

Documents

SMPTE PDA Now-AVC PDF.ppt - Onstream Mediasas-origin.onstreammedia.com/origin/smpte0109/SMPTE PDA Now/2… · Subset of High Profile t hat uses only I picture High 444 Intra These