IMPLEMENTATION AND PERFORMANCE ANALYSIS of Dirac VIDEO CODING STANDARD AND COMPARISON WITH AVS CHINA...

IMPLEMENTATION AND PERFORMANCE ANALYSIS of Dirac VIDEO CODING STANDARD AND COMPARISON WITH AVS CHINA

Under the guidance of Dr. K R. Rao

Electrical Engineering DepartmentThe University of Texas at Arlington

BySaumya Raval (1000746720)

Saumya.raval@mavs.uta.edu

Multimedia Processing (EE 5359)Project Report

Project Objective

• This project will give an overview of the working, performance and hardware requirements of Dirac and AVS-China codecs.

• The objective of this project is to analyze the performance of the baseline profiles of the Dirac and AVS China video codecs based on various factors like video quality, bit rates, compression ratio, etc.

• Also using sample videos, factors such as PSNR, MSE and SSIM [24] will be derived for two standard formats at various bit rates.

Introduction

• A software or a device that enables video compression and decompression is known as a video codec [1].

• The need for video coding standards arose with the increased commercial interest in video communications.

• Video coding standards Dirac and AVS China are the latest standards adopted by Broadcasting Corporation BBC and China standards organization respectively [2].

Basic processing of Video files

Figure 1: Block diagram of general video coding and decoding process [36]

Dirac [10]

• Dirac[10] is a video compression system developed by the British Broadcasting Corporation (BBC) utilizing motion compensation and wavelet transforms.

• Dirac video codec applications span from mobile, internet, Ultra HDTV to film and video production.

• The Dirac encoder architecture is shown in Figure 2. The decoder shown in Figure 3 performs the inverse operations.

Figure 2: Dirac encoder block diagram [10]

Figure 3: Dirac decoder block diagram [2]

Wavelet Transform

• Wavelet filters split the signal into 4 frequency sub-bands namely LL (Low-Low), LH (Low-High), HL (High-Low) and HH (High-High)

• In further stages the LL is decomposed.• Daubechies wavelet filters are used to

transform and divide the data in sub-bands

Figure 4: Decomposition of bands into sub-bands by applying wavelet transforms [3]

AVS China [5]

• AVS video codec is developed by the audio video coding standard working group of China[5].

• AVS China comprises of four different profiles namely Jizhun, Jiben, Shenzan and Jiaqiang of which the Jiben profile (basic profile) is defined in AVS Part 7 for mobile applications [5].

Profiles Key applications

Jizhun profile Television broadcasting, HDTV, etc.

Jiben profile Mobility applications, etc.

Shenzhan profile Video surveillance, etc.

Jiaqiang profile Multimedia entertainment, etc.

Table 1: Applications of the various profiles of AVS China [5]

Table 2: Summary of profiles in AVS-Video [5]

Various profiles of AVS China [18]• AVS-Video Jizhun profile (base profile)

- First profile of AVS-Part2- Focuses on digital video applications like commercial broadcasting and storage media, including high-definition applications. - Preferable for high coding efficiency on video sequences of higher resolutions, at the expense of moderate computational complexity.

• AVS-video Jiben profile (basic profile) - Mobility video applications featured with smaller picture resolution- Ability on error resilience is needed due to the wireless transporting environment

• AVS-Shenzhan profile (extended profile) - Solutions of standardizing the video surveillance applications

• AVS-Jiaqiang profile (enhanced profile) - Movie compression for high-density storage- Relatively higher computational complexity

Parts of AVS China [2]

The following are the parts of AVS China.1. System2. Video3. Audio4. Conformance Test5. Reference software6. Digital media rights management7. Mobile video8. Transmit AVS via IP (Internet protocol) network9. AVS file format10. Mobile speech and audio coding

Figure 5: AVS China encoder block diagram [6]

Figure 6: AVS China decoder block diagram [5]

Layered Structure of AVS China [28]

Figure 7: Layered data structure [28]

Coding tools in AVS China

• 8x8 Intra Predictions [18]– Decoded information in the current

frame as the reference of prediction– Five luminance four chrominance – Four 8x8 luminance blocks can be

predicted using one of the five intra-prediction modes

– Prediction of the most probable mode is according to the intra-prediction modes of neighboring blocks

Figure 8 : Neighbor pixels in luminance intra prediction [18]

Table 3: Contest based most probable intramode decision table [40]

• Five luminance prediction modes are illustrated in Fig. 9.

• DC mode (mode2), diagonal down left (mode3) mode and diagonal down right mode (mode 4) and a three-tap low-pass filter (1,2,1)

Figure 9: Five luminance intra prediction modes [18]

• Inter prediction [5] – Derived from the decoded frames– Precision of motion vector in inter prediction is up to

1/4 pixel– Sub-pixel interpolation in AVS-video is called as two steps

four taps (TSFT) interpolation [27] and three kinds of filters are applied

– Filter of (-1, 5, 5, -1) to get the half-pixel reference pixel values as the first step and a filter of (1, 7, 7, 1) is applied for quarter-pixel reference pixel values either horizontally or vertically as the second step

– Exception of the second step is that for quarter-pixel reference pixel values of e, g, p, r , a diagonal bilinear filter is used

Figure 10 : Interframe prediction block sizes

Figure 11: Position of integer pixels, 1/2 pixels and 1/4 pixels [18]

Experimental Results• The software which has been used to perform Dirac is Dirac

1.0.2 [11] obtained from the official Dirac website [11], for AVS China Part 2 it is RM 9.0.2 [38] and for Part 7 it is RM 3.3.7 [39].

• Microsoft Visual C++ 2008 express edition [34] has been used to run the code and build the project for all the codecs.

• After building the project, code will generate two application files namely encode.exe and decode.exe.

• We run these two files using appropriate and necessary parameters and obtain the final result which is a decoded file.

• The original file and decoded file are than evaluated using MSU video quality measurement tool. The values of PSNR, MSE and SSIM are obtained from it.

Dirac• QCIF sequence: coastguard_qcif.yuv• Height: 176, Width: 144• Total no. of frames: 300 • Frames used: 100 • Original File size: 3713 KB (Kilobytes)• Frame Rate = 30 fps (Frames per Second)

Quality Factor (QF)

Compressed File Size (KB)

Compression Ratio Bitrate (KBps) PSNR (dB) SSIM MSE Time (sec)

0 29.7 125:1 72.456 22.34 0.511 378.8 70.92

3 39.9 93:1 97.254 26.335 0.69 151.21 74.29

5 71.6 52:1 174.408 30.84 0.867 53.472 81.02

8 213 17:1 520.22 37.133 0.966 12.58 82.97

10 450 8.25:1 1097.009 41.76 0.987 4.33 81.58

Lossless 2119.68 1.75:1 5163.324 100 1 0 92

Original Sequence QF = 0

QF = 5 QF = 10

Figure 12: Video quality at different QF valuesFile: coastguard_qcif.yuv, QCIF format

• CIF sequence: foreman_cif.yuv• Height: 352, Width: 288• Total no. of frames: 300• Frames used: 100• Original File size: 14,850 KB• Frame Rate: 30 fps

QF Compressed File Size(KB)

Compression Ratio

Bitrate (KBps) PSNR (dB) SSIM MSE Time (sec)

0 107 139:1 261.155 25.674 0.738 176.05 385.803

3 144 103:1 351.842 30.748 0.86 54.729 396.647

5 213 70:1 518.849 34.675 0.921 22.162 401.199

8 542 27:1 1319.337 39.709 0.968 6.951 418.026

10 1208 12:1 2953.936 43.135 0.982 3.159 429.424

Lossless 7844 2:1 19100.713 100 1 0 440.438

Original Sequence QF = 0

QF = 5 QF = 10Figure 13: Video quality at different QF values

File: foreman_cif.yuv, CIF format

AVS China Part 2• QCIF sequence: coastguard_qcif.yuv• Height: 176, Width: 144• Total no. of frames: 300• Frames used: 100• Original File size: 3713 KB• Frame Rate = 30 fpsQuantization

Parameter (QP)Compressed File Size (KB)

Compression Ratio

Bitrate (KBps) PSNR (dB) SSIM MSE Time (sec)

0 1628 2.25:1 6391.72 60.53 0.99 0.05 50.824

16 578 6.5:1 2815.79 44.64 0.96 2.23 47.281

32 127 29:1 950.96 34.14 0.92 25.01 42.79

48 20.9 175:1 192.98 24.13 0.62 250.73 29.831

63 5.74 638:1 33.7 18.86 0.34 844.71 27.601

Original Sequence QP = 0

QP = 32 QP = 63

Figure 14: Video quality at different QP valuesFile: coastguard_qcif.yuv, QCIF format

• CIF sequence: foreman_cif.yuv• Height: 352, Width: 288• Total no. of frames: 300• Frames used: 100• Original File size: 14,850KB• Frame Rate = 30 fps

QP Compressed File Size (KB)

Compresson Ratio

Bitrate (KBps) PSNR (dB) SSIM MSE Time (sec)

0 5652 2.5:1 15734.97 60.7 0.99 0.05 210.896

16 1352 11:1 3509.97 45.16 0.98 1.97 189.529

32 239 45:1 542.88 37.21 0.94 12.35 175.071

48 56.6 177:1 118.41 30.54 0.84 57.32 125.977

63 19.1 547:1 37.02 24.25 0.68 243.98 117.542

Original Sequence QP = 0

QP = 32 QP = 63Figure 15: Video quality at different QP values

File: foreman_cif.yuv, CIF format

AVS China Part 7• QCIF sequence: coastguard_qcif.yuv• Height: 176, Width: 144• Total no. of frames: 300• Frames used: 100• Original File size: 3713KB• Frame Rate = 30 fps

QP Compressed File Size (KB)

Compression Ratio

Bitrate (KBps) PSNR (dB) SSIM MSE Time (sec)

0 2437 1.5:1 6014.02 58 0.999 0.1 15.207

16 1157 3.2:1 2855.7 45.24 0.982 1.94 11.642

32 405 9:1 996.04 34.92 0.931 20.92 8.863

48 95.7 39:1 235.01 27.12 0.68 125.92 5.597

63 28 132:1 68.76 19.93 0.43 660.12 5.179

Original Sequence QP = 0

QP = 32 QP = 63

Figure 16: Video quality at different QP valuesFile: coastguard_qcif.yuv, QCIF format

• CIF sequence: foreman_cif.yuv• Height: 352, Width: 288• Total no. of frames: 300• Frames used: 100• Original File size: 14,850KB• Frame Rate = 30 fps

QP Compressed File Size (KB)

Compression Ratio

Bitrate (KBps) PSNR (dB) SSIM MSE Time (sec)

0 7782 2:1 19145.55 58.07 0.99 0.1 51.6

16 3000 5:1 7389.63 45.86 0.98 1.68 39.446

32 901 16.5:1 2214.26 37.31 0.94 12.05 31.429

48 282 53:1 695.2 29.87 0.83 66.94 21.522

63 125 119:1 307.7 21.93 0.65 416.32 20.329

Original Sequence QP = 0

QP = 32 QP = 63Figure 17: Video quality at different QF values

File: foreman_cif.yuv, CIF format

Graphs

0 5000 10000 15000 20000 250000

20

40

60

80

100

120

PSNR(dB) vs Bitrate(KBps)

DiracAVS Part 2AVS Part 7

Bitrate(KBps)

PSN

R (d

B)

File: coastguard_cif.yuv; Format: CIF 4:2:0

File: foreman_cif.yuv; Format: CIF 4:2:0

0 5000 10000 15000 20000 250000

20

40

60

80

100

120

PSNR(dB) vs Bitrate(KBps)

DiacAVS Part 2AVS Part 7

Bitrate(KBps)

PSN

R (d

B)

File: coastguard_qcif.yuv; Format: QCIF 4:2:0

0 1000 2000 3000 4000 5000 6000 70000

0.2

0.4

0.6

0.8

1

1.2

SSIM vs Bitrate(KBps)

DiracP2P7

Bitrate (KBps)

SSIM

0 1000 2000 3000 4000 5000 6000 7000 80000

0.2

0.4

0.6

0.8

1

1.2

SSIM vs Bitrate(KBps)

DiracAVS Part 2AVS Part 7

Bitrate (KBps)

SSIM

File: foreman_qcif.yuv; Format: QCIF 4:2:0

0 1000 2000 3000 4000 5000 6000 70000

100

200

300

400

500

600

700

800

900

MSE vs Bitrate(KBps)

DiracP2P7

Bitrate (dB)

MSE

File: coastguard_qcif.yuv; Format: QCIF 4:2:0

0 1000 2000 3000 4000 5000 6000 7000 80000

200

400

600

800

1000

1200

1400

MSE vs Bitrate(KBps)

DiracAVS Part 2AVS Part 7

Bitrate (dB)

MSE

File: foreman_qcif.yuv; Format: QCIF 4:2:0

1 2 3 4 50

50

100

150

200

250

300

350

400

450

Computational Time

DiracAVS Part 2AVS Part 7

Picture Quality

Tim

e (s

ec)

File: coastguard_qcif.yuv; Format: QCIF 4:2:0

1 2 3 4 50

50

100

150

200

250

300

350

400

450

500

Computational Time

DiracAVS Part 2AVS Part 7

Picture Quality

Tim

e (s

ec)

File: foreman_qcif.yuv; Format: QCIF 4:2:0

• The diagram for the computational time is based on the total time taken to encode 100 frames. The graph shows X-axis as picture quality. The X-axis signifies that picture quality is increasing as move right on X-axis. The picture quality taken in consideration for computational time is as below.

• For Dirac as we increase the QF, the quality increases. But for AVS China as we increase QP from 0 to 63, the quality of picture decreases. Hence for X-axis AVS China QP has been taken in reverse order.X-axis Dirac AVS Part 2 AVS Part 7

1 0 63 63

2 3 48 48

3 5 32 32

4 8 16 16

5 10 0 0

MSU Video quality measurement tool [30]

Fig 18: Snapshot of MSU tool

Conclusion

• It can be concluded from the results that Dirac stands out in terms of performance with respect to compression ratio, quality and applications over AVS China part 7 and part 2. The graphs and tabulations clearly show that the PSNR, MSE and SSIM [13] of the video sequences improve as the bit rate increases, while the bit rate is varied using the quantization parameter.

• Also from the computaional time shown we can see that Dirac gives better performance but takes the highest time to encode whereas AVS China Part 7 gives least performance but also takes least time to encode. Hence depending on the type of application computational time can also be a critical parameter that needs to be taken into consideration.

Abbreviations and Acronyms

• AIF – Adaptive Interpolation Filter • AU - Access Unit • AVC – Audio Video Coding• AVS - Advanced Video Standard • AVS-M - Audio Video Standard for mobile • BBC - British Broadcasting Corporation• CIF - Common Intermediate Format • FPS – Frame per second• HD - High Definition • ICT - Integer Cosine Transform • I-Frame - Intra Frame • IEC - International Electrotechnical Commission• IMS IP - Multimedia Subsystem • IP – Internet Protocol• ISO – International Organization for

Standardization• ITU-T International Telecommunication Union• JVT - Joint Video Team

• KBps – Kilo bytes per second• MB - Macroblocks • MBPAFF - Macro Block Pair Adaptive Field Frame• MPEG - Moving Picture Experts Group• MSE – Mean Square Error • NAL - Network Abstraction Layer • PAFF - Picture Adaptive Field Frame• P-Frame - Predicted Frame • PIT – Pre-scaled Integer Transform • PSNR – Peak Signal to Noise ratio• QCIF - Quarter Common Intermediate Format • QF – Quality Factor• QP - Quantization Parameter • RTP – Real-time Transport Protocol• SD - Standard Definition • SSIM – Structural Similarity Metric • TV - Television• VCEG - Video Coding Experts Group• VCL – Video Coding Layer• VLC - Variable Length Coding

References [1] T. Sikora, “Digital video coding standards and their role in video communications”, Signal processing for multimedia, J.S.

Byrnes (Ed.), IOS press, pp. 225-251, 1999. [2] K. R. Rao and D. N. Kim, “Current video coding standards: H.264/AVC, Dirac, AVS China and VC-1,” IEEE 42nd southeastern

symposium on system theory (SSST), pp. 1-8, March 2010. [3] K. Onthriar, K. K. Loo and Z. Xue, “Performance comparison of emerging Dirac video codec with H.264/AVC,” IEEE International

conference on digital telecommunications, ICDT 2006, vol. 6, Page: 22, Issue: 29- 31, Aug. 2006. [4] X. F. Wang and D. B. Zhao, “Performance comparison of AVS and H.264/AVC video coding standards,” Journal of computer

science and technology, Vol. 21, No. 3, pp.310-314, May 2006. [5] L. Yu, S. Chen and J. Wang, “Overview of AVS video coding standards,” Signal processing: image communication, Vol. 24, Issue

4, pp. 247-262, April 2009. [6] L. Fan et al, “Overview of AVS video standard”, IEEE International conference on multimedia and expo (ICME), Vol. 1, pp. 423 -

426, June 2004. [7] T. Borer and T. Davies, “Dirac video compression using open technology,” BBC EBU technical review, July 2005. [8] T. Borer, “Dirac coding: tutorial and implementation,” EBU networked media exchange seminar, EBU, Geneva, 22-23 June

2009. [9] Dirac specification, Version 2.2.3, Available: http://diracvideo.org/download/specification/dirac-spec-latest.pdf [10] BBC research on Dirac: http://www.bbc.co.uk/rd/projects/dirac/technology.shtml [11] Dirac video download source reference: http://diracvideo.org/download/ [12] YUV video sequences source: http://trace.eas.asu.edu/yuv/ [13] Z. Wang, et al, “Image quality assessment: From error visibility to structural similarity”, IEEE Transactions on Image

Processing, Vol. 13, No. 4, pp. 600-612, April 2004. [14] A. Ravi and K.R. Rao, “Performance analysis and comparison of the Dirac video codec with H.264/MPEG-4 part 10 AVC”,

International journal of wavelets, multi-resolution and information processing , Vol. 09, Issue: 04, pp: 635-757, Jan. 2010. [15] L. Fan, “Mobile multimedia broadcasting standards”, Springer Publication, ISBN 978-0-387-78263-8, 2009. [16] AVS-China official website: http://www.avs.org.cn

[14] A. Ravi's Thesis under Dr. K.R. Rao, “Performance analysis and comparison of the Dirac video codec with H.264/MPEG-4 part 10 AVC”, International journal of wavelets, multi-resolution and information processing , Vol. 09, Issue: 04, pp: 635-757, January 2010.

[15] L. Fan, “Mobile multimedia broadcasting standards”, Springer Publication, ISBN 978-0-387-78263-8, 2009. [16] AVS-China official website: http://www.avs.org.cn [17] Dirac video codec - A programmer's guide: http://dirac.sourceforge.net/documentation/code/programmers_guide/toc.htm [18] L. Yu et al, “Overview of AVS-Video: tools, performance and complexity,” SPIE VCIP, Vol. 5960, pp. 596021-1~596021-12,

Beijing, China, July 2005. [19]Dirac developer support: Wavelet transform:

http://dirac.sourceforge.net/documentation/algorithm/algorithm/wlt_transform.xht [20] T. Davies, “The Dirac algorithm”: http://dirac.sourceforge.net/documentation/algorithm/ [21] H. Eeckhaut, et al, “Speeding up Dirac’s entropy coder”, 5th WSEAS International conference on multimedia, internet and

video technologies, pp. 120-125, Greece, Aug. 2005. [22] M. Tun, K. K. Loo and J. Cosmas, “Semi-hierarchical motion estimation for the Dirac video codec”, 2008 IEEE international

symposium on broadband multimedia systems and broadcasting, pp. 1-6, 2008. [23] M. Tun and W. A. C. Fernando, “An error-resilient algorithm based on partitioning of the wavelet transform coefficients for a

DIRAC video codec”, Tenth international conference on information visualization, 2006, IV, Vol. 5-7 , pp. 615 –620, Issue : July 2006.

[24] W. Gao et al, “AVS - The Chinese next-generation video coding standard”, NAB, Las Vegas, 2004.[26] Power point slides by L.Yu, chair of AVS video : http://www.ee.uta.edu/dip/Courses/EE5351/ISPACSAVS.pdf[27] R. Wang, et al., “Sub-pixel motion compensation interpolation filter in AVS”, 2004 IEEE International Conference on

Multimedia and Expo, pp. 93-96, 2004.[28] Project proposal on "Low Complexity AVS-China Part-2 video using data mining techniques" by Jennie Abraham:

http://www-ee.uta.edu/Dip/Courses/EE5359/Thesis%20Project%20table%20docs/ jennieproposal.doc[29] A. Ravi "Performance analysis and comparison of Dirac video codec with H.264/MPEG4 part 10 AVC“, M.S. Thesis, EE Dept.,

University of Texas at Arlington, Aug. 2009.

[30] MSU video quality measurement tool : http://compression.ru/video/quality_measure/video_measurement_tool_en.html [31] S Xu and S. Wang, “Main program flow analysis in AVS-M”, 2nd International conference on networking and digital society

(ICNDS), Vol. 2, pp. 621-624, May 2010.[32] F. Zhou and W. Xiong, “Realization and optimization of AVS coding algorithm ”, International Conference on system science,

engineering design and manufacturing informatization (ICSEM), Vol. 1, pp. 282-285, Oct. 2011.[33] Dirac programmers’ guide: http://diracvideo.org/download/dirac-research/documentation/programmers_guide-1.0.0.pdf[34] Microsoft Visual C++ 2008 express edition: http://msdn.microsoft.com/en-us/express/future/bb421473[35] Dirac algorithm: http://diracvideo.org/download/dirac-research/documentation/[36] J.Ostermann et al, “Video coding with H.264/AVC: Tools, Performance, and Complexity”, IEEE Circuits and Systems Magazine,

vol. 4, Issue:1, pp. 7 – 28, Aug. 2004.[37] S.Kwon, A. Tamhankar and K.R. Rao, “Overview of H.264 / MPEG-4 Part 10”, J. Visual Communication and Image

Representation, vol. 17, pp.186-216, April 2006. [38] AVS China software: Part 2: ftp://124.207.250.92/incoming/video_codec/AVS1_P2/RM09[39] AVS China software: Part 7: ftp://124.207.250.92/incoming/video_codec/AVS1_P7