1

Analysis of Motion Estimation Algorithm in HEVC

Multimedia Processing EE5359Spring 2014

Advisor: Dr. K. R. RaoDepartment of Electrical EngineeringUniversity of Texas, Arlington

Tuan Ho1001006444tuanpho@mavs.uta.edu

Update: 2/27/2014

2

Outline

● Scope of this work● HEVC Overview● Motion Estimation● Challenges for Motion Estimation in HEVC● Approaches● Implementation and Results

3

Outline

● Scope of this work● HEVC Overview● Motion Estimation● Challenges for Motion Estimation in HEVC● Approaches● Implementation and Results

4

Scope

● Analysis the computational complexity in HEVC motion estimation (ME) and state-of-the-art approaches.

● Implement the approaches into HEVC Model.

● Quality assertment methods:

● PSNR [20]● SSIM [21]● Bjontegaard BD-PSNR [22]

5

Outline

● Scope of this work● HEVC Overview● Motion Estimation● Challenges for Motion Estimation in HEVC● Approaches● Implementation and Results

6

HEVC Overview

● HEVC is the current joint video coding standardization project of the ITU-T and ISO/IEC MPEG [1].

● Aimed at doubling the compression ratio of H.264 at the same level of visual quality [1].

● Exceed H.264 in terms of performance and computational complexity.

● Lot of tools and features have been brought forth into HEVC: quadtree partion, SAO, etc.

7

HEVC Overview

HEVC Encoder Block Diagram (Decoder components modeling in light gray) [1]

8

HEVC Quadtree Partition

HEVC Quadtree Partition [15]

9

Outline

● Scope of this work● HEVC Overview● Motion Estimation (ME)● Challenges for Motion Estimation in HEVC● Approaches● Implementation and Results

10

ME Overview

Motion Estimation Process Illustration [3]

11

ME Computational Complexity

● Inter prediction can take up to 84% of the entire encoding time [17].

HEVC encoding time portions. These execution time results were obtained by profiling the encoding of the “BasketballDrive” Full HD (1920×1080) sequence. [17]

12

ME Computational Complexity

● Largest block size in HEVC is 64x64, 4 times bigger than its of H.264.

● This significantly raises the ME computational complexity.

13

Outline

● Scope of this work● HEVC Overview● Motion Estimation● Challenges for Motion Estimation in HEVC● Approaches● Implementation and Results

14

ME Challenges

● Many partition sizes in the quadtree HEVC 8x8 to 64x64

Quadtree Partition [1]

15

ME Challenges

● HEVC ME for large-sized partition (i.e. 64x64) requires heavy computation exceeding its predecessor. In H.264 largest block size is 16x16. [Ref]

- Big interpolation logic.- Time-consuming SAD process.- Large memory storage.

HEVC ME for 64x64 PU

64x64PU

16

ME Challenges

● HEVC Interpolation on large-sized partitions

HEVC Interpolation Filter [1]

17

Outline

● Scope of this work● HEVC Overview● Motion Estimation● Challenges for Motion Estimation in HEVC● Approaches● Implementation and Results

18

Approaches

Fast search algorithms - (a) Two dimensional logarithmic (2DLOG), (b) Three steps search (TSS), (c)New three steps search (NTSS) [6]

Employ optimized fast search algorithms.

19

Approaches

Fast search algorithms - (a) Four steps search (FSS), (b) One-at-a-time search (OTA), (c) Orthogonalsearch algorithm (OSA) [6]

Employ optimized fast search algorithms.

20

ApproachesHierarchical Search

Hierarchical Search [6]

21

Approaches

Complexity Control Employment

Complexity of processing regarding to CU Depth [15]

22

Approaches

Complexity Control Employment (to be cont')

23

Outline

● Scope of this work● HEVC Overview● Motion Estimation● Challenges for Motion Estimation in HEVC● Approaches● Implementation and Results

24

Implementation and Results

● To be added

25

References

[1] G. Sullivan et al, "Overview of the High Efficiency Video Coding (HEVC) Standard," IEEE Transactions On Circuits and Systems For Video Technology, Vol. 22, No. 12, pp. 1649 – 1668, Dec. 2012.

[2] T. Sotetsumoto et al, "Low complexity algorithm for sub-pixel motion estimation of HEVC," 2013 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC), pp. 1-4, Aug. 2013.

[3] N. Purnachand et al, "Fast Motion Estimation Algorithm for HEVC," 2012 IEEE International Conference on Consumer Electronics - Berlin (ICCE-Berlin), pp. 34 – 37, Aug. 2012.

[4] C. Yan et al, "Highly Parallel Framework for HEVC Motion Estimation on Many-Core Platform", IEEE Data Compression Conference (DCC), pp. 63 – 72, Mar. 2013.

26

References

[5] G. Sullivan et al, “Standardized Extensions of High Efficiency Video Coding (HEVC),” IEEE Journal of Selected Topics in Signal Processing, Vol. 7, No. 6, pp. 1001 – 1016, Dec. 2013.

[6] M. Jakubowski and G. Pastuzak, “Block based motion estimation algorithms – a survey,” Opto- Electronics Review, Vol. 21, No.1, pp. 86-102, Mar. 2013.

[7] HEVC development at Fraunhofer Heinrich Hertz Institute: http://hevc.hhi.fraunhofer.de/

[8] HEVC Bitstreams: ftp://ftp.kw.bbc.co.uk/hevc/

[9] HEVC Model (HM) repository: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/

[10] HEVC HM Software Manual: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-9.2-dev/doc/software-manual.pdf

27

References

[11] HEVC Development Guidelines: http://phenix.int-evry.fr/jct/doc_end_user/documents/8_San%20Jose/wg11/JCTVC-H1001-v1.zip

[12] HEVC Specification:Complexity control of high efficiency video encoders for power-constrained devices http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=7243

[13] T. Wiegand et al, “Overview of the H.264/AVC video coding standard,” IEEE Transactions on Circuits and Systems for Video Technology, Vol.13 , No. 7, pp. 560 - 576, July 2003.

[14] G. Correa et al, “Complexity control of high efficiency video encoders for power-constrained devices,” IEEE Transactions on Consumer Electronics, Vol. 57, No. 4, pp.1866 – 1874, Nov. 2011.

[15] G. Correa et al, “Coding Tree Depth Estimation for Complexity Reduction of HEVC,” IEEE Data Compression Conference (DCC), pp. 43 – 52, Mar. 2013.

28

References

[16] S. Kim et al, “A Novel Fast and Low-complexity Motion Estimation for UHD HEVC,” Picture Coding Symposium (PCS), pp. 105 – 108, Dec. 2013.

[17] M. Grellert et al, “An adaptive workload management scheme for HEVC encoding,” 20th IEEE International Conference on Image Processing (ICIP), pp. 1850 – 1854, Sep. 2013.

[18] D. Han; A. Kulkarni and K. R. Rao, “ Fast inter-prediction mode decision algorithm for H.264 video encoder,” 9th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 1 – 4, May 2012.

[19] K. R. Rao et al, “Video coding standards - AVS China, H.264/MPEG-4 PART 10, HEVC, VP6, DIRAC and VC-1,” Springer, 2014.

29

[20] PSNR article in Wikipedia: http://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio

[21] Z. Wang et al, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600 – 612, Apr. 2004.

[22] G. Bjontegaard (2008), “Improvements of the BD-PSNR model,” Document of ITU-T Q.6/SG16, Berlin, Germany, July 16-18, 2008.

30

Acronyms

● SAO: Sample Adaptive Offset

● 2DLOG: Two dimensional logarithmic

● CTU: Coding Tree Unit

● CU: Coding Unit

● FSS: Four steps search

● HEVC: High Efficiency Video Coding

● HD: High Definition

● JCT-VC: Joint Collaborative Team on Video Coding

● PU: Prediction Unit

● ME: Motion Estimation

● MPEG: Moving Picture Experts Group

31

Acronyms

● NTSS: New three steps search

● OSA: Orthogonal search algorithm

● OTA: One-at-a-time search

● SAO: Sample Adaptive Offset

● T.B.D: To Be Determined

● TSS: Three steps search

● VCEG: Video Coding Experts Group

32

The End

Thank you!

33

Old: 3D Video

● Two type of 3D display● Stereoscopic displays: need 3D-display glasses to

generate multi views for viewer' eyes.● auto-stereoscopic displays: do not require glasses

for viewing.