High performance hardware architectures for Intra BlockCopy and Palette Coding for HEVC Screen Content
Coding extension
Rishan Senanayake1, Namitha Liyanage1, Sasindu Wijeratne1, SachilleAtapattu1, Kasun Athukorala1, P.M.K. Tharaka1, Geethan
Karunaratne1, R.M.A.U. Senarath1, Ishantha Perera1, AshenEkanayake1 and Ajith Pasqual2
1Paraqum Technologies,Colombo, Sri Lanka
2Department of Electronic and Telecommunication EngineeringUniversity of Moratuwa, Sri Lanka
ASAP 2017, The 27th Annual IEEE International Conference onApplication-specific Systems, Architectures and Processors
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 1 / 23
Outline
1 Background
2 Approaches
3 Novelties
4 Intra Block Copy
5 Palette Coding
6 Results
7 Conclusion
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 2 / 23
Background
H.265, introduced in 2013 achieves around 50% compressionefficiency compared to H.264
Mainly targets camera captured content
Modern videos include text, graphical motion and animations too
Extension to HEVC =⇒ HEVC Screen Content Coding(HEVC-SCC) finalized in 2016
HEVC-SCC achieves about 50% efficiency over HEVC for syntheticcontent
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 3 / 23
Features of Synthetic Content
Large uniformly flat areas
Repeated patterns
Highly saturated areas
Use of limited colors in an area
Numerically identical blocks
HEVC-SCC introduces Intra Block Copy (IBC) and Palette Coding (PLT)in addition to conventional transformation based coding
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 4 / 23
Examples : Gaming
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 5 / 23
Examples : Console Sharing
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 6 / 23
Examples : Navigation
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 7 / 23
Examples : Desktops
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 8 / 23
Challenge
50% bit rate reduction
...... But Encoding time increases by more than 2x
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 9 / 23
Approaches
SCC test model (SCM) approach - developed for testing andevaluating of the standard
Reduce complexity through pruning and low complexity estimating
Hash based speed ups by Kuo et.al and Tsang et.al.
Search area reduction by Tsang et.al.
Optimizing K-means clustering by Saegusa et.al and Winterstein et.al.
=⇒ Novel hardware based architectures proposed for IBC and Palettecoding
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 10 / 23
Novelties
Novel Three stage architectures for IBC and palette coding
New Hash Table to reduce resource usage by 8 times
Palette sorting architecture used in palette comparisons
Alternate coding approach to code palette syntax in parallel forRate-Distortion (RD) cost estimation
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 11 / 23
Intra Block Copy
Copies the content from a previously coded similar block
Challenge =⇒ Identifying best block
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 12 / 23
Architecture
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 13 / 23
Hash Optimization
....Proposed scheme reduces table size by a factor of 8
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 14 / 23
Palette Coding
Sending of color values of each pixel using indices
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 15 / 23
Palette Coding
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 16 / 23
Results - IBC
11% coding overheadcompared to SCMencoder
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 17 / 23
Results - Palette
5% coding overheadcompared to SCMencoder
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 18 / 23
Comparison
Total = 66K LUTs Total = 51K LUTs
Prototype implementations were synthesized for Xilinx VC707 platform
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 19 / 23
Conclusion
Hardware solutions are suitable for real time encoding of 1080p 30fpsfor HEVC-SCC
Significant amount of hardware resources can be saved with a slightcoding overhead
These novel architectures can be added to existing HEVC hardwareencoders to support HEVC-SCC profile
Future work will be on easy adaption of these architectures to existingHEVC encoders
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 20 / 23
References I
SCM Software repositoryAvailable : https://hevc.hhi.fraunhofer.de/scc/Accessed April 2017
C.-W. Kuo, H.-M. Hang, and C.-L. ChienIntra block copy hash reduction for hevc screen content codingSignal and Information Processing Association Annual Summit andConference (APSIPA) 2016
S.-H. Tsang, Y.-L. Chan, and W.-C. SiuHash based fast local search for intra block copy (intrabc) mode inhevc screen content codingSignal and Information Processing Association Annual Summit andConference (APSIPA), 2015
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 21 / 23
References II
S.-H. Tsang, W. Kuang, Y.-L. Chan, and W.-C. SiuFast hevc screen content coding by skipping unnecessary checking ofintra block copy mode based on cu activity and gradient.Signal and Information Processing Association Annual Summit andConference (APSIPA) 2016
T. Saegusa and T. MaruyamaReal-time segmentation of color images based on the k-meansclustering on fpga.International Conference on Field-Programmable Technology, Dec2007
F. Winterstein, S. Bayliss, and G. A. ConstantinidesFpga-based k-means clustering using tree-based data structures.23rd International Conference on Field programmable Logic andApplications
High performance hardware architectures for Intra Block Copy and Palette Coding for HEVCScreen Content Coding extension ASAP 2017 22 / 23