Hybrid Interpolation Algorithm for Reconstruction of Sub-sampled Chrominance Components

Jong-Ho Kim, Hong lin Jin and Yoonsik Choe School of Electrical and Electronic Engineering, Yonsei University, Seoul, South Korea

Abstract—This paper proposes a hybrid interpolation algorithm for reconstruction of chrominance components to original resolution which are sub-sampled as 4:2:0 format. The proposed approach combines a simple interpolation method with a more complex one. It adaptively selects the most appropriate interpolation method for a given pixel position using our own decision criteria in order to provide high-quality images while reducing the complexity efficiently.

Keywords-hybrid interpolation; sub-sampled chrominance; 4:2:0 color format

I. INTRODUCTION Most of the existing and emerging image and video coding

standards employ YCbCr 4:2:0 color format [1]. In this format, the chrominance components (Cb, Cr) are sub-sampled by a factor of two in both horizontal and vertical directions because of the bandwidth reduction and the coding efficiency. The chrominance components of image data should be reconstructed to the original resolution for displaying after decoding a given compressed image or video. The process of reconstruction requires the up-sampling technique which generates full resolution as 4:4:4 color format.

A large number of interpolation methods have been proposed, and they have their own characteristics in terms of complexity and visual performance. The nearest-neighbor interpolation simply takes the value of the closest lattice pixel. So it is the simplest but does not have good visual quality in the area around the edge. Higher order interpolation methods including cubic, spline, and Lanczos provide noticeably better images, but complexity increases significantly due to using more filter coefficients. Although a complex interpolation method generally outperforms a simple interpolation method, the differences are negligible for most chrominance pixels, with major differences occurring around edges where high frequency components are dominants.

In this paper, we propose an adaptive decision algorithm that combines a simple interpolation method with a more complex one to determine which interpolation method is the most appropriate method for a given pixel in terms of visual quality and complexity. It adaptively uses the nearest-neighbor interpolation at the homogeneous areas and the cubic interpolation at the areas around the edge like as the object boundary. So, the proposed algorithm can reduce the computation complexity efficiently while keeping the relatively high visual quality.

Figure 1. The histogram of the difference values between image interpolated with the nearest-neighbor and one interpolated with the cubic interpolation.

II. PROPOSED ALGORITHM We describe how two interpolation methods can be

combined to provide a high quality image without significantly increasing the computational complexity. The nearest-neighbor interpolation, which assigns the nearest pixel value, works very well for homogeneous areas where the pixel to be interpolated is similar to the original pixel. In the homogeneous areas there is no difference in visual quality between the nearest-neighbor interpolation and the cubic interpolation. In the other areas the visual quality degradation of the nearest-neighbor interpolation is high. In this case a more complex interpolation, such as cubic interpolation, is needed to get high quality results.

Fig.1 shows the histogram of the difference values between image interpolated with the nearest-neighbor and one interpolated with the cubic interpolation. As in Fig.1, for about 63% of the whole pixels, there is no difference between two interpolation methods. Furthermore, for more than 96% of the pixels, the differences are smaller than or equal to 2. Typically, when the difference is small enough, the human eye cannot distinguish between them. The large differences occur along the edges where pixel values change abruptly. Thus, if it is possible to decide a criteria to predict whether the difference between the nearest-neighbor interpolation and the cubic interpolation algorithms will be large or small, high-quality images can be obtained without increasing the number of operations significantly.

It is noted that the decision criteria, which predicts whether the difference between two interpolation methods will be large or small, should be simple and fast. If the decision is complex and requires a large number of operations, the computational complexity reduction may not be as large as one may wish. The decision criteria is the smoothness of the pixels around the

This work was supported by the LG Display Co., Ltd.(C2010-005620)

2011 IEEE International Conference on Consumer Electronics - Berlin (ICCE-Berlin)

978-1-4577-0234-1/11/$26.00 ©2011 IEEE 247

pixel to be interpolated. When the interpolation is performed as a series of 1-D operations, the criteria for the decision is to check the difference between two pixels that are nearest to the pixel to be interpolated C(i, j). If the difference between the two adjoining pixels C(i-1, j) and C(i+1, j) for row j is smaller than threshold d {0,255}, a nearest-neighbor interpolation is used. Otherwise, a cubic interpolation algorithm is used.

|C(i-1, j)- C(i+1, j)| d (1)

This is also carried out for the vertical direction. The value of the threshold should be chosen so that it reduces the number of operations with no significant degradation in image quality.

III. EXPERIMENTAL RESULTS In the following experiment, nine images with different

characteristics regarding edge, texture and spatial resolution have been tested. Fig.2 shows the PSNR values of the proposed hybrid interpolation method for all nine images over the different thresholds. At d=0 all of the pixels are interpolated using cubic interpolation. As the threshold d increases, a decline of the PSNR value is observed. It is clear that for d=2 the PSNR values remain almost the same as for d=0. Fig. 3 shows the proportion of the nearest-neighbor interpolation in the hybrid interpolation method for the different thresholds. In UHD images the proportion of the nearest-neighbor is relatively high than in SD images, because a large number of pixels of UHD images are in smooth areas. As can be seen in the Fig. 3, for d=2 already 35% to 85% of the pixels are interpolated using the nearest-neighbor interpolation. Therefore d=2 was chosen as the best tradeoff between complexity and performance.

Fig. 4 shows the original image UHD2 and the pixel maps where the cubic interpolation was used in the hybrid interpolation method. The pixels along the edges are interpolated using the cubic interpolation based on our decision criteria.

Table I shows the numbers of operations required for three interpolation methods per pixel to be interpolated. The number of operations for hybrid interpolation method depends highly on the proportion (p) of pixels interpolated with nearest-neighbor interpolation. Taking the threshold d=2, for p=0.35 the saving on operations is 5.6% and for p=0.85 it is already 49.4% without visual quality degradation.

TABLE I. NUMBERS OF OPERATIONS PER PIXEL TO BE INTERPOLATED

Nearest-neighbor Cubic Hybrid Store 1 1 1 Add 0 3 (1-p)*3+1 Multiply 0 4 (1-p)*4 Compare 0 0 1 Total 1 8 (1-p)*7+3

IV. CONCLUSION This paper presents a hybrid interpolation method for

reconstruction of sub-sampled chrominance components. The proposed scheme employs two interpolation methods based on the adaptive decision criteria. It is possible to obtain high-

quality images without significantly increasing the number of operations.

Figure 2. The performance of the proposed hybrid interpolation algorithm

for different thresholds. (SD : 640x480, HD : 1920x1080, UHD : 3000x2000)

Figure 3. The proportion of the nearest-neighbor interpolation for the

different thresholds. (SD : 640x480, HD : 1920x1080, UHD : 3000x2000)

Figure 4. The pixel maps where the cubic interpolation is used : (a) original

image UHD2, (b) Pixel maps.

REFERENCES [1] Itoh, Y. and Ono, T., “Up-sampling of YCbCr4:2:0 image exploiting

inter-color correlation in RGB domain,” IEEE Trans. on Consumer Electronics, Vol. 55, No. 4, pp.2204-2210, Nov. 2009.

248