2011 2nd International Conference on Instrumentation Control and Automation 15-17 November 2011, Bandung, Indonesia

Haar Wavelet Based Reduced Reference Quality Assessment Technique for JPEG/JPEG2000 Images

Antony Ralim Department of Computer Engineering

Universitas Multimedia Nusantara Scientia Garden, J1. Boulevard Gading Serpong

Tangerang - 15810, Indonesia Email: antony-sk@unimedia.ac.id

Abstract-We propose a new Reduced Reference model for assessing the quality of JPEG and JPEG2000 images. We utilize the Haar Wavelet Decomposition as a tool to model the image information along with the distortions presents in JPEG or JPEG2000 images. One of the superiority of our method is the ability to independently assess the quality of JPEG and JPEG2000 images without using any additional discrimination method. Our method also incorporate a simple design to model the distortions in the image without using complex Human Visual System (HVS) model. Despite its simple design, our method has demonstrated an accurate and reliable quality metric. Experi­mental results using several categories of JPEG and JPEG2000 images using LIVE Image Database Release 2 will be presented to verify our metric. The results of our experiments show that the method has achieved high correlation with the subjective data in most of the categories.

I. INTRODUCTION

Image quality assessment plays a very important role in image processing. Many real world image or video applica­tions nowadays are in the need of high quality images such that image quality has become one of the most important aspects in those applications. In some other applications, image quality factor sometimes serves as a trade-oil for storage and bandwidth requirement for the application. Commonly, most of real world applications have a limited bandwidth and storage. Limited bandwidth relates to the possibility of image and video data to be transmitted via network. High quality images and video data can be extremely large, thus it can become a bottleneck when the data is transmitted over a network. Also, high quality image and video data can take up lot of storage space, resulting in an inefficient utilization of storage space.

In most applications, image encoding technique is used to compress and reduce the size of image or video data. Most of image encoding techniques reduce the quality of an image as a compensation for the reduced size of data needed to represent the image. Due to loss of information that is caused by the encoding process to reduce the size of the data, it will produce various distortions in the image according to its encoding technique. One of the most encoding techniques used in image processing field is JPEG encoding. The JPEG encoding is a block-based encoding technique that will introduce blockiness

978-1-4577-1460-3/11/$26.00 ©2011 IEEE

Irwan Prasetya Gunawan Department of Information Technology

Universitas Bakrie J1.H.R.Rasuna Said Kav. C-22

Jakarta - 12920, Indonesia Email: irwan.gunawan@bakrie.ac.id

artifacts in the image. Another commonly used encoding technique is JPEG2000 encoding, which mainly introduces blur and ringing distortions to the image.

Image encoding techniques have been an active research field in image processing. Many researchers investigate new encoding technique to come up with a better compression rates for image and video data. However, image quality serves as a benchmark for encoding technique. A good encoding technique must able to give an efficient data reduction whilst maintaining the quality of the images.

Performing image quality assessment can be very difficult, since it needs to represent the visual perception of human viewers. Subjective test performed by several human viewers is believed as the most accurate way to perform image quality assessment. However, its drawback lies in time and cost for performing the test. Due to these factors, objective quality assessment that performs image quality assessment through an algorithmic approach is more attractive and preferred than its counterpart. To perform such an objective quality assessment that represents human visual perception is very challenging due to the variability and uncertainty of human judgements.

Objective image quality assessment technique is divided into three models: 1) Full Reference (FR) Model, which needs a reference image to perform quality assessment. Such techniques are Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), Universal Quality Index (UQI) [1], and Multiscale Structural Similarity (MSSIM) [2]. 2) Reduced Reference (RR) Model, which use certain information from the reference image to perform quality assessment. An example of such technique is local harmonic strength (LHS) [3]. 3) No Reference (NR) Model, which perform quality assessment in absence of any reference image. Some example of the NR model is the No-Reference Perceptual Blockiness Met­ric (NPBM) [4], perceptual blur metric [5], JPEG quality metric [6], blur and noise metric [7], and JPEG/JPEG2000 metric [8]. Each of the models has its own advantages and disadvantages. For FR model, it is easier to perform image quality assessment with a relatively good accuracy due to the availability of the reference image. However, not every application has the luxury of having the reference image. In some applications, RR model is preferable. The RR model uses

92

2011 2nd International Conference on Instrumentation Control and Automation 15-17 November2011, Bandung, Indonesia

original vide transmission f+--+- coded video

subjective subjective evaluation ----. score

(a) Full reference

original video transmission f+--+- coded video

subjective subjective evaluation ----. score

(b) Reduced reference

transmission ..... ----l� coded video

score

(c) No-reference

Fig. I. Objective quality assessment model

a certain features from the image as the reference information to perform image quality assessment. It is more practical than FR, i.e. the features data is smaller than a size of image data thus it can be easier to be transmitted over the network or to be preserved, while maintaining the capability for performing accurate quality assessment. Finally, the NR model is specifically designed for practical approach when no reference image information needed to perform quality assessment. However, it is likely more difficult for NR model to perform image quality assessment than both FR and RR model. The objective quality assessment model is illustrated in Fig. 1.

In this paper, we focus on RR model due to its practical approach and the advantages of reference information that can be used to perform image quality assessment. We propose a new Reduced Reference (RR) model of quality metric for assessing both JPEG and JPEG2000 images. We utilized Haar Wavelet Decomposition (HW D) to analyse information and distortion presents in the image. The HW D analysis was previously used by [9] to blindly perform blur assessment of digital images by distinguishing blurred and unblurred images. The HW D is used by [9] to classify various edge structures. By analysing the detail coefficients of the HW D, the method was able to single out blurred edges (distorted edges) information on the image. In this paper we introduce a modification to the method used by [9] so that the new method can be used for RR quality assessment of JPEG and JPEG2000 images.

978-1-4577-1460-3/11/$26.00 ©2011 IEEE

The superiority of our proposed method is twofold. Firstly, we are able to perform quality assessment of JPEG and JPEG2000 images using a single unified method, unlike pre­vious approach presented in [8] which has to rely on some discrimination methods to identify the encoding type and subsequently apply separate analysis for quality assessment purposes. Secondly, the size of reference data that is need to be transmitted and the simplicity of the method. Our proposed method uses only a single quality value of reference image for any image size as the reduced reference data. The quality value of reference image itself is obtained by using the same algorithm to assess the reference image without using any extra information (NR model for reference image). The quality value of the reference image is used to compensate the quality value of the test/decoded images to gives a highly accurate prediction of image quality. Our proposed method also does not employ any specific HVS functions that can increase the complexity of the proposed method.

This paper is organized as follows. In Section II we will describe our proposed method to assess the quality of JPEG and JPEG2000 images. Subsequently, in Section III we will present our experimental result and analysis of our proposed method. Last but not least, we will conclude our paper in Section IV.

II. METHOD

The proposed method employs a HWD analysis to an image and constructs local maxima features that are used to assess the quality of JPEG/JPEG2000 encoded images. By measuring

93

2011 2nd International Conference on Instrumentation Control and Automation 15-17 November 2011, Bandung, Indonesia

the normalized difference between the local maxima features, the proposed algorithm is able to measure the rate of the distortions presents in the image and computes the quality metric. The quality metric thus is normalized using certain flat parameter to produce the final quality score. The detail of the proposed method is as follows.

Let J (x, y) be a W x H images, W is the width of the image, and H is the height of the images. Some additional alignments is performed to match the size of the image by a unit block

I6x 16 pixels. Let [Ai(ki, li) IHi(ki, li) IVi(ki, li) I Di(ki, li)] = H W D(Ai-d, i = {I, 2, 3} represents the Haar Wavelet Decomposition of an image J (x, y). Ai represents the ap­proximation coefficients extracted from the HWD at level of decomposition i. Hi(ki, li), Vi(ki' li), Di(ki, li) represents the horizontal, vertical, and diagonal detail coefficients extracted from the HWD at level of decomposition i respectively. Note that for level decomposition i, the HWD takes the approxima­tion coefficients Ai-1 as an input, given Ao = J (x, y).

An edge map EmaPi (ki' li) of the image J(x, y) thus is constructed at level of decomposition i using the detail coefficients obtained from the HW D. The Emapi (ki' li) is expressed as:

with i = 1, 2, 3. For a complex number, the magnitude of the complex detail coefficients is used. A local maxima of edge map within 2(4-i) x 2(4-i) blocks for level of decomposition i thus is constructed. Let Bsti represents the blocks at EmaPi (ki' li), 8 ::::: Ki/2(4-i), t ::::: Li/2(4-i). Ki, Li represents the row and column of the edge map at level of decomposition i. The local maxima of the edge map is com­puted in block Br;:�x. The local maxima matrix Emaxi(8,t) thus can be expressed as: (BmaX Bmax Bmax) 11i 12i lti

B21ax B:n�x BJJ:�x Emaxi(8,t) = : '

B;I�x B;I�x Br;:�x (2)

Emaxi (8, t) is used in [9] to classify different type of edge structure. For the proposed method, the ditlerence between Emaxi (8, t) at level i and i -I is used as a feature that represents the detail gain in the blocks. Let 91 (8, t) and 92( 8, t) represents the difference matrix. Then 91 (8, t) and 92( 8, t) can be expressed as:

91(8,t) = IEmax2(8,t) -Emax,(8,t)1 92(8, t) = IEmax3 (8, t) -Emax2 (8, t)1

(3)

(4)

After that, the total information gain at point 8, t, Gall (8, t) is computed by mUltiplying 91(8,t) and 92(8,t) and then normalized the value to the range [0 .. 1] by dividing Gall (8, t) at all point 8, t with the maxima of the Gall (8, t). This process is expressed as:

(5)

978-1-4577-1460-3/11/$26.00 ©2011 IEEE

In this case, each element of 91 and 92 is multiplied to introduces a spatial masking characteristics to the information features. For the next step, the edge information along with distortion information in the images is computed and modelled based on Emaxi (8, t). It can be easily done by applying the rules from [9] to characterized edge point and the Roof and Gstep edge structure in the image based on Emaxi (8, t). The original rules from [9] are as follows:

1) For any Emaxi (8, t), if Emaxl (8, t) > 6e or Emax2 (8, t) > 6e or Emax3(8,t) > 6e, then 8,t is an edge point;

2) For any edge point, if Emax,(8,t) > Emax2(8,t) >

Emax3 (8, t), then 8, t is a Astep-edge or Dirac-edge structure;

3) For any edge point, if Emaxl (8, t) < Emax2 (8, t) <

Emax3(8,t), then 8,t is a Roof-edge or Gstep-edge structure;

4) For any edge point, if Emax2 (8, t) > EmaXl (8, t) and Emax2(8,t) > Emax3(8,t), then 8,t is a Roof-edge structure;

5) For any Roof and Gstep edge structure if Emaxl (8, t) < 6e, then 8, t is a blurred edge.

The proposed method only applies rule number 1,3,4 and 5. The blurred edges defined in rule number 5 thus are generalized as distorted edges. Note that the parameter 6e is a value that acts as a threshold to characterize an edge in point 8, t.

In a heavily compressed JPEG/JPEG2000 images, severe blockiness or blur artefacts will likely to occur. Severe block­iness artefacts are usually followed by a severe flat area. Similar cases also happen to severe blur or ringing distortions, although the characteristic of flat area is different from those following blockiness artefacts. In general, the heavy loss of information in a heavily compressed JPEG/JPEG2000 will trigger the appearance of flat-distorted-area. The proposed method thus also detects the flat-distorted area by applying additional rule:

6) For any point (8, t), if 6how < Emax,,2,3 (8, t) < 6high' then 8, t is a flat­distorted area.

The sixth rule is applied to mark the possible flat-distorted­area in the image. The rule used two thresholds which are 6 how and 6 high. The flat-distorted-area rule is derived based on observation that in a flat distorted area, all of the Emaxi (8, t) value for each level of decomposition i are low. Based on this observation, thus 6 how is defined as the lower limit of Emaxi (8, t) value that is considered as flat-distorted-area, while 6high represents the maximum value of Emaxi (8, t) that is tolerated to be still categorized as flat-distorted-area. The rule thus marks the flat distorted area by measuring all of the Emaxi (8, t) whose values for every level of decomposition i are between 6 how and 6 high.

For computational approach, binary masks Md(8, t), Mfd(8, t) is calculated to mark distorted edges and flat-

94

2011 2nd International Conference on Instrumentation Control and Automation 15-17 November 2011, 8andung, Indonesia

distorted area respectively as expressed in equation (6)-(7):

Md(s, t) = ' (6) {I if (s, t) is a distorted edge

0, otherwise

Mfd(s, t) = ' (7) {I if (s, t) is a flat-distorted area

0, otherwise

Note that when dealing with flat-distorted-area the infor­mation gain between Emax is minimal. Thus, another param­eter for flat-distorted-area must be computed. Let Pflat(S, t) represents the flat-distorted-area information parameter. The flat-distorted-area information parameter Pflat(S, t) can be expressed as:

( t) 3c5!high-[LT�lEmaxi(s,t)l (8) Pflat S, = 315 fhigh

Using all the information given above, a distortion feature matrix d features (S, t) is computed by applying a simple power operation of a constant K with the weighted coefficient for distorted weight Wd of the binary mask Md(S, t) multiplied by Gall(s, t) which is expressed as:

d (t) K[WdXMd(S,t)] G ( t) features S, = X all S, (9)

The base distortion score Dm is then computed using:

Dm = log L L dfeatures(s, t) (10) t

To get the quality metric Qm, the base distortion metric is normalized according to the flat-distorted metric. Let the flat­distorted-metric is expressed as Pm which can be expressed as:

1 Pm = log (100 + [Ls LtPflat(s, t) X Mfd(S, t)]) (11)

After obtaining Pm the quality metric Qm can be computed using the following equation:

Qm = �: (12)

The proposed method adopted an RR model to provide better accuracy for the prediction. The model used a simple quality information Qm of a reference image to compute the final quality of the decoded image. Let Q�) represents the

quality value of the decoded images. Let Q�) represents the quality value of a reference image. The maxima and minima of the reference image score thus is predicted by adding and

substracting Q�) with a certain constant Cmax and Cmin respectively. Let Qmax and Qmin represents the maxima and

minima of the reference quality score Q�), the following equations thus applied to compute the raw final quality metric Q final of a decoded or distorted image:

Q . - Q(r) _ C . m1,n � m ffi1,n (d) Q Qm - Qmin final = Q _ Q . max ffi1,n

978-1-4577-1460-3/11/$26.00 ©2011 IEEE

(13)

(14)

(15)

Note that a higher Q final will imply a more distorted or degraded image that corresponds to a low quality. On the other hand, a lower Qfinal implies a better quality of images.

To take into account the non-linearity characteristics of the HVS, a non linear mapping is performed to the raw final quality metric Q final to get the subjective DMOS value using equation specified by [11]. The 4-parameters logistic curve is used as the non-linear mapping which is defined as:

� b1 - b2 Qfinal = { b } + b2

1 + exp 3�Qfinal

Ib41

(16)

given b1, b2, b3, and b4 are the parameters of logistic curve to be estimated using the subjective data. The final estimated quality metric Q final is used as our reduced-reference quality index.

III. EXPERIMENTAL RESULT

LIVE Image Database Release 2 [12] is utilised to test our proposed method. We used all the images from JPEG and JPEG2000 categories (some of the image samples are illustrated in Figure 2). We set the parameters of the algorithm as c5e = 42, c5j,ow = 0, c5!high = c5e, Wd = 10, C = 6.2, Cmin = 0.2, and Cmax = 1.8. All of these parameters are estimated using experimental method and we choose the parameters that yield the best result.

For the experiment, we divided the entire 344 test images in LIVE Image Database Release 2 (JPEG and JPEG2000) into training set (165 images) and validation set (179 images). Using the training set we computed the non-linear regression coefficients as b1 = 63.4870, b2 = -3.5350, b3 = 0.2975, and b4 = 0.3946. Following Video Quality Experts Group (VQEG) [11], we assessed our objective quality metric pre­diction accuracy and prediction mono tonicity using Pearson and Spearman correlation coefficients, respectively. The per­formance results of our proposed method are given in Table I.

Comparison of the proposed method with other reduced re­ference and full reference methods is also presented in Table I. LHS, Gain, and Loss are categorized as reduced reference methods [3] whilst PSNR and FR blockiness detector [10] are full reference. A quick look on Table I might suggest that the performance of the proposed method is slightly below the performance of LHS quality metric. However, if we take a closer examination on the LHS method [3] it is worth noting that the LHS needs a typically 330 real numbers (represented as a matrix) as the reduced reference information according to the size of the images when performing quality assessment of images from LIVE Image Database Release 2. Compared to the size of side information of the LHS method, our proposed method here is superior because it only requires a scalar as

the reduced reference information; i.e., Q�) that represents the quality score of the reference image which is computed blindly by the proposed algorithm. This will provide more significant benefit when dealing with larger image size because data reduction of the LHS is largely dependent on image size. According to [3] the LHS will typically provide 1/1024 data

95

(a)

(c)

(e)

2011 2nd International Conference on Instrumentation Control and Automation 15-17 November 2011, Bandung, Indonesia

Fig. 2. Sample of images used in the experiment

(b)

(d)

(f)

978-1-4577-1460-3/11/$26.00 ©2011 IEEE 96

2011 2nd International Conference on Instrumentation Control and Automation 15-17 November 2011, Bandung, Indonesia

TABLE I PREDICTION PERFORMANCE IN TERMS OF PEARSON CORRELATION (P C) AND SPEARMAN RANK CORRELATION (SC) OF OUR PROPOSED METHOD AND

COMPARISON WITH OTHER METHODS.

Model JPEG (PC) I JPEG(SC) I JP 2K(PC) I JP2K(SC) ALL(PC) I ALL(SC) I P roposed 0.94 0.89 Reduced-reference LHS [3] 0.98 0.97 Gain from LHS 0.85 0.93 Loss from LHS 0.92 0.93 FR blockiness detector [!O] 0.91 0.90 PSNR 0.88 0.89

reduction compared to original bit rates of the image, and this ratio will become larger when the bit rate is higher. Such increase in the size of side information does not apply to the method presented here. Besides the LHS, it can be easily verified from Table I that the proposed method outperforms all other reduced reference and full reference methods (Gain, Loss, PSNR, and FR blockiness detector) in terms of the correlations with respect to the subjective data.

Figure 3 illustrates the scatter plot of the proposed me­thod against subjective data (DMOS) of the LIVE Image Database Release 2. The scatter plot consists of training sets and validation sets. The correlation between predicted quality score (DMOSp) and the DMOS subjective data shows a good linearity as illustrated in the Figure 3 either for training and validation set.

IV. CONCLUSION

We proposed a RR model for performing JPEG and JPEG2000 image quality assessment. We used the Haar

90

80

70 + + 60 +

(/) *rl � 50

j*� :� � * 0

40 + t -41* f** � 30 *t� ++ ** + * validation set 20 + train set

1 �L5--�2LO--�2L5--�3LO--�3�5---4�O---4�5�- 5�O��55��60��65 DMOSp

Fig. 3. Fitted prediction DMOS plotted against DMOS from LIVE Image Database Release 2.

978-1-4577 -1460-3/11/$26.00 ©2011 IEEE

0.94 0.95 0.94 0.92 0.96 0.93 0.96 0.95 0.87 0.89 0.67 0.89 0.93 0.94 0.92 0.93 0.60 0.60 0.74 0.74 0.89 0.90 0.87 0.90

Wavelet Decomposition to extract the reduced reference data. The proposed method only uses one real number as the reduced reference information for quality assessment purposes. Whilst the performance of the proposed method appears slightly below the LHS method, it is worth pointing out that our proposed method is superior because it provides additional data reduction starting from 1/330 that varies depends on the size of the images compared to the LHS method. Besides LHS, we have shown that our proposed method also outperforms other Reduced Reference and Full Reference methods.

REFERENCES

[I] Z. Wang and A. C. Bovik, "A universal image quality index," IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81-84, 2002.

[2] Z. Wang, A. C. Bovik, H. R. Seikh, and E. P. Simoncelli, "Image quality assessment : From error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.

[3] 1. P. Gunawan and M. Ghanbari, "Image quality assessment based on harmonics gain/loss information," in IEEE International Conference on Image Processing, vol. 1,2005, pp. 429-432.

[4] R. Liu, Z. Li, and J. Jia, "Image partial blur detection and classification," in Computer Vision and Pattern Recognition, 2008.

[5] P. Marziliano, F. Dufaux, S. Winkler, and T. Ebrahimi, "No-reference perceptual blur metric," in IEEE International Conference on Image Processing, 2002, pp. 56-70.

[6] Z. Wang, H. R. Seikh, and A. C. Bovik, "No-reference perceptual quality assessment of jpeg compressed image," in IEEE International Conference on Image Processing, 2002.

[7] M. G. Choi, 1. H. Jung, and J. W. Jeon, "No-reference image quality assessment using blur and noise," International Journal of Computer Science and Engineering, vol. 3:2, pp. 76-80, 2009.

[8] Y. Horita, S. Arata, and T. Murai, "No-reference image quality assess­ment for JPEGIJPEG2000 coding," in European Conference on Signal Processing, vol. 1,2004, pp. 1301-1304.

[9] H. Tong, M. Li, H. Zhang, and C. Zhang, "Blur detection for digital images using wavelet transform," in IEEE Conference of Multimedia and Expo, vol. 17-20, 2004.

[I 0] K. T. Tan, "Objective picture quality measurement for mpeg-2 coded video," P h.D. Thesis, Department of Electronic Systems Engineering, University of Essex, 2000.

[11] "RRNR-TV group test plan draft version 1.6," [Online] Available http://www.vqeg.org, Video Quality Expert Group, Tech. Rep., May 2004.

[12] H. R. Seikh, Z. Wang, L. Cormack, and A. C. Bovik, "Live image quality assessment database release 2," Available [Online], 2006. [Online]. Available: http://1ive.ece.utexas.edulresearchlquality

97