1 Marco Carli VPQM 2006 26/01/2007 ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS Nikolay Ponomarenko (*), Flavia Silvestri(**), Karen

1

Marco Carli VPQM 2006 26/01/2007

ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS

FUNCTIONS

Nikolay Ponomarenko (*), Flavia Silvestri(**),Karen Egiazarian (***), Marco Carli (**),

Jaakko Astola (***) and Vladimir Lukin (*)

(*) National Aerospace University, Kharkov, Ukraine(**) University of Rome "Roma TRE", Rome, Italy

(***) Tampere University of Technology, Tampere, Finland

2


Outline

Outline

1. Introduction2. Proposed model of between-coefficient contrast masking of DCT basis functions3. Modification of PSNR using a new masking model 4. MATLAB implementation of the proposed measure5. A set of test images for comparative analysis for taking into account the masking

effect in quality metrics6. Subjective experiment to test quality measures7. Results of the experiment8. Examples of quality assessment of test images9. Example of use of the proposed model to masking noise on a real image10.Summary and Conclusion

3


Introduction

Human visual sensitivity varies as a function of several key image properties, such as:

Light levelSpatial frequency ColorLocal image contrast EccentricityTemporal frequency

Goal of the research:Efficient accounting for local image contrast using a model of between-coefficient contrast masking of DCT basis functions

Masking model can be used in :

Image and video compression Image filtering Digital watermarkingValidation of effectiveness of image processing methods

Requirements to the model:Images compressed (filtered or processed) with accounting the model can be visualized in unknown illumination conditions, monitor brightness, distance to the monitor, viewing angle, etc. Thus such model should operate by only some averaged parameters of image visualization

4


Proposed model of between-coefficient contrast masking of DCT basis functions

Let us denote a weighted energy of DCT coefficients of an image block 8x8 as Ew(X):

(1)

where Xij is a DCT coefficient with indices i,j, Cij is a correcting factor determined by the CSF. The DCT coefficients X and Y are visually undistinguished if Ew(X-Y) < max(Ew(X)/16, Ew(Y)/16), where Ew(X)/16 is a masking effect Em of DCT coefficients X (normalizing factor 16 has been selected experimentally).

7

0i

7

0jij

2ijw CX)X(E

Reducing of the masking effect due to an edge presence in the analyzed image block: we propose to reduce a masking effect for a block D proportionally to the local variances V(.) in blocks D1, D2, D3, D4 in comparison to the entire block:

Em(D) = Ew(D)δ(D)/16, (2)

where δ(D) = (V(D1)+V(D2)+V(D3)+V(D4))/4V(D), V(D) is the variance of the pixel values in block D.

5


Proposed model of between-coefficient contrast masking of DCT basis functions

i\j 0 1 2 3 4 5 6 7

0 0 0.8264 1.0000 0.3906 0.1736 0.0625 0.0384 0.0269

1 0.6944 0.6944 0.5102 0.2770 0.1479 0.0297 0.0278 0.0331

2 0.5102 0.5917 0.3906 0.1736 0.0625 0.0308 0.0210 0.0319

3 0.5102 0.3460 0.2066 0.1189 0.0384 0.0132 0.0156 0.0260

4 0.3086 0.2066 0.0730 0.0319 0.0216 0.0084 0.0094 0.0169

5 0.1736 0.0816 0.0331 0.0244 0.0152 0.0092 0.0078 0.0118

6 0.0416 0.0244 0.0164 0.0132 0.0094 0.0068 0.0069 0.0098

7 0.0193 0.0118 0.0111 0.0104 0.0080 0.0100 0.0094 0.0102

Values of Cij

16 11 10 16 24 40 51 61

12 12 14 19 26 58 60 55

14 13 16 24 40 57 69 56

14 17 22 29 51 87 80 62

18 22 37 56 68 109 103 77

24 35 55 64 81 104 113 92

49 64 78 87 103 121 120 101

72 92 95 98 112 100 103 99

JPEG Quantization table of Y component

Values of Cij have been obtained using the quantization table for the color component Y of JPEG (the values of quantization table JPEG have been normalized by 10 and squared)

6


Modification of PSNR using a new masking model

A basis of the proposed metric is a PSNR-HVS (Egiazarian K., Astola J., Ponomarenko N., Lukin V., Battisti F., Carli M. “New full-reference quality metrics based on HVS”, CD-ROM Proceedings of the Second International Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p).

Block 8x8of original

image

Block 8x8of distorted

image

DCT ofdifferencebetween

pixelvalues

Reductionby value of

contrastmasking

MSEHcalculationof the block

Flow-chart of PSNR-HVS-M calculation

Reduction by value of contrast masking in accordance to the proposed model is carried out in the following manner. First, the maximal masking effect Emax is calculated as max(Em(Xe), Em(Xd)) where Xe and Xd are the DCT coefficients of a original image block and a distorted image block, respectively. Then, the visible difference between Xe and Xd is determined as:

X∆ij =

where Enorm is .

otherwise,C/EXX

C/EXX,C/EXX

C/EXX,0

0j,0i,XX

ijnormdijeij

ijnormdijeijijnormdijeij

ijnormdijeij

dijeij

64/Emax

7


MATLAB implementation of the proposed measure

The MATLAB implementation of PSNR-HVS-M is available on www.cs.tut.fi/~ponom/psnrhvsm.htm

8


A set of test images for comparative analysis for taking into account the

masking effect in quality metricsWhile creating an image test set we took into consideration the following:

Such set should contain images with both spatially uncorrelated and correlated noise (the latter one is typical for images formed by digital cameras and is more visible for humans);

The set should contain images with noise distributed spatially uniformly and with noise which is masked or unmasked (concentrated in regions with maximal and minimal masking properties, respectively);

The set is to be maximally simple for visual comparison by humans (because of this in our set we used only three values of noise variance σ2 and a total number of distorted test images was 2x3x3 = 18 images).

Original test images having a lot of different type regions with high masking effect

9


Subjective experiment to test quality measures

Result of the experiment: the test image set ordered according to subjective visual quality.Number of observers: 155 (45 from Finland, 43 from Italy, 67 from Ukraine). Number of comparisons of visual appearance of test images: 8192 (on average 53 for each observer). 17” or 19” Monitor Resolution: 1152x864 pixels.Number of experiments carried out using CRT monitors: 128.Number of experiments carried out using LCD monitors: 27.

Group of observers Spearman correlation Kendall correlation

Finland – Italy 0.996 0.895

Finland – Ukraine 0.996 0.935

Italy - Ukraine 0.997 0.961

CRT - LCD 0.998 0.922

Cross correlation factors

10


Results of the experiment

Measure Reference Spearman correlation

Kendall correlation

PSNR-HVS-M This paper 0.984 0.948

PSNR-HVS Egiazarian K., Astola J., Ponomarenko N., Lukin V., Battisti F., Carli M. “New full-reference quality metrics based on HVS”, CD-ROM Proceedings of the Second Intern. Workshop on Video Processing and Quality Metrics, Scottsdale, USA, 2006, 4 p

0.895 0.712

NQMDamera-Venkata N., Kite T., Geisler W., Evans B. and Bovik A. "Image Quality Assessment Based on a Degradation Model", IEEE Trans. on Image Processing, Vol. 9, 2000, pp. 636-650

0.857 0.673

DCTune Solomon J. A., Watson A. B., and Ahumada A. “Visibility of DCT basis functions: Effects of contrast masking”. Proc. of Data Compression Conf., 1994, pp. 361-370http://vision.arc.nasa.gov/dctune/ - DCTune 2.0 page

0.829 0.712

UQI Wang Z., Bovik A. “A universal image quality index”, IEEE Signal Processing Letters, vol. 9, March, 2002, pp. 81–84 0.550 0.438

PSNR Peak Signal to Noise Ratio 0.537 0.359VQM Xiao F. “DCT-based Video Quality Evaluation”, Final Project for EE392J, 2000 0.441 0.281

SSIM Wang Z., Bovik A., Sheikh H., Simoncelli E. “Image quality assessment: from error visibility to structural similarity”, IEEE Trans. on Image Proc., vol.13, 2004, pp.600-612 0.406 0.358

VIF Sheikh H. R. and Bovik A. C., "Image Information and Visual Quality", IEEE Transactions on Image Processing, vol. 15, February, 2006, pp. 430-444 0.377 0.255

PQSMiyahara, M., Kotani, K., Algazi, V.R. ”Objective picture quality scale (PQS) for image coding”, IEEE Transactions on Communications, vol. 46, issue 9, 1998, pp. 1215-1226

0.302 0.242

11


Examples of quality assessment of test images

DCTune = 24.9, PSNR-HVS-M = 33.20 dBPSNR-HVS-M says: “This is better!”

DCTune = 24.5, PSNR-HVS-M = 29.31 dBDCTune says: “This is better!”

12


Examples of quality assessment of test images

SSIM = 0.80, PSNR-HVS-M = 25.50 dBSSIM says: “This is better!”

SSIM = 0.79, PSNR-HVS-M = 31.29 dBPSNR-HVS-M says: “This is better!”

13


Example of use of the proposed model to masking noise on a real image

Original test image Baboon The image with masked noise, PSNR=26.18 dB, MSE=158,

PSNR-HVS=34.43 dB, PSNR-HVS-M=51.67 dB

14


Summary and Conclusion

Summary

A simple and efficient model of between-coefficient contrast masking of DCT basis functions is proposed;

A modification of PSNR that takes into account this masking model is proposed;

Subjective experiments on comparison of known quality metrics are carried out;

Conclusions

The proposed measure based on the designed masking model has demonstrated the best correspondence to the results of the subjective experiments. However for providing more reliable conclusions on efficiency of the proposed model it is necessary to carry out additional more extensive experiments and research.

The proposed test set has allowed to demonstrate drawbacks of many well known metrics that do not fully or even badly correspond to human visual perception.

Documents

1 Marco Carli VPQM 2006 26/01/2007 ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS Nikolay Ponomarenko (*), Flavia Silvestri(**), Karen