Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Image Quality Measurements:Necessity, Numbers
and '....nesses'.Ralph Jacobson
Emeritus Professor of Imaging ScienceUniversity of Westminster
Digital Futures 2009: Image Physics & PsychophysicsNovember 3rd, The Institute of Physics, London
Overview• Need for measurement• Quality defined• Historical perspective and transference• Imaging with numbers but without too many
numbers!• The ‘Nesses’• Physical & Psychophysical measures
psychophysics/psychometricsrelationships with observations (HVS),
• Conclusions
Why Measure?To provide a quantitative basis for the useand comparison of products and devices
To provide a means of improving quality
To provide a means for modelling systemsand deciding what aspects need improvingTo understand the systems
What is Quality?Degree of excellence, relative nature or, kind or,character. (OED)
All those features of product (or service) which arerequired by the customer. (ISO 9000)
The integrated set of perceptions of the overalldegree of excellence of an image.
(Engeldrum, 2000)Others to be defined later
The ‘Nesses’Proposed by Engledrum as a characteristics ofimages that we sense (see).
Perceptual attributes :Brightness HuenessChromaness LightnessColorfulness NaturalnessContrastness SharpnessFineness (detail) TexturenessGraininess (noisiness) Usefulness
Prototyping
PsychologicalEffects
Brain
Brain
Memory
PhysicalEffects
PhysiologicalEffects
Colour ofGrass
InPicture
MIND
UsualColour of
Grass
COLOUR PICTURE
ORIGINAL OBJECTS
Hunt, 1967
PrototypingSubjective judgement made by a mentalcomparison of an external image withimage impressions stored andremembered more or less distinctly by theobserver, who allows for loss of detail inareas too small to be resolved by the eye.
Schade, 1975
Acceptability/Perceptibility
Fidelity (Perceptibility):Visibility of a factor, orDiscrimination between images
Klein, 1993, Farrell 1996
Quality (Acceptability):Preference of one image over an other, ordegree to which a factor is bothersome
Acceptability/Perceptibility
0 4 8
16 6432
20R
-20R
30R
-30R
0
10R
-10R
Acceptability/Perceptibility
Naturalness
The degree of apparent match betweenthe reproduced image and the internalreferences, e.g. memory prototypes
Endrikhovski, 2002
Naturalness
OriginalColoursshifted toyellow -Sunset
Coloursshifted toblue -Moonscape
Coloursshifted topurple -Unnatural
Endrikhovski, 2002
Visuo-Cognitive Processing
Janssen, 2001
Image perception
Internalrepresentation interpretation
Interpretedscene
semanticprocessingtask response
memoryrepresentation
UsefulnessRequires modification according toapplication – usefulness or fitness forpurpose:The degree of apparent suitability of thereproduced image to satisfy thecorresponding task.
Usefulness
Examples of usefulness criteria might bethe ability to:
• resolve a defined detail• discriminate one area from another• successfully diagnose or interpret
UsefulnessDental X-ray
Powell, May 2004
Usefulness
Image Quality Approaches
virtual reality
Usefulness
Fidelity
Naturalness
holiday pictures
fine art
Mars imagesmedical images
advertisement
Endrikhovski, 2002
Problems
Image quality has no single unique definition, yetas observers, we are able to decide almostinstantly whether a particular image is of good orpoor quality but for us to quantify how good animage is, and the scale of quality is far moredifficult.
Multidimensional in character
Additional ‘nesses’ for digital imaging
Digital ‘Nesses’ ArtefactsEffect Possible CausesContouring poor bit-depthBlocking (Gibbs effects) compressionJaggies inappropriate pixel sizeRinging sharpening, compressionAliasing samplingStreaking pixel-to-pixel non uniformityPatterning poor spatial resolution,
ditheringColour misregistration images from different
channels not geometricallyidentical
Measurement
• Physical measures• Psychophysical aspects• Inter – relationships
MeasurementsAttribute Physical MeasureColour Spectral data, Chromaticities,
Colour spaces etc.Tone (contrast) Gamma, Density, PV, Characteristic
Curve, Tone Reproduction Curve,Density Histogram, OECF
Resolution (detail) Resolving Power, l/mm, dpi, ppiSharpness (edges) Acutance, PSF, OTF, LSF, MTF, SFRNoise Granularity, Standard Deviation
(graininess, electronic noise)Noise-Power (Wiener) Spectrum
Information Entropy, Information Capacity
Physical MeasurementsBeginnings
Fundamental PhysicalMeasures
(1975)
(1974)
Opto-ElectronicConversion Function (OECF)
050
100150200250
0 0.5 1.0 1.5 2.0 2.5 3.0
Log Relative Exposure
Mean
Outp
ut Le
vel
Methods for MeasuringOpto-electronic ConversionFunctions (OECF): ISO 14524The standard describes methods formeasuring and reporting the relationshipbetween the input scene log luminancevalues and the digital output levels for adigital camera.
Determination of ISO SpeedISO 12232
• ISO speed of a digitalcamera attempt to matchISO rating of film camerasystems
• provides a method formeasuring and reportingISO speed metrics thatcorrelate with imagequality
(Saturation and noise based)
108 109 1010 1011 1012 1013
106
105
104
103
102
101
100
SaturationLevel
EXPOSURE (Photons/sec/cm2 )
SIGN
AL (e
lectro
ns)
Signal
Photon Sho
t Noise
Read Noise
Dark Noise
108 109 1010 1011 1012 1013
106
105
104
103
102
101
100108 109 1010 1011 1012 1013
106
105
104
103
102
101
100
2 )
Signal
Photon Sho
t Noise
Read Noise
Dark Noise
Bestexposure
Muammar, 2008
ISO Test Chart ISO 15739
18% reflectance for calculating noise
Patches used for calculatingincremental gain
Black referencefor calculatingDSC dynamic
range
OECF
Measurements of VisualResolution & SFR: ISO 12233
0.00
0.20
0.40
0.60
0.80
1.00
0 1000 2000 3000 4000 5000Frequency [LW/PH]
Spati
al Fr
eque
ncy R
espo
nse
SFRhorizontaSFRverticalAverage
Aliasing Region
SFR
Visual
Universal Test Target
Image Engineering Dietmar Wueller , 1 Sep 2009
Psychophysics
Science of investigations of the quantitativerelationships between physical events andthe corresponding psychological events.
i.e., quantitative relationships betweenstimuli and responses
PsychometricsProvide quantification of qualitative attributes
2000 2002
Scaling Methods• Pair Comparisons• Categorical Methods• Rank Order• Interval Judgment• Ratio Judgment• Magnitude Estimation• Pass/fail
Part 1: Overview of psychophysical elementsDescribes how the standard could be extendedto include other psychometric techniques.
Part 2: Triplet comparison method Method forsubjective image quality assessment
Part 3: Quality Ruler Method Describes amethod for generating quality rulers varying insharpness.
Psychophysical ExperimentalMethod to Estimate Image
Quality: ISO 20462
Just Noticeable Difference (JND)Attribute JNDmeasure of the detectability of appearancevariations, corresponding to a stimulus differencethat leads to a 75:25 proportion of responses in apaired comparisonQuality JNDmeasure of the significance or importance ofquality variations, corresponding to a stimulusdifference that leads to a 75:25 proportion ofresponses in a paired comparison task in whichmultivariate stimuli pairs are assessed in terms ofoverall image quality
Just Noticeable Difference (JND)
JND
Engledrum, 2000
Metrics
Visual Image Quality Metric (VIQM):Single numbers (figures of merit) derivedfrom physical measurements for the systemand the eye which relate to perceptions ofimage quality
Image Quality Metric (IQM):Single numbers (figures of merit) derivedfrom physical measurements for thesystem which relate to perceptions ofimage quality
Image Quality Metrics
HVS +IQC
HVSOutputdevice
Fidelity
QualityDistortion
ImageData
Distortion(IQMs)
(VIQMs)
Multivariate Metric (Minkowski)
IQM given by:
[0.413(sharpness)-3.14 + 0.422 (10-graininess)-3.4]-1/3.4 – 0.532
Bartleson, 1982
Minkowski Metric
Engledrum, 2002
Image Quality Metric (IQM)Colour Difference:
CIE 1976 CIELAB Colour Difference:
∆∆∆∆ *)(*)(*)(* 222 baLEab
Later colour difference formulae (CIE94,CIEDE2000) do include viewing conditions
• Does not take viewing conditions into account
Variable Exponent MinkowskiMetric
∆Q = ( Σ ∆Q ε )1/ε
In JNDs of quality change
Keelan, 2000
Visual Image Quality Metrics(VIQMs)
VIQM
Contrast(Gamma, Tone) ColourVisualsystem
MTF(SharpnessResolution)
Noise
Generally are based on some form of signal to noise ratio
VIQM Approach
VIQM
020406080
100
0 20 40 60 80 100
JND
PsychophysicalObservations
VisualsystemNoise
Contrast(Gamma, Tone)
MTF(SharpnessResolution)
Sharpness and NoiseMultivariate IQ
012345678
0 1 2 3 4 5 6 7 8
r 2 = 0.944
Perceived Quality
Calcu
lated
IQ
Stone, Jacobson & Attridge 1994
du](u)N+(u)MN(u)(u)M(u)MS(u)+[1=SNR
eye2eye
2eye
2sysln∫
Higgins, 1977
Visual Image Quality MetricBarten’s Square Root Integral (SQRI):
)d(ln)()(
)2ln(1
tu
maxu
minuumuMJ ∫=
Barten, 1990
MTFdisplaysystem
modulationthreshold function
of the eye
Visual Image Quality Metric
udu
)()()()()(
2lnSQRI 225.0
0 22
21 max kuNuMuNk
uMuSk ueyeeye
eyen +
+= ∫
)()........()()({ 2210 uMuMuSuS n= }
(Barten’s SQRIn reformulated by Töpfer & Jacobson, 1993)
Square Root Integral with Noise (SQRIn)
VIQM (SQRIn ) and Sharpness
Töpfer & Jacobson,1993
r 2 = 0.9950
20
40
60
80
100
0 20 40 60 80 100Measured SQRIn (JND)
AMTA
(JND
)
Measured
Predicted
udu
)()()()()(
2lnSQRI 225.0
0 22
21 max kuNuMuNk
uMuSk ueyeeye
eyen +
+= ∫ udu)()()( )()(2lnSQRI 2
25.0
0 22
21 max kuNuMuNk
uMuSk ueyeeye
eyen +
+= ∫
Examples of VIQMsMTFs, Signals, Sharpness, Noise etc: Schade, 1950+System Modulation Transfer Acutance (SMTA) : Crane, 1964Signal-to-Noise Ratio (SNR) : Nelson, 1973Modulation Transfer Acutance (AMTA): Crane, 1983(Quality: graininess and sharpness (Qg/s) : Bartleson, 1982Square-root Integral (SQRI) : Barten, 1990+Perceived Information Capacity (PIC) : Töpfer & Jacobson, 1993Visible Differences Predictor (VDP) : Daly, 1992Visual information and processing: Janssen & Blommaert, 1997+Effective Pictorial Information Capacity (EPIC) : Jenkin et al., 2005Colour Reproduction Index (CRI) : Pointer, 1986+Colour Difference: CIE,1976, CIECAM97s :CIE, 1997, CIEDE2000R-LAB: Fairchild and Berns, 1993S-CIELAB:Zhang et al., 1996Cognition: Usefulness, Naturalness: Endrikhovski et al, 1999+CSF/CIEDE2000, Colour Image Difference Metric:
Johnson and Fairchild , 2002
Metrics Applied to DigitalImages
Mean-Square-Error (MSE, RMSE)Mean-Square-Error after Non-Linearity(MSENL)Mannos-Sakrison (MANNOS)Logarithmic Image Processing (LIP)Distortion Contrast (DCON)Bit Rate (BITS)Mean Intensity (MI)Spectrum Slope (SS)Spectrum Slope over Mean Intensity (SS/MI)Local Contrast (LCON)
Root Mean Square Error (RMSE)
• Cognitive aspects not considered
RMSE = −=
−∑
=
−∑1
0
1 2
0
1
XYf x y f x y
y
Y
x
X( , ) '( , )
( f, f’ are original and changed images at spatial locations x,y, X and Y are total number of horizontal and vertical pixels)
• No account of visual significance
• Output device not considered
Effective PictorialInformation Capacity (EPIC)
1.Determine total MTF of system components and the eye2. Inverse Fourier transform to find effective pixel size3. Determine total noise using effective pixel size as
aperture, including scene dependent noise for distorted(e.g. blurred) images
4.Find Information capacity ( I = n log2m ) in bits5. Convert to bits/steradian from image size and viewing
distanceJenkin, Triantaphillidou and Richardson, 2007
Effective PictorialInformation Capacity (EPIC)
R2 = 0.94
050
100150200250300350400450
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5Subjective Quality Rating
EPIC
(kBy
tes/
ster
adia
n)
Jenkin, Triantaphillidou and Richardson, 2007
Specific approaches
Standards : e.g. NCITS W1.1Image Quality for Printer Systems –
Considerations of:Text and Line quality, Micro uniformity, Macrouniformity, Gloss/Gloss uniformity,Color rendition, Effective tone levels,Effective resolution in pictorials, Adjacency
VIQM Approaches
VIQM
Psychophysicalobservations
Variable ExponentMinkowski Metrics
(C)VDP basedapproaches
Cognitiveapproaches
Visio-cognitive
Information
CIE + spatial∆E approaches
Physicalmeasure,
MTF basedapproaches
Industry ledStandards
approaches
InformationCapacity
approaches
Colour Principles andMeasurement
Colour Reproduction Index(CRI)
• For determinations of absolute colourappearance
• Based on Hunt model for colour vision• Includes viewing conditions• Excludes spatial aspects
Pointer, 1986
Image Quality Index
For prediction of colourreproduction perceived asnatural, unnatural, pleasant orunpleasant for an averageobserver.
Endrikhovski, 1999
Original Processed
Enhancing Perceived Quality
Endrikhovski, 2002
S-CIELAB
ColourImage
Lum
R/G
B/Y
XYZ S-CIELAB
Spatial filter
Zhang & Wandell, 1997
CIECAM02
Pointer 2009
i-CAMTwo input images are given:
an original and a reproduction
The input images aretransformed into an opponentcolour space
The opponent channels arefiltered using contrast sensitivity
functions which are adaptedbased on the spatial
information in the image. Thefiltering decreases information
that is not visible and increasesinformation that is most visible
Models of local attention andlocal contrast are applied to thefiltered images
The filtered images are thenconverted into CIELAB
coordinates and a Pixel-byPixel colour difference
calculated
Johnson and Fairchild, 2002
iCAM
Orfanidoua, Triantaphillidou and Allen, 2008
The ‘…..ishes’More than 50 VIQMs have been proposedSignal to noise will always give goodcorrelations!Use of charts in VIQM measurementsObserversQuestion asked of ObserversStandard observerScenes, Scene Dependency, ROI(salience)Spatial and colour approachesTraditional to digital transfers
Contrast Sensitivity of theHuman Eye
Barten, 1992
00.20.40.60.81.0
0 5 10 15 20 25 30 35 40Spatial frequency (cycles/degree)
Sens
itivity
16.4o, 34 cd/m2
measured values
The Standard ObserverTC1-60 of Division 1 of the CIE terms ofreference:1) To specify a baseline achromatic CSF withits reference conditions and referenceobserver.2) To specify CSF extensions based ondiscrimination thresholds, as well aschromatic CSFs for both detection anddiscrimination.
Chairman: Eugenio Martinez-Urigas
Scene Dependency andCompression
Non-compressed JPEG 60:1 JPEG2000 60:1
S. Triantaphillidou, E. Allen, R.E. Jacobson and G.G. Attridge, 2002
Standard Images
Lena
ISO 12640-3: 20078 natural scenes
Importance of Lightness inImage Quality
Original
Lightnesssharp
Huesharp
Saturationsharp
Allunsharp
Importance of Lightness inImage Quality
Original ScrambledforHUE
Scrambledfor
CHROMAScrambled
forLIGHTNESS
Endrikhovski, 2002
Digital Transfer Difficulties
• Artefacts• Anisotropy• Non-linearity• Non-stationary
Digital systems are difficult to deal withusing conventional mathematical processes
• Some success in relating complexphysical measures to perceptions ofimage quality
• Provide modelling approach• Require extensive validation• Research & development tool• Move away from VIQM single number
approach to determining metrics by processsteps (e.g.EPIC, S-CIELAB, i-CAM)
Conclusions VIQMs
• All approaches work and are applied!• Provide useful data for modelling system
changes, benchmarking andimprovements
• Lead to determination of fundamentalparameters
• Used by all manufacturers and systemdevelopers
• Are providing new insights and impetus inperception and measurement of imagequality
General Conclusions