1
*Emil BashkanskyTamar Gadrich
ORT Braude College of
Engineering, Israel
ENBIS-11 Coimbra, Portugal, September 2011, 11:50 – 13:20
2
3
4
5
Examples of ordinal scale usageExamples of ordinal scale usage
DAILY LIFEDAILY LIFE MEDICINEMEDICINEQUALITY QUALITY
MANAGEMENTMANAGEMENT EngineeringEngineering
• Sports results:
a win, tie, loss• Voting results:
pro, against,
abstain• Academic
ranks• …
• Rankin score
(RS) - level of
disability
following a
stroke • Side effect
severity• …
• Quality level
estimation and
sorting• Customer
satisfaction
surveys • Ratings of
wine colour,
aroma and
taste • FMECA• …
• The Mohs
scale of
mineral
hardness• Dry-chemistry
dipsticks (e.g.,
urine test)• The Beaufort
wind force
scale• ....
6
* ISO/IEC Guide 99: International vocabulary of metrology —
“Basic and general concepts and associated terms (VIM)”
7
Classic continual:
The probability density function pdf (Y/X) of receiving result Y, given the true value of the measurand X , in it's simplest form:
pdf (Y/X) = Normal (X+bias, ) Ordinal:
The conditional probabilities that an object will be classified as level j, given that its actual/true level is i .
2
/ (1 , )j i i j mP
/1
1m
j ij
P
8
1 0 ... 0
0 1 ... 0
0 0 ... 1
P I
9
Factual
Actu
al + -+-
1-α α
1- ββ
10
(4)1 0 0.5 0.5 0 1 1 0
(1) (2) (3)0 1 0.5 0.5 1 0 1 0
P P P P
11
The likelihood that a measured level j is
received, whereas the true level is i
1
//
/
m
i
p Pi j iQ i j
p Pi j i
1
//
/
m
i
Pj iQi j
Pj i
/1
1m
i ji
Q
12
1
1
1
11
1
Q̂
13
/E P Ii j i jj i
1 1( )
2
2/
m m
i jP I
Am
i jj i
(0 1)A
2 20.5 ( )A
Error matrix:
A
15
16
Blair & Lacy (2000)
)4
1(
)1(1
1
m
FFVAR
m
k kk
0 1VAR
17
1
1/ /(1 )
4 1 ,1
[ ]m
kk i k iF F
VAR i j mmi
1
2
4 (1 )
4 (1 )
VAR
VAR
VAR
/ /1
k
k i j ij
F P
- the expected cumulative frequency of data/items classified up to the k-th category, given that its actual/true level is i
18
1
2/ .
1 1 1
1
1 4
K K K
i i i k i ki k i
totalVAR p VAR p F FK
. /1
K
k i k ii
F p F
- the expected cumulative frequency of items belonging up to the k-th category after measurement
ORDANOVA: DECOMPOSITION OF TOTAL DISPERSION AFTER MEASUREMENT/CLASSIFICATION
19
20
)(iba
- conditional joint probability of sorting the measured object to the a-th level by the first MS (called A), and the b-th level by the second MS (called B), given the actual/true category i
111
)(
m
b
m
a
iba
21
A & B MSs classification matrices
( ) ( )
1. /
m
ba a i ab
i iP
( ) ( )
1. /
m
ab b i ab
i iP
22
( )
1
m
i
piab abi
pi - the probability that an object being measured
relates to category i, ( )1ip
ab - the joint probability of sorting the item as a by the first measurement system (A) and b by the second measurement system (B).
23
MODIFIED KAPPA MEASURE OF AGREEMENT
1
1( )
actual agreement agreement by chance
agreement by chance
agreement by chance
P P
P
P m
m 2 3 4 5 6 7 8 9 10 20 100
0
0.2
5
0.3
3
0.3
8 0.4 0.42 0.43 0.44 0.44
0.4
7 0.5
When a half of all items are correctly classified:
24
1
1
11
mi
kki k
m
m
25
1
1
11
m
kkk
m
m
1
mi
ii
p
26
B B
A
ABn – observed number of
units in which the feature was
found by both A and by B
ABn – observed number of
units in which the feature was
found by A but not by B
A
ABn – observed number of
units in which the feature was
not found by A but was found
by B
ABn – number of units in
which the feature was missed by
both A and B is unknown!
Binary case example
2
1ABABn n n
27
QUALITY CATEGORY
SOLUBLE SOLIDS
CONTENT(SSC)
TITRATABLE ACIDITY
(TA)PH
TOTALSUGARS
MASS FRACTIONS
SKIN COLOR("A" VALUE)
FLESH FIRMNESS
MASS
(%) (%) (G/KG) (N) (G)HIGH (TYPE 1) 11.2-
15.30.51-1.01
3.6-3.9 80-110 5-29 7-28 >105
MEDIUM (TYPE 2)
11.2-14.8
0.66-1.16
3.5-3.8 50-80 13-30 25-60 78-115
LOW (TYPE 3) <12 >1.1 <3.6 <50 >25 >55 <85
Typical relation between quality level and commonly used chemical/physical features for yellow-flesh nectarines
28
i 1 2 3
ip 0.53 0.27 0.2
i
0.84 0.03 0.02
0.02 0.04 0.02
0.02 0.01 0.00
0.04 0.02 0.01
0.03 0.76 0.06
0.01 0.07 0.00
0.01 0.02 0.02
0.03 0.02 0.10
0.03 0.10 0.67
i 0.82
0.70
0.55
0.89 0.08 0.03
ˆ 0.07 0.85 0.08
0.05 0.15 0.80AP
0.88 0.08 0.04ˆ 0.08 0.85 0.07
0.07 0.14 0.79BP
( ) 0.792A ( ) 0.781B
weighted total kappa equals 0.734
Ternary scale example(fruit quality classification)
0.458 0.0253 0.0173
0.0247 0.2304 0.0468
0.0193 0.0442 0.134
33
Let's consider arbitrary ordinal scale with m categories and suppose, that n repeated measurements of the same object were performed resulting in vector:
(n=n1+n2+…+nm)
},...,{21 mnnnn
34
/ /11
1 1
! !exp ln
! !
jm m
n
i j i j j im mjj
j jj j
n nL P n P
n n
The maximum likelihood estimation must be made in favor of such, most plausible i , that maximizes the scalar product:
/1
[ ln( )]j i
m
jj
n P n lni
1 2{ln( ), ln( )...ln( )}/ / /ln P P Pi i m ii
35
(1)0.89 0.08 0.03
ˆ 0.07 0.85 0.08
0.06 0.14 0.80
P
0.254
0.2774
0.4328
pVAR
36
50.9997 0.0002 9.9 10
(10 )0.0003 0.9971 0.0026
53.1 10 0.0014 0.9986
ˆ
P 10
0.0008
0.0058
0.0028P
VAR
(1)0.89 0.08 0.03
ˆ 0.07 0.85 0.08
0.06 0.14 0.80
P0.254
0.2774
0.4328
pVAR
38
1.On an ordinal measurement scale the essential for evaluating the error, repeatability and uncertainty of the measurement result base knowledge must be the classification/measurement matrix. Given this matrix, authors introduced a way to calculate the classification/measurement system’s accuracy, precision (repeatability & reproducibility) and uncertainty matrix.
2.In order to estimate comparability and equivalence between measurement results received on an ordinal scale basis, the modified kappa measure is suggested. Three of the most suitable usages of the measure were thoroughly analyzed. The advantage of the proposed measure vs. the traditional one lies in the fact that the former follows the superposition principle: the total measure equals the weighted sum of partial measures for every ordinal category.
3.As it is well known, repeated measurements may improve the quality of the measurement result. When decisions are ML based, one can find how many repetitions are necessary in order to achieve the desired accuracy level using the algorithm suggested by the authors,.
40
E-mail: [email protected]
41