1
From the analytical uncertainty to uncertainty in data interpretation
D. Concordet, J.P. [email protected]
2
Biological data vary according several sources of variations
Biological- species- status (healthy, sick)- breed- format, age...- inter-individual- intra-individual
Metrological- intra- laboratory variations (imprecision)- inter-laboratories variations (bias)
controlled
of interest
not controlled
parasitic
3
Influence of imprecision on the reference interval
Pl-creatinine (mol/L)
10 30 50 70 90 110 130 150
CV=0%
Population distribution of Pl-Creatinine of healthy dogs
4
Influence of imprecision on the reference interval
Pl-creatinine (mol/L)
10 30 50 70 90 110 130 150
CV=0%
CV=10%
Population distribution of Pl-Creatinine of healthy dogs
Y = X + with E (
5
Influence of imprecision on the reference interval
Pl-creatinine (mol/L)
10 30 50 70 90 110 130 150
CV=0%
CV=10%
CV=20%
Population distribution of Pl-Creatinine of healthy dogs
Y = X + with E (
6
Sampling variations of the reference interval
95%
CI
of th
e re
fere
nce
inte
rval
Pl-
crea
tinin
e (
mol
/L)
10
30
50
70
90
110
130
150
170
N=100 dogs
CV=0% CV=10% CV=20%
7
Use of reference interval
Pl-creatinine (mol/L)40 135
Effect of large imprecision
8
Use of reference interval
Pl-creatinine (mol/L)40 135
When the precision is better
9
Replicates decrease the influence of imprecision
Y1 = X +
Y2 = X + with E (i
SD(Yi)= SD(i
Yp = X + p
p
YSD
CV(Yi) = 20 %
p CV(Y)
2 14.1%
3 11.5%
4 10.0%
10
20 40 60 80 100 120 140 160 180
Ref.
Influence of inter laboratories variability on the reference interval
Pl-creatinine (mol/L)
11
20 40 60 80 100 120 140 160 180
Ref.
CV = 10% / Ref
Influence of inter laboratories variability on the reference interval
Pl-creatinine (mol/L)
12
20 40 60 80 100 120 140 160 180
Ref.
CV = 10% / Ref
CV = 20 % / Ref
Influence of inter laboratories variability on the reference interval
Pl-creatinine (mol/L)
13
Reference range of Pl-creatinine11 books of animal clinical biochemistry
0
25
50
75
100
125
150
175
200
225
250P
l-C
reat
inin
e (µ
mo
l/l)
[2] [6] [7] [8] [27] [28] [34] [45] [46] [55] [59]
Lefebvre HP et al. 1998.
35 µmol/l
245 µmol/l
Reference number
14
A “demographic” source of variation
GFR value
0
1
2
3
4
5
Dachshund Labrador Great Dane
GFR
(m
L/kg
/min
)M
ean
SD
8 8 6
GFR value depends on breed
Lefebvre HP unpublished results
15
Influence of the population’s structure
20 70 120 170 220 270
breed 1 = 30%
breed 2 = 30%
breed 3 = 20%
breed 4 = 10%
breed 5 = 5%
breed 6 = 5%
20 70 120 170 220 270
breed 1 = 30%
breed 2 = 30%
breed 3 = 20%
breed 4 = 10%
breed 5 = 5%
breed 6 = 5%
total
130Pl-creatinine (mol/L)
16
Influence of the population’s structure
20 70 120 170 220 270
breed 1 = 16%
breed 2 = 16%
breed 3 = 16%
breed 4 = 16%
breed 5 = 16%
breed 6 = 20%
20 70 120 170 220 270
breed 1 = 16%
breed 2 = 16%
breed 3 = 16%
breed 4 = 16%
breed 5 = 16%
breed 6 = 20%
total
210Pl-creatinine (mol/L)
17
Influence of multiplicity : case of independence
Pl-creatinine (mol/L)40 135
40
95% of healthy animals
95%
of
hea
lth
y an
imal
s
90% of healthy animals
18
Imprecision and multiplicity
Pl-creatinine (mol/L)40 135
40
19
Imprecision and multiplicity
Pl-creatinine (mol/L)40 135
40
CV= 15 %82 % of healthy animals
CV = 15%93% of healthy animals
20
Diagnostic tests : a better way to proceed
20 70 120 170 220 27020 70 120 170 220 270
Diseased animalsWithout the disease
Threshold
Considered as sick
Pl-creatinine (mol/L)
Considered as healthy
Sensitivity :SeSpecificity :Sp
21
Performances of the test
Sensitivity = Se% of diseased animals with a result > threshold
Specificity = Sp% of animals without the disease with a result < threshold
22
Several definitions of specificity
Specificity = Sp
• Sp1 :% of healthy animals with a result < threshold
• Sp2 :% of animals without the disease (healthy + with any other disease) with a result < threshold
23
20 70 120 170 220 270
Several definitions of specificity
Healthy animalsSp1
Threshold
Healthy animals+ with other diseasesSp2
Sp2<<Sp1
24
The operational indices : the predictive values
Spe
ePPV
1Pr1SPr
SPr
Spe
SpNPV
Pr1S1Pr
Pr1
Positive Predictive Value (PPV) : Probability that the animal has the disease when its result >threshold
Negative Predictive Value (NPV) : Probability that the animal has not the disease when the result < threshold
25
The clinician experience : Pr
Pr = Pre-test probabilityprobability that the animal has the disease
Pr = 1 : The clinician is sure that the animal has the disease
Pr = 0 : The clinician is sure that the animal has not the disease
PPV =1
NPV =0whatever Se and Sp
PPV =0
NPV =1whatever Se and Sp
Positive diagnostic gain : PPV-Pr
Negative diagnostic gain : NPV-(1-Pr)
Pr = 0.5 : The clinician does not know (coin tossing)
26
20 70 120 170 220 270
Sp = 80%
Influence of imprecision
Threshold
Pl-creatinine (mol/L)20 70 120 170 220 270
Se = 92%
healthy1
Sp = 73%
CV = 15 %
Se = 81%
CV = 15%
27
0
0.10.2
0.3
0.40.5
0.6
0.7
0.80.9
1
0 0.2 0.4 0.6 0.8 1
PPV
NPV
Consequences on interpretation
Pre-test probability
PP
V/N
PV
0
0.10.2
0.3
0.40.5
0.6
0.7
0.80.9
1
0 0.2 0.4 0.6 0.8 1
PPV
NPV
PPV CV 15%
NPV CV 15%
28
Sampling variations of sensitivity and specificity
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
0 50 100 150 200 250 300
Sample size : n95%
co
nfi
den
ce in
terv
al o
f S
e=0.
92, S
p=
0.80
N
SSS
N
SSS
12;
12
Se = 0.92
Sp = 0.80
95% confidence interval of Se/ Sp :
29
Consequences on interpretation9
5% c
on
fid
enc
e in
terv
al o
f P
PV
/NP
V
Pre-test probability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
N=50
30
Consequences on interpretation9
5% c
on
fid
enc
e in
terv
al o
f P
PV
/NP
V
Pre-test probability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
N=100
31
Consequences on interpretation9
5% c
on
fid
enc
e in
terv
al o
f P
PV
/NP
V
Pre-test probability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1
N=300
32
Effects of the biological sources of variation
Increase the overall dispersion of the results
Improvement possible if Se and Sp are determined for each level of the factors of variation (e.g. breed)
decrease Se and Sp
decrease PPV and NPV for a fixed pre-test proba
33
The future ?
Individualisation
Blood sample when the animal is young and healthy
Follow-up of the evolution of the appropriate marker
Critical difference
Independent of demographic factors (breed, sex…)
dependent only on intra-individual variability
analytical errors
34
The end