Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Proficiency Test: Gun Shot Residues 2005Proficiency Test: Gun Shot Residues 2005
Statistical analysis of the results
PD Dr. Steffen Uhlig
Statistical analysis of the results
PD Dr. Steffen Uhlig
May 2005
22 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
quo data provides scientific services for establishing measurement methods and quality assurance� Interlaboratory studies
� In-house studies
� Optimisation projects
� Development of validation criteria
� Training and workshops
quo data provides software� for interlaboratory studies
� for in-house validation
� for the optimization of measurement methods
� ...
quo data develops workstations for the analysis of organic compounds� ELISA
� Yeast assays
Products and services of quo dataProducts and services of quo data
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
33 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Method assessment
1. Standard Deviation of Particle Number
2. Method detection capability
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
44 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Standard Deviation of Particle Number
• Target value = no. of particles per size class – 1
• Standard deviation of the number of correctly detected particles according to DIN 38302 A 45 = CD ISO 20 612
(ProLab 2006):
⇒ Robust statistical method (outlier resistant), hence nooutlier elimination necessary
⇒ Classical statistical requirement such as normaldistribution assumption not required
⇒ Specifically developed for asymmetrical and discrete distributions
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
55 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Standard Deviation of Particle Numbers
total no.
of part.
ref. s.d. s.d.
target
s.d.
on
sample
X empirical (10% of
reference
X)
used
1.2µm + 56 55 3.1 5.5 3.1
0.8µm +86 85 6.7 8.5 6.7
2.4µm24 23 1.1 2.3 1.1
1.2µm32 31 3 3.1 3
0.8µm30 29 3.8 2.9 2.9
0.5µm14 13 4.6 1.3 1.3
measure-
ment charac-
teristics
SPS-5P-2A/B/C quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
66 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Method detection capability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
Particle Size [µm]
Dete
cti
on
cap
ab
ilit
y
identified detected
Probability to identify or to detect a particle (lab selected by random)
90% detection capability: 1.5µm90% identification capability: 1.6µm
1 µm particles will be detected (identified) with a probability of approx. 79% (76%).
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
77 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
assessment of the detection capabilities
Probability for correctly detected particles in GSR2005, GSR2003 and including only participants from both interlaboratory studies
90% detection capability2003: 1.8 µm2005: 1.5 µm
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
Particle Size [µm]
Dete
cti
on
cap
ab
ilit
y
2005 2003 2005 only with participants from 2003
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
88 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Laboratory assessment
1. Z Scores
2. Method detection capability
3. Indicative and false positive results
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
99 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Z Scores
Z scores of count of correctly detected particles (Part 1)(standardized deviations from target)
|Z| > 3, 2 < |Z| ≤ 3, |Z| ≤ 2
-3 0 3 -3 0 3 -3 0 3 -3 0 3 -3 0 3 -3 0 3
COP05 COP08 COP12 COP24 COPG08 COPG12010_1010_2
011012013014015016017019
021_2021_6021_8
110111112113114
115_1115_2
117118119120122123124310311312313314315316317319320321410411412
-2,308-2,308
-2,308-2,308
-2,308-2,308
-2,308-2,308
-2,759-2,759
-2,414-2,414
-2,069-2,069
-2,069-2,069
-2,699-2,699
-2,791-2,791
-2,791-2,791
-2,819-2,819
-2,374-2,374
-2,226-2,226
-2,904-2,904
-2,259-2,259
-6,923-6,923
-3,846-3,846
-10,000-10,000
-10,000-10,000
-6,923-6,923
-3,077-3,077
-3,077-3,077-6,923-6,923
-10,000-10,000
-4,615-4,615
-10,000-10,000-8,462-8,462
-4,615-4,615
-10,000-10,000
-10,000-10,000
-6,154-6,154
-10,000-10,000
-8,462-8,462
-6,154-6,154
-10,000-10,000
-4,138-4,138
-10,000-10,000
-5,862-5,862
-8,276-8,276
-10,000-10,000
-10,000-10,000
-6,897-6,897
-9,784-9,784
-3,374-3,374
-9,784-9,784
-4,652-4,652
-4,652-4,652
-9,199-9,199
-6,825-6,825
-5,045-5,045
-5,193-5,193
-9,347-9,347
-3,116-3,116
-10,326-10,326
-5,163-5,163
-3,227-3,227
-10,649-10,649
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1010 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Z Scores
Z scores of count of correctly detected particles (Part 2)(standardized deviations from target)
|Z| > 3, 2 < |Z| ≤ 3, |Z| ≤ 2
-3 0 3 -3 0 3 -3 0 3 -3 0 3 -3 0 3 -3 0 3
COP05 COP08 COP12 COP24 COPG08 COPG12413414415416417418
419_1419_2
420421422423610611612614
615_1615_2616_1616_2
617619620
621_1621_2
710711712713714716717718719720721722723724725726727
-2,759-2,759
-2,699-2,699
-2,362-2,362
-2,024-2,024
-2,791-2,791
-2,581-2,581
-7,692-7,692-10,000-10,000
-10,000-10,000
-3,846-3,846
-3,846-3,846
-6,923-6,923-3,077-3,077-10,000-10,000
-10,000-10,000
-10,000-10,000
-10,000-10,000
-5,385-5,385
-5,517-5,517
-10,000-10,000
-4,138-4,138
-10,000-10,000
-8,966-8,966
-10,000-10,000
-9,655-9,655
-10,000-10,000
-10,000-10,000
-5,398-5,398
-3,711-3,711
-10,458-10,458-4,386-4,386
-8,771-8,771
-10,458-10,458
-3,711-3,711
-6,073-6,073
-7,759-7,759
-3,722-3,722
-4,652-4,652
-10,235-10,235
-4,451-4,451
-5,341-5,341
-4,006-4,006
-9,051-9,051-4,154-4,154
-9,644-9,644
-9,199-9,199
-6,528-6,528
-7,270-7,270
-7,864-7,864
-5,163-5,163
-4,195-4,195
-4,518-4,518
-10,003-10,003-6,131-6,131
-12,262-12,262
-10,326-10,326
-4,840-4,840
-6,131-6,131
-7,422-7,422
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1111 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Classification of Z scores
Sat is-
factory
Quest io-
nable
Unsat is-
factory
(|z| < 2) (2 < |z| < 3) (|z| > 3)
1.2µm+ 64 (79%) 3 (4%) 14 (17%)
0.8µm+ 51 (74%) 3 (4%) 15 (22%)
2.4µm 75 (90%) 3 (4%) 5 ( 6%)
1.2µm 65 (80%) 3 (4%) 12 (15%)
0.8µm 49 (69%) 5 (7%) 17 (24%)
0.5µm 26 (43%) 4 (7%) 31 (51%)
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1212 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Enclosure to the certificate of participation
Description
True
value
Assigned
value*
Lab
result** s.d. used Z score
Count of 0.5 µµµµm particles
correctly detected 14 13 4 1.3 -6.9
Count of 0.8 µµµµm particles
correctly detected 30 29 26 2.9 -1
Count of 1.2 µµµµm particles
correctly detected 32 31 31 3 0
Count of 2.4 µµµµm particles
correctly detected 24 23 23 1.1 0
Count of particles ≥ 0.8 µµµµm
correctly detected 86 85 82 8.5 -0.4
Count of particles ≥ 1.2 µµµµm
correctly detected 56 55 55 3.2 0
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1313 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Detection capability per participant
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
50% - Percentile: m50 = 0.6 µm;
90% - Percentile: m90 = 0.8 µm;
steepness:s = -6
s
m
+
=
50
Size Particle1
1cap.detection
Particle size (µm)
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1414 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Atypical detection capability curves
s
m
+
=
50
Size Particle1
1cap.detection
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
Detectionprobability not converging to 100%.
Problems with plate?
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1515 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Atypical detection capability curves
s
m
+
=
50
Size Particle1
1cap.detection
Detection probability not depending on particle size?
Problems with plate?
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1616 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Atypical detection capability curves
All particles detected; percentiles cannot be calculated.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
1717 PT GSR 2005 – Statistical resultsPT GSR 2005 – Statistical results
Atypical detection capability curves
Low detection capability and low increase for larger particle size.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
quo data: productsand services
Method assessment
Laboratory assessment
System assessment
quo data: productsand services
Method assessment
Laboratory assessment
System assessment