Upload
juan-jimenez
View
215
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Jeff
Citation preview
The New Paradigm in Forensic Science
Geoffrey Stewart Morrison
Quotations
�
�
D. V. Lindley:
“Numeracy is not favoured by British justice.”
R. A. Carr-Hill:
“I believe Lindley’s suggestion is not only mad, it is extremely
dangerous.”
Lindley, D. V. (1977). Probability and statistics. , 27(3), 203–220.The Statistician
Imagine you are driving to the airport...
Imagine you are driving to the airport...
� This is Bayesian reasoning
– It is about logic
– It is not about mathematical formulae
– There is nothing complicated or unnatural about it
– It is the logically correct way to think about many problems
Imagine you work at a shoe recycling depot...
� You pick up two shoes of the same size
– Does the fact that they are of the same size mean they were
worn by the same person?
– Does the fact that they are of the same size mean that it is
highly probable that they were worn by the same person?
Imagine you work at a shoe recycling depot...
�
�
You pick up two shoes of the same size
– Does the fact that they are of the same size mean they were
worn by the same person?
– Does the fact that they are of the same size mean that it is
highly probable that they were worn by the same person?
Both and mattersimilarity typicality
Imagine you are a forensic shoe comparison expert...
�
�
�
The footprint at the crime scene is size 10
The suspect’s shoe is size 10
– What is the probability of the footprint at the crime scene
being size 10 if it had been made by the suspect’s shoe?
(similarity)
Half the shoes at the recycling depot are size 10
– What is the probability of the footprint at the crime scene
being size 10 if it had been made by the someone else’s shoe?
(typicality)
Imagine you are a forensic shoe comparison expert...
�
�
�
The footprint at the crime scene is size 14
The suspect’s shoe is size 14
– What is the probability of the footprint at the crime scene
being size 14 if it had been made by the suspect’s shoe?
(similarity)
1% of the shoes at the recycling depot are size 14
– What is the probability of the footprint at the crime scene
being size 14 if it had been made by the someone else’s shoe?
(typicality)
Imagine you are a forensic shoe comparison expert...
�
�
�
The footprint at the crime science is size 10
similarity / typicality = 1 / 0.5 = 2
The footprint at the crime science is size 14
similarity / typicality = 1 / 0.01 = 100
If you didn’t have a database, could you have made subjective
guesses at relative proportions of different shoe sizes in the
population and applied the same logic to arrive at a
conceptually similar answer?
similarity / typicality = likelihood ratio
The New Paradigm for Forensic-Comparison Science
�
�
�
Use of the likelihood-ratio framework for the evaluation of evidence
– logically correct
– adopted for DNA in the mid 1990s
Use of objective measurements, databases representative of the
relevant population, and statistical models
– transparent and replicable
Empirical testing of validity and reliability under conditions reflecting
those of the case at trial
The New Paradigm for Forensic-Comparison Science
�
�
�
Morrison, G. S. (2009). .
, 49, 298–308.
Morrison, G. S. (2010). . In I. Freckelton, & H.
Selby (Eds.), (Ch. 99). Sydney, Australia: Thomson
Reuters.
Morrison, G. S. (submitted).
. Manuscript submitted for publication, minor
revisions requested.
Forensic voice comparison and the paradigm shift
Forensic voice comparison
Measuring the validity and reliability of
forensic likelihood ratios
Science & Justice
Expert Evidence
The Likelihood-Ratio Frameworkfor the Evaluation of Evidence
Given that it is a cow, what is the probability of it having four legs?
p( 4 legs | cow ) = ?
Given that it has four legs, what is the probability that it is a cow?
p( cow | 4 legs ) = ?
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
0.06
-0.04
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
Given two voice samples with acoustic properties and ,
what is the probability that they were produced by the same speaker?
x x1 2
p( same speaker | acoustic properties , ) = ?x1 x2
p( same speaker | ) = ?acoustic properties ,x x1 2
p( cow | legs ) = ?x
posterior odds
likelihood ratio prior odds
p( same speaker | )
=
acoustic properties ,
p( different speaker | acoustic properties , )
p( acoustic properties , | same speaker ) p( same speaker )
p( acoustic properties , | different speaker ) p( different speaker )
x x
x x
x x
x x
1 2
1 2
1 2
1 2
×
Bayes’ Theorem:
¡¡¡ However !!!
The forensic scientist acting as an expert witness
can give the posterior probability. They can
give the probability that two speech samples were
produced by the same speaker.
NOT NOT
Why not?
�
�
�
The forensic scientist does not know the priors.
Determining the probability of guilt (same speaker) is the task of
the trier of fact (judge, panel of judges, or jury), not the
forensic scientist.
The task of the forensic scientist is to present the
which can be extracted from the speech samples.
strength of
evidence
posterior odds
likelihood ratio prior odds
p( same speaker | )
=
acoustic properties ,
p( different speaker | acoustic properties , )
p( acoustic properties , | same speaker ) p( same speaker )
p( acoustic properties , | different speaker ) p( different speaker )
x x
x x
x x
x x
1 2
1 2
1 2
1 2
×
Example
�
�
The likelihood ratio is 100
Whatever the trier of fact’s belief as to the relative probabilities of
the same-speaker versus the different-speaker hypotheses
before being presented with the likelihood ratio, after
they should be 100 times
more likely than before to believe that the voices on the two
recordings belongs to the same speaker rather than to different
speakers.
being
presented with the likelihood ratio
Calculating forensic likelihood ratiosusing objective measurements,databases representative of the
relevant population,and statistical models
Likelihood Ratio:
p( acoustic properties , | same speaker )
p( acoustic properties , | different speaker )
x x
x x1 2
1 2
p( legs | cow )xp( legs | not a cow )x
0
0.2
0.4
0.6
0.8
1
1 2 3 4 5 6 7 8
cows
not cows
legs
pro
port
ion
For continuous data rather than histograms, probability density
functions (PDFs) must be used.
0
0.002
0.004
0.006
0.008
0.010
0.012
0.014
0 20 40 60 80 100 120 140 160 180 200
rectangle width: 10
0 20 40 60 80 100 120 140 160 180 200
rectangle width: 5
(a) (b)
0 20 40 60 80 100 120 140 160 180 200
rectangle width: 2.5
(c)
0 20 40 60 80 100 120 140 160 180 200
rectangle width: 0.1
(d)
0
0.002
0.004
0.006
0.008
0.010
0.012
0.014
20 40 60 80 100 120 140 160 180 2000
0.005
0.010
0.015
0.020
0.025
LR = 11.35
suspect modelbackground modeloffender value
Empirically Testing the Validity of aForensic-Comparison System
Measuring Validity
�
�
�
�
Test set consisting of a large number of pairs known to be sameorigin and a large number of pairs known to be different origin
Test set must represent the relevant population and reflect theconditions of the case at trial
Use forensic-comparison system to calculate LR for each pair
Compare output with knowledge about input
Measuring Validity
�
�
Goodness is to which LRs from same-origin pairs > 1, anddifferent-origin pairs < 1
extentLRs from
Goodness is to which log(LR)s from same-origin pairs > 0,and log(LR)s from different-origin pairs < 0
extent
1/1000 1/100 1/10 1 10 100 1000
-3 -2 -1 0 +1 +2 +3
LR
log (LR)10
� �CN LR N
LRllr
ss i
N
ss ds j
N
ds
ss
i
ds
j� �
�
���
�� � �
�
���
��
� �� �
1
2
11
1 112
1
2
1
log log
� A metric which captures the gradient goodness of a set of likelihoodratios derived from test data is the log-likelihood-ratio cost, Cllr
Log Likelihood Ratio10
Cllr
-3 -2 -1 0 1 2
1
2
3
4
5
6
7
8
9
3
R v T
�
�
�
“32. It is clear that likelihood ratios have been used in other areas of
expertise by forensic experts when expressing their
conclusions. We are solely concerned in this appeal with the
use in relation to footwear mark evidence.”
“61. [The Forensic Science Regulator] suggested that it was not
logical to adopt the position that the Bayesian or likelihood
ratio approach could be used in some areas, but not in others...”
“76. ...We do not agree with the observations of the Regulator that a
similar approach is justified in all areas of forensic expertise.
Each area requires a separate analysis because of the
differences that there are in the nature of the underlying data.”
R v T
�
�
“79. The paper by Jackson, Champod and Evett [2001] rejected the
suggestion that hard data were needed to evaluate a likelihood
ratio...”
“80. We cannot agree with this in so far as it suggests that a
mathematical formula can be used. An approach based on
mathematical calculations is only as good as the reliability of
the data used...”
R v T
�
�
“83. ... the data on footwear distribution and use is quite unlike
DNA. A person’s DNA does not change and a solid statistical
base has been developed which enable accurate figures to be
produced...”
“84. Use of the FSS’s own database could not have produced
reliable figures as it had only 8,122 shoes whereas some 42
million are sold every year...”
R v T
� “87. It is of course regrettable that there are, at present, insufficient
data for a more certain and objective basis for expert opinion
on footwear marks, but it cannot be right to seek to achieve
objectivity by reliance on data which does not enable this to be
done. We entirely understand the desire of the experts to try
and achieve the objectivity in relation to evidence of footwear
marks, but the work done has never before, as we understand
it, been subject to open scrutiny by a court.”
Further reading
�
�
�
�
�
�
R v George [2007] EWCA Crim 2722
R v GK [2001] NSWCCA 504
Morrison, G. S. (2009). Comments on Coulthard & Johnson’s (2007) portrayal of the
likelihood-ratio framework. , 41,
155–161.
Rose, P., & Morrison, G. S. (2009). A response to the UK position statement on
forensic speaker comparison.
, 16, 139–163.
Balding D. J. (2005). . Chichester, UK:
Wiley.
Robertson, B., & Vignaux, G. A. (1995). .
Australian Journal of Forensic Sciences
International Journal of Speech, Language and the
Law
Weight-of-evidence for forensic DNA profiles
Interpreting evidence Chichester, UK: Wiley.
Thank You
http://geoff-morrison.net
http://forensic-voice-comparison.net
http://forensic.unsw.edu.au