Forensic Science International 212 (2011) 51–60

Recent misconceptions about the ‘database search problem’: A probabilisticanalysis using Bayesian networks

A. Biedermann *, S. Gittelson, F. Taroni

University of Lausanne, Ecole des sciences criminelles, Institut de police scientifique, le batochime, 1015 Lausanne-Dorigny, Switzerland

A R T I C L E I N F O

Article history:

Received 22 January 2011

Received in revised form 18 March 2011

Accepted 11 May 2011

Available online 12 June 2011

Keywords:

Database search

Evidential value

Bayesian networks

A B S T R A C T

This paper analyses and discusses arguments that emerge from a recent discussion about the proper

assessment of the evidential value of correspondences observed between the characteristics of a crime

stain and those of a sample from a suspect when (i) this latter individual is found as a result of a database

search and (ii) remaining database members are excluded as potential sources (because of different

analytical characteristics). Using a graphical probability approach (i.e., Bayesian networks), the paper

here intends to clarify that there is no need to (i) introduce a correction factor equal to the size of the

searched database (i.e., to reduce a likelihood ratio), nor to (ii) adopt a propositional level not directly

related to the suspect matching the crime stain (i.e., a proposition of the kind ‘some person in (outside)

the database is the source of the crime stain’ rather than ‘the suspect (some other person) is the source of

the crime stain’). The present research thus confirms existing literature on the topic that has repeatedly

demonstrated that the latter two requirements (i) and (ii) should not be a cause of concern.

� 2011 Elsevier Ireland Ltd. All rights reserved.

Contents lists available at ScienceDirect

Forensic Science International

jou r nal h o mep age: w ww.els evier . co m/lo c ate / fo r sc i in t

1. Introduction

In a recent recommendation [1], the German Stain Commissionwrites:

‘‘Speculative searches in a national DNA database using DNAprofiles from unsolved crime cases are a powerful tool toidentify individuals who cannot be excluded from beingcontributors of these DNA profiles, and thus may be consideredsuspects in these cases. When a crime scene profile matches aperson’s profile as a result of a database search, a statisticalevaluation of the weight of evidence of this database match isoften requested by the investigating authorities. The GermanStain Commission has developed recommendations on how toadequately take into account the probability of an adventitiousmatch given the database size. Following these recommenda-tions, the relevant match probability can be derived from thefrequency of the DNA profile corrected by the actual number ofpersons in the database. Based on theoretical considerationsand using simple examples, a statistical concept is describedthat allows to calculate either a match probability or alikelihood ratio without overestimating the weight of evidencefollowing a database search.’’ [at p. 113]

* Corresponding author. Tel.: +41 (0)21 692 46 07.

E-mail address: alex.biedermann@unil.ch (A. Biedermann).

0379-0738/$ – see front matter � 2011 Elsevier Ireland Ltd. All rights reserved.

doi:10.1016/j.forsciint.2011.05.013

In a letter to the editor of Rechtsmedizin, Taroni et al. [2] notethat elements of the proposed recommendations have alreadybeen widely discussed in both literature and practice, and found tobe problematic. The discussion—known in the context also as the‘database search problem’—dates back to the NRC reportspublished in the 1990s [3,4]. At that time, the suggestion wasthat a database search weakens a case against a matching suspect.Actually, it was recommended to multiply the random matchprobability p with the number n of database entries (also referredto as the np-rule). Subsequently, some commentators also went onto argue that one could not consider a proposition of the kind ‘thesuspect (some other person) is the source of the crime stain’because this refers to a propositional level that is said to bedependent on the DNA evidence [5, e.g.]. Instead, one ought to use aproposition of the kind ‘the source of the crime stain is (is not) inthe database’. Although such argument has since then convincinglybeen demonstrated to be inappropriate, it still appears to attractsupport. The recommendation issued by the German StainCommission is a typical example for this. It is in contradictionwith the present state of publications, which provides extensivestatistical explanations in support of the currently most widelyestablished view that a database search result strengthens the caseagainst a matching suspect if other members of the database can be‘excluded’ as potential crime stain donors [6–9, e.g.].

In a reply to the reaction of Taroni et al. [2], Fimmers et al. [10]defend the Stain Commission’s recommendations and reaffirm theview that (i) the random match probability ought to be multipliedby a factor equal to the size of the database (which tends to reduce

A. Biedermann et al. / Forensic Science International 212 (2011) 51–6052

the value of the likelihood ratio), and that (ii) so-called data-dependent propositions (e.g., ‘the suspect (some other person) isthe source of the crime stain’) should be avoided by choosingpropositions of the kind ‘the source of the crime stain is a personinside (outside) the database’. Based on a hypothetical example,the authors argue that this tends to reduce the probability of falseconvictions.

On the basis of a probabilistic analysis, the aim of this paper isto point out that the reply of Fimmers et al. [10] does not addressthe database search problem appropriately. Particular attentionwill be drawn to the—according to the view of the authors here—misconceived view of the probability of a false conviction. A moredetailed account of what this amounts to is given given bytranslated quotes from Fimmers et al. [10] which will beintroduced in the forthcoming sections as the discussionproceeds.

In order to clarify the various arguments and set them incontext, a graphical probabilistic approach (i.e., Bayesian networks[11, e.g.] is adopted here. Prior to the consideration of thisgraphical modelling approach, Section 2 will discuss the topic offormulating propositions when evaluating DNA evidence in caseswhere the suspect has been selected by a database search. Section3 focuses on the effect of excluding individuals in a database. Here,results from existing literature on the topic are invoked. Section 4will go on to present and discuss the issue of false convictions asput forward by Fimmers et al. [10]. The analysis of their argumentis pursued in further detail in Section 5 by using Bayesiannetworks. A discussion and conclusions are presented in Section 6.As a main output, this paper shows that the concerns of Fimmerset al. [10] about the risk of false convictions is based onassumptions that are known to be disconnected from reality.Their conclusions should thus be considered hypothetical, yetdebatable. Arguably, there appears to be no need to review currentevaluative procedures by introducing a correction factor that tendsto reduce the value of the likelihood ratio, nor to adoptpropositions that do not directly state the suspect as a potentialsource.

2. The formulation of propositions

A main aspect that distinguishes the view of Fimmers et al. [10]from current understandings of evaluating results from databasesearches is that of formulating propositions. Specifically, on page 4,the authors write [10]:

‘‘[. . .] Proceeding in this way [i.e., choosing the proposition ‘thesuspect is the source of the crime stain’] clearly violates afundamental principle of statistical decision theory, respec-tively the Bayesian approach for evaluating information. Underboth approaches, the independence between the formulatedhypotheses and the data to be treated is a fundamentalrequirement. The hypotheses must be formulated without theuse of information that comes from the data to be treated.’’1

Such a claim is unfounded. In both Bayesian and non-Bayesianliterature, such a ‘fundamental principle’ does not exist. One isperfectly allowed to formulate hypotheses designed to explaindata that are already known. For the purpose of illustration,consider a coin tossing experiment, a widely known situation bothof statistical theory and practice [12]. In such a setting, the sample-average statistic s/n, where s represents the number of times a coinlands heads and n represents the number of tosses, is the so-calledmaximum-likelihood estimator, also referred to as the ‘propensity’

1 Translation and text in brackets added by the authors.

of the coin to land heads. Stated otherwise, maximum-likelihoodestimation essentially aims at generating the best explanatoryhypotheses from the data.

Two related points that have however been discussed in theliterature of philosophy of science literature are:

1. Do the data confirm the theory that predicts them?2. Does a hypothesis derive greater confirmation from novel

predictions than from the already known phenomena that itexplains?

The first of these questions asks whether evidence E confirmshypothesis H that is designed to explain E. The second questionasks if H is confirmed more strongly by novel prediction than it isby the known data E.

This merges with the so-called ‘old evidence problem’ forBayesian theory. As for itself, Bayesian theory expresses patterns ofaccepted reasoning on the basis of data in terms of the way the dataaffect one’s personal beliefs. The ‘old evidence’ problem appearswhen one formulates a hypothesis which turns out to account for(or, explain) data that are already known before the hypothesiswas first proposed. The question then is whether such data canconfirm the hypothesis. The problem of explaining and justifyingsuch a confirmation can be considered, more formally, throughBayes’ theorem:

PrðHjEÞ ¼ PrðEjHÞPrðHÞPrðEÞ

As may be seen, if E is known at the time H is proposed, then E ispart of the background information of the individual reasoner sothat Pr(E) = Pr(E|H) = 1. Consequently, Pr(H|E) = Pr(H) and thisexpresses that the evidence E is not about to affect the probabilityof the proposition H.

This ‘problem’ is usually dealt with in the Bayesian literature inphilosophy of science [13, e.g.] by requiring that the backgroundknowledge is that which does not contain E. This is a natural way ofproceeding because when asking how much evidence E improvesour belief in H relative to what we already know, then this existingbody of background knowledge is just that which does not includeE. More formally expressed, this is I � {E}. This is entirely inagreement with current understanding of DNA evidence evalua-tion which requires that the definition of I embraces the non-DNAevidence that will be put to the court in relation to the mainproposition of interest, H [7, e.g.]. The request thus is aconditioning on I without E:

PrðEjH; fI � EgÞ and PrðEjH; fI � EgÞ (1)

where H denotes some alternative proposition which is mutuallyexclusive with respect to H.

The crucial issue to answer the two questions formulated aboveis: what is the probability of obtaining the evidence E if the mainproposition of interest, H, is false, that is PrðEjHÞ? The answer tothis question is based on agreed factual understanding about therelationship between the proposition H and the evidence E and thisunderstanding is independent of whether E has been used togenerate H or not.

In the ‘database search’ problem, as well as DNA and scientificevidence evaluation in general, requirement (1) is generallysatisfied because one is never conditioning evidence E by itself.That is, the ‘old evidence’ problem does not seem to apply to thedatabase search issue because the formulation of the propositionthat the suspect is the source of the crime stain is not necessarily aconsequence of the DNA profile match. On the contrary, thisproposition actually exists even before a match is found [14, e.g.].By searching a DNA profile database we express the view that each

3 It is assumed here that the number of matches X follows a binomial distribution

denoted in short-hand as X � Bin(n, p) where n denotes the number of trials and p

the probability of a success in any individual trial. Here, a trial amounts to a

A. Biedermann et al. / Forensic Science International 212 (2011) 51–60 53

of the individuals (indexed as 1, 2, ..., n) in that database could,potentially, be the source of the crime stain. If this is not the case,then there would be no point in searching that database. Arguably,one starts with a set of propositions H1, H2, ..., Hn where eachproposition Hi states that individual i is the source of the crimestain. In view of this, a claim that the formulation of proposition H

is made on the basis of, or is preceded by, existing knowledge of amatch is obviously unfounded.

According to Fimmers et al. [10], a potential dependency ofpropositions on data can be avoided by choosing a pair ofpropositions such as:

� The source of the crime stain is in the database.� The source of the crime stain is not in the database.

But such an argument, too, has only limited logical support.In fact, established literature on the topic has pointed out anargument known in the context as post-data equivalence. Thatis, after excluding all individuals in a database other than thesuspect, the pair of propositions mentioned above is logicallyequivalent to the pair of propositions ‘the suspect (someperson other than the suspect) is the source of the crime stain’[8, e.g.].

Despite the extended detour taken in this section to argueagainst the latter pair of propositions, it is worth emphasizingthat these arguments can further be strengthened, in a ratherstraightforward and compelling way, by considering that thesepropositions simply do not address the question that the courtmust answer. This has already been repeatedly pointed out inexisting literature, as is pointed out by the following quote [23],at p. 605: ‘‘[. . .] at trial a court is concerned only with thesuspect and not with the collective guilt or innocence of thedatabase.’’

3. The effect of excluding individuals in a database

In their comment, Fimmers et al. [10] ask what to

‘‘[. . .] recommend in a case in which, after finding exactly onematch in a database of size 630.000 [. . .], the RMP with 5analysed STR loci is in the magnitude of 1 in 600.000? It is in ourview inconceivable to assume, in such a situation, that theevidential value due to the database match is higher thanwithout a database search. Rather, it seems that the contrary isof significance.’’2

There are several issues with this statement. One is that the sizeof the database, let us abbreviate it by n throughout this paper, isgreater than the inverse of the random match probability (let uswrite g for this latter parameter). Implicit in the statementmentioned above seems to be a suggestion according to which it isprobable, hence little surprising, to encounter a match among anumber of individuals when that number is greater than theinverse of the rarity of the compared characteristic. On the basis ofthis perception, another issue is that of arguing that the less anobserved correspondence represents something unexpected, thelower should be the significance of a correspondence, once that oneis actually found.

Both of these aspects are debatable. For the setting assumed inthe quotation above, there is by no means a preponderance ofprobability to encounter a match simply because the size of thedatabase exceeds the inverse of the random match probability.Actually, the probability of encountering a target characteristic

2 Translation by the authors.

among n = 630.000 individuals, that is exactly one correspon-dence, when the rarity of that characteristic is g = 1/600.000 is0.367.3 The degree to which an event with such a probabilityrepresents something unsurprising is a matter of personaljudgement. Let us notice solely that this value is not very differentfrom the probability of observing no match at all, which is, in thiscase, 0.35.

A main twist with the statement mentioned above is that itfocuses on the probability of encountering a match by chance, thatis the term g, as a main criterion of evaluating the significance of adetected correspondence in a search scenario. This view is at leastshorthand, if not misleading. The main reason for this is that thereare further parameters that require consideration. Prior topursuing this in terms of a more detailed discussion in later partsof this paper, let us note at this point that—depending on thequestion that is being addressed—the rarity of the comparedcharacteristic may be entirely irrelevant in a discussion aboutsearching a database. It may even be entirely irrelevant to knowwhether or not the suspect actually matches. This stems from thefact that one can contemplate the proposition according to whichthe suspect is the source of the crime stain on the basis of the soleinformation that other potential crime stain donors, and mostnotably members of a database, are effectively ‘excluded’ aspotential sources. Such ‘exclusions’ may be accepted when theprofiles of the candidates at hand are found to have characteristicsthat are different from those of the crime stain. Stated otherwise,one can think about the proposition—let us call it H1—that asuspect (without knowledge about his ‘matching’-status) is thesource of the crime stain in the light of information that a certainnumber of other potential sources (here: all members on adatabase other than the suspect) are excluded. In some sense, thisis like searching (i.e., comparing) an entire database except thesuspect. This is just another way of conceptualizing the perceptionthat, whenever a pool of potential sources is reduced, necessarilyprobability must be proportionally redistributed among theremaining individuals.

In order to illustrate this point, consider a pool (i.e., population)of size N = 100, including the suspect. From these individuals,n = 91 are in a database. When considering an initial probability of1/N = 0.01 for each of the 100 individuals to be the source of acrime stain, excluding n � 1 = 90 of these people via a databasesearch leaves one with 10 people that could have left the crimestain, including the suspect. That is the suspect as the soleremaining individual in the database along with 9 individualsoutside the database. There is Bayesian argument [9, e.g.] to showthat the posterior probability of H1, given information about the90 excluded individuals, will be 1/(N � (n � 1)) = 0.1. Thisamounts to a proportional redistribution of probability amongeach member of the remaining potential sources from, initially,0.01 to 0.1. As may be seen, this kind of inference does not requirereference to the rarity of an analytical characteristic nor does itrequire information about whether or not the suspect actuallymatches. Stated otherwise, information about individuals ex-cluded during a database search can, on its own, be used toevaluate the probability that the suspect is the source of a crimestain independently of knowing whether or not the suspectactually matches.

More generally, the probative value of excluding individuals ona database of size n can also be expressed in terms of a likelihoodratio. In order to clarify this point, let X2 & ... & Xn denote the fact

comparison and a success to a correspondence. The probability of a correspondence

is given by g.

A. Biedermann et al. / Forensic Science International 212 (2011) 51–6054

that none of the n � 1 individuals indexed 2, ..., n in the databasehas a corresponding profile. The suspect is indexed as individual 1.In addition, let us write Hi for the proposition that individual i (fori = 1, ..., N) is the source of the crime stain. The likelihood ratio, letus call it LRdb for short, is one for multiple propositions and can bewritten as follows [15, e.g.]:

LRdb ¼PrðX2&:::&XnjH1Þf

XN

i¼2

PrðHiÞg

XN

i¼2

PrðX2&:::&XnjHiÞPrðHiÞ: (2)

In agreement with existing literature on the topic [8, e.g.], thisterm can be shown to reduce to:

LRdb ¼N � 1

N � n: (3)

Applied to the numerical example introduced above, one canwrite Bayes’ theorem in odds form and find the posterior odds of 1to 9, equating probability 0.1, as follows:

PrðH1jX2&:::&XnÞPrðH1jX2&:::&XnÞ|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}

posterior odds

¼ PrðH1ÞPrðH1Þ|fflfflffl{zfflfflffl}prior odds

�N � 1

N � n|fflffl{zfflffl}LRdb

¼ 1=N

ðN � 1Þ=N� N � 1

N � n

¼ 1

99� 99

9¼ 1

9: (4)

This result suggests several insights:

� Excluding individuals in a database as potential sources of acrime stain strengthens the case against a suspect. Note thatwhen n is very small with regard to N, the impact of theexclusions on the final posterior odds is very small comparedwith the impact of the likelihood ratio for the match on the finalposterior odds. However, its impact will always supportproposition H1, that is increase the posterior odds in favor of H1.� The evidential value of excluded individuals does not depend on

the rarity g of the compared characteristic. Only the size of thedatabase n relative to the size of the population N is relevant. Asshown by Eq. (3), the likelihood ratio will be maximal whenN = n.� The posterior odds, Eq. (4), after considering information about

database entries excluded as potential sources, can serve as the‘prior’ odds before considering information about the correspon-dence between the suspect’s profile and that of the crime stain.This clearly points out that a match with a suspect, denoted M1,and non-matches with other database members, X2 & ... & Xn, aredistinct items of evidence. This fact of extended evidence alsoexplains why the overall case against a suspect may be strongerin a setting in which a database search has been performed.Assuming a match with a suspect to be independent of the non-matches among the other database members conditional uponknowledge of H, this combination of evidence can be writtenmore formally as follows:

PrðH1jM1; X2&:::&XnÞPrðH1jM1; X2&:::&XnÞ|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}

posterior odds

¼ PrðH1jX2&:::&XnÞPrðH1jX2&:::&XnÞ|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}

prior odds

� PrðM1jH1ÞPrðM1jH1Þ

¼ 1

N � n� 1

g: (5)

In order to extend the numerical illustration pursued so far,consider a value of g = 10�3 and leave N = 100 and n = 91unchanged as before. For such a situation, the posterior probabilityof H1 is high and can be found to be 0.991. Moreover, it can readily

be seen that this conclusion is strengthened the larger the size ofthe database relative to the population.

4. Searching and convicting when the true source is not in thepopulation of interest: a relevant question to ask?

A further topic mentioned by Fimmers et al. [10] is that anoperational national DNA database may contain many ‘old’ profilesthat have typing results for only a limited number of loci [5, e.g.].Their rarity may be in the order of one in one million. The authorsargue that such a figure may still lead to ‘‘(. . .) an apparentlyprobative likelihood ratio in a case with one suspect’’ [emphasisadded] [10], at p. 4. However, they consider it problematic thatsuch a figure will also apply in a case in which a database searchhas been conducted. This viewpoint is justified by arguing thatwith profiles of increased occurrence, database correspondenceswill not be rare. This point is illustrated with the followingexample.

‘‘Let us consider a population of size 100 million [N = 108] and adatabase with one million [n = 106] entries of persons that comefrom that population. For the purpose of simplicity, we assumethat a suspect will certainly be convicted in every case in whichthe rarity of the corresponding DNA profile is [at least] one inone million [g = 10�6]. After a crime, a stain with a rarity of 10�6

is found on the crime scene. Unknown to investigators is thefact that the true source of the crime stain is neither in thepopulation of interest, nor in the database. Hereafter, twoscenarios will be considered. In order to compare the respectiveways of proceeding, we will focus our interest on the expected

frequency of false decisions.

1. There is a suspect. The DNA profile of that person is determinedand found to correspond to that of the crime stain. The person isgoing to be convicted on the sole basis of this correspondence.Given the assumptions in this example, we know that theconviction is erroneous, because the true author escaped. Howhigh are the odds, in our scenario, of this to happen by chance?The probability of the DNA profile is 1:1,000,000 and this is theprobability for a correspondence by chance with the stain. Theprobability for a false decision thus is 0.000001.

2. There is no suspect. A search in the database is conducted, andexactly one person is found. That person is convicted on thebasis of the same argument as that in scenario 1. The convictionis of course, again, false, because the data of the true author arenot stored in the database. What is the probability for such afalse decision? The answer is somewhat more complicated thanthat in scenario 1. An error occurs notably when exactly oneperson is found in the database. (. . .) We will find with aprobability of 0.368 (that is in approximately every third similarcase) exactly one person, which will subsequently be convicted,even though the true author is not in the database. Theprobability for an error in scenario 2 is therefore considerablygreater than in scenario 1.’’ [10], at p. 4 (italics and words insquare brackets added by the current authors)

These conclusions may follow from deduction, but the generalframing and the conceptual logic of the example are faulty forseveral reasons.

A main point is that a conditional profile probability of 10�6 cannever be convincing in a population of 108 because of an expectedhigh number of correspondences in the population. Whether or notthere has been a database search is—in practice—irrelevant. It isimportant to recognise that very strong evidence is needed toconvict someone, and the scenario presented here is imprudent inthat respect: no reasonable person should convict on the basis a10�6 conditional profile probability in a population of 108 in either

A. Biedermann et al. / Forensic Science International 212 (2011) 51–60 55

of the two hypotheticals (‘there is a suspect’ and ‘there is nosuspect’).

Moreover, the idea of convicting a person exclusively on DNAevidence is in complete disagreement with current legal theoryand practice, both in adversarial judicial systems as well as injurisdictions of central European countries such as Germany [19]and Switzerland [20]. The DNA evidence is not to be regarded inisolation, but to be connected with other evidence in a case. Inprinciple, it is not the case that there are convictions based only onDNA evidence, because practically there is never only DNAevidence. On a formal account, it should also be reminded thatthe consideration of the probative value of a ‘match’ with a crimestain, in view of an inference about a so-called ‘crime-level’proposition [17] (i.e., that the suspect is the offender or, if the crimehas been committed by more than one individual, one of theoffenders), is logically bound by the relevance of the crime stain,but also by further parameters, such as the number of offendersand the probability of leaving a stain for innocent reasons [18, e.g.].

Besides, it is also worth noting that, in general discussion, therelevant population is defined by what is implied by thecombination of the proposition forwarded by the defence (Hd)and information that is part of the framework of circumstances (I)[15, e.g.]. Although people might be familiar with the idea ofregarding a national population—say the 80 million inhabitants ofGermany—as the ‘relevant population’, and it may be a realisticpossibility that the offender for some crimes will be onlytemporarily in Germany and subsequently leave the country,one should draw the pool of relevant persons such that all those areincluded who, on all reasonable grounds, could have committedthe crime.

The discussion is also distracting at the point where Fimmerset al. [10] propose a decision rule based solely on the probability ofobtaining a corresponding analytical characteristic, that is theprobability of the evidence independent of the probability that thematching individual is, or is not, the donor of the trace. Convicting asuspect based on the probability of the evidence is inappropriatebecause this probability in itself says nothing about the probabilityof this suspect actually being the donor of the crime stain. Inaddition, the stated decision rule turns out to be particularnonsense in situations in which several correspondences would befound.4 That is, each individual with a corresponding DNA profilewould be convicted even though the crime stain can but come fromexactly one person.

As a general conclusion, Fimmers et al. [10] argue that alikelihood ratio procedure would be appropriate in their scenario 1(see translated quote at the beginning of this section), but would leadto an unacceptably high number of false decisions in their scenario 2.This critique of the likelihood ratio has no support essentiallybecause in their scenario 2, a deterministic decision rule is adopted,based on the sole observation of a match. There is no likelihood ratiowhatsoever involved in this issue. In conjunction, these aspects leadto a confusing issue, which is that of the probability of a falseconviction. This topic is analysed hereafter within a separate section(Section 5), by using a Bayesian network approach.

5. Probabilistic analysis using Bayesian networks

5.1. Structure for a Bayesian network

As noted in the previous section, it is conceptually misleading toassume that the true source of a crime stain is not among theindividuals that make up the population of interest. That is, by

4 We refer here to an extension of the questioned decision rule. In their original

paper, Fimmers et al. do not discuss any other application than that of exactly one

match.

definition, the population of potential sources is thought tocontain, by all logic, the individual that actually is the source. Inorder to provide further clarification of the degree to which thiscentral aspect compromises the coherence of the artificial scenarioof Fimmers et al. [10] quoted earlier in Section 4, consider arepresentation of that setting in terms of a Bayesian network as theone shown in Fig. 1. An analysis and discussion of theargumentative implications of this model is pursued later inSection 5.2.

The Bayesian network shown in Fig. 1 is defined by thefollowing nodes and associated probability tables:

� H: This node covers various propositions about the source of acrime stain at hand. The state H1 specifies that the suspect is thesource of the crime stain whereas the state H2_n considers thatone of the n � 1 individuals (other than the suspect) from thedatabase of size n is the source of the crime stain. The state Hn+1_N

accounts for the possibility that an individual outside thedatabase, but within the population of size N, is the source of thecrime stain. A fourth state, abbreviated HN+, models the situationin which the true source of the crime stain is not among themembers of the population of size N. This is a central assumptionof the scenario put forth by Fimmers et al. [10] (Section 4).Although such a state is conceptually problematic, it is adoptedhere in order to make plain its misguiding influence onsubsequent inferences. The probability table, shown hereafter,reflects the conditioning of node H on whether or not the truesource of the crime stain is or is not within the population of sizeN (i.e., node S in N ?), the size of the database (node n) as well asthe size of the reference population (node N). For a setting inwhich the true source of the crime stain is among the members ofthe population of size N, it is assumed here that each individualhas the same initial probability to be the source of the crimestain. This implies a probability of 1/N for the proposition H1 andcumulative probabilities (n � 1)/N and (N � n)/N for the com-posite propositions H2_n and Hn+1_N, respectively. Notice thatthese latter probability assignments are chosen for the solereason of convention and may be modified as required. Theremaining assignments of 0 and 1 (see array below) reflect logicalimplications.

� Node S in N?: This node is Boolean and represents the propositionaccording to which the true source of the crime stain is or is notamong the members of the reference population of size N.� Nodes n and N: The size of the database is modeled here in terms

of a numerical node n whereas the size of the referencepopulation is implemented in terms of a numerical node N. Inorder to reflect the assumptions of the currently discussedsetting, only a single state is defined for each of these nodes(additional states may be defined as required). A numerical state106 is defined for n and a state 108 for N.� Node g: This is a numerical node that represents the rarity of the

observed analytical characteristic. A single numerical state 10�6

is adopted in order to agree with the currently discussed setting(as outlined earlier in Section 4).� Node M1: This node has two states M1 and M1 which represent

the propositions according to which the profile of the firstindividual (i.e., the suspect) corresponds, respectively does notcorrespond to the profile of the crime stain. The probability tableof M1 reflects dependencies upon the proposition H and g and is

Fig. 1. Bayesian network for representing an artificial situation of searching a DNA

profile of a crime stain (with rarity g) against a database of size n when the reference

population is of size N. A number n of individuals of that population are in the

database. The node H covers propositions about the source of the crime stain

whereas the node S in N ? defines whether the true source of the crime stain is or is

not within the population defined by N. The node M1 represents a correspondence

with the profile of the suspect and # M in (n � 1) models correspondences among

the members in the database other than the suspect. The node D models the event of

a conviction. Further constructional details are given in the text (Section 5).

A. Biedermann et al. / Forensic Science International 212 (2011) 51–6056

completed as shown hereafter.

� Node # M in (n � 1): This is a node with numerical states 0,1 and�2 that represent the number of correspondences found amongthe n � 1 members in the database other than the suspect. Thenode table is completed as follows:

The probability of observing no match in a comparison is givenby one minus the probability of a match. Therefore, in all cases inwhich the true source of the crime stain is not among the (n � 1)individuals in the database other than the suspect (i.e., when H2_n

is false), the probability of observing no match among theseindividuals is given by (1 � g)n�1. The probability of exactly onematch is given by the probability mass function f(1 ; n � 1, g), thatis a binomial distribution with parameters (n � 1) and g. Theprobability of observing two or more matches is given by oneminus the cumulative probability of observing zero and onecorrespondence, formally written as 1 � F(1 ; n � 1, g). Let us notehere that the capital F(�) denotes the cumulative probability, whichis different from the lower case f(�), denoting the non-cumulativeprobability mass function. If the true source is among the (n � 1)database entries, then a correspondence will be certain. Conse-quently, the probability of having exactly one match depends onthe probability of having zero matches among the (n � 2)individuals other than the suspect and the true source. Thisprobability is given by (1 � g)n�2. The complement of thisprobability is the probability of having two or more correspon-dences.

� Node E: This node is Boolean and models the probability ofencountering exactly one match when searching the database ofsize n. As shown in the table hereafter, this node assumes thestate ‘true’ whenever the suspect matches and no individual

among the remaining (n � 1) individuals matches or when thereis exactly one match among the (n � 1) individuals and thesuspect does not match.

� Node D: This node is Boolean and represents the event of aconviction. According to the currently discussed hypotheticalscenario, a conviction is rendered whenever there is a singlematch as well as a value for g of (at least) 10�6. This impliesdependencies upon the nodes E and g and a probability table asgiven hereafter. Further discussion of this aspect of the model—which is considered reproachable in the view of the authorshere—is presented below (Section 5.2).

5.2. Outline of the proposed Bayesian network

The Bayesian network described so far allows one to address afirst question asked by Fimmers et al. [10]: ‘‘What is theprobability of encountering one match when searching a databasethat does not contain the true source of a crime stain?’’ Theprobability of this event, denoted E here, is given by the sum of the

probabilities of two component situations. One is that of a matchwith the suspect and no match among the (n � 1) other databasemembers. This probability is given by g, for the match with thesuspect, multiplied by (1 � g)n�1, for no match among (n � 1)individuals. A second event is that of exactly one match among the(n � 1) database entries, given by Pr(1|n � 1, g), along with asuspect who does not correspond, given by probability (1 � g).Notice that Pr(1|n � 1, g) is the binomial probability previouslydenoted by f(1 ; n � 1, g) in Section 5.1. In more formal notation,one thus has:

PrðEÞ ¼ PrðM1ÞPrðM in ðn � 1Þ ¼ 0Þ þ PrðM1ÞPrðM in ðn � 1Þ ¼ 1Þ

¼ gð1 � gÞn�1 þ ð1 � gÞPrð1jn � 1; gÞ:

Fig. 2 illustrates that these elements of thought can be trackedwithin the Bayesian network constructed so far. The probability fora match with the suspect is shown in node M1. In fact, the scenariohere assumes that the suspect is not the source of the crime stain(i.e., H1 is false), but someone outside the population of size N is.According to the probability table defined above for node M1, onehas a probability g = 10�6 that the node M1 is in state M1. Next, theprobabilities of there being, respectively, zero and one match aredepicted in the node # M in (n � 1). Again, these are probabilitiesconditioned on assuming HN+ to be true, according to the definitionof the probability table for the node # M in (n � 1). On the basis of

Fig. 2. Expanded representation of the Bayesian network shown in Fig. 1. Probabilities are shown in %. The numerical specification is as given in the text. The node S in N,

shown with a bold border, is set to ‘false’.

A. Biedermann et al. / Forensic Science International 212 (2011) 51–60 57

this input, the node E provides the probability 0.36788 of therebeing exactly one match among the n members of the database:

PrðE ¼ trueÞ ¼ gð1 � gÞn�1 þ ð1 � gÞPrð1jn � 1; gÞ

¼ 10�6 � 0:36788 þ 0:999999 � 0:36788 ¼ 0:36788:

In summary, this result amounts to the binomial probability ofobserving exactly one adventitious match among n individuals(none of which is the source of the crime stain), formally given byf(1 ; n, g).

Following the procedure of Fimmers et al. [10], any correspon-dence with a profile with rarity of at least g = 10�6 is sufficient for aconviction. For this reason, the previously found probability for thetruth of the node E is also the probability of the node D, whichmodels the event of a conviction.

5.3. The fallacious idea of convicting adventitiously matching people

The proposed Bayesian network illustrates the cascadedinference that follows from initial assumptions about the ‘location’of the crime stain’s true source (node H), the rarity of the comparedcharacteristics as well as the respective sizes of the referencepopulation and the database. This can serve the purpose ofproviding an interface through which the reasonableness ofmodelling assumptions can be analysed and discussed. This ispursued in some further detail here below.

The model shown in Fig. 2 provides a pictorial representation ofthe definitions of target events along with their associatedprobabilities. In particular, the graphical model shows that theprincipal event of interest in the hypothetical setting of Fimmerset al. [10], as described in Section 4, is the probability of a match ofsome unspecified person in a database. Most importantly, thatunspecified person is known not to be the source of the crime stain,but is nevertheless convicted subsequently. On a purely computa-tional account, there is nothing objectionable in this. As seen inSection 5.2, the answer to this question is one that follows fromdeduction, based on (in our view) disputable premisses, and theoutput of the Bayesian network obviously agrees with thesecalculations.

On a conceptual account, however, for reasons outlined earlierin Section 4, this setting is logically misconceived, and this can befurther clarified with the proposed Bayesian network. Consider thefollowing:

� Firstly, reality is that one is not concerned with a matchingindividual that could not, at least in principle, be the source ofthe crime stain. Stated otherwise, the focus of attention is to bedirected to individuals among which the true source canreasonably be assumed [16]. That individual may be in thedatabase (of size n), but if not, the person is to be among theN � n other individuals of the overall pool of N potential sources.If that cannot be guaranteed, then there would be no point ofsearching the database, nor considering that particular popula-tion of N individuals. Therefore, the node H in the Bayesiannetwork cannot reasonably be fixed to the state HN+. By all logic,one of the states H1, H2_n and Hn+1_N must be true in order topursue the evaluation of any evidence against one of thesesuspects. This is why the state of the node‘S in N?’ is to be set to‘true’ and the node H allowed to have a probability distributionover the three states H1, H2_n and Hn+1_N. Technically speaking,this is the prior distribution for the main propositions ofinterest.� Secondly, the inference problem associated with a database

search situation is not one of deduction, as suggested byFimmers et al. [10]. That is, one is not concerned with assumingHN+ to be true and then finding, in a top–down directedreasoning, the probability for an adventitious match. The veryinference problem associated with a database search problem isinductive: one starts with an observed correspondence and thenseeks to infer something about the state of node H. Thereasoning thus goes in the reverse direction. One has observed,for example, a match M1 along with zero matches among then � 1 other database members. This is communicated to theBayesian network by assigning probabilities one to the node M1

and to the state 0 of the node # M in (n � 1). This is illustrated byFig. 3.� Thirdly, it is not very helpful to conceptualise the event of a

conviction as a categorical consequence of the event of a matchand the rarity threshold g. Such an approach deprives decision

Fig. 3. Expanded representation of a modified Bayesian network for evaluating (i) the result of a database search and (ii) the correctness C of a decision D to consider a

matching suspect as the source of the crime stain. Nodes that have been set to one of their possible states are shown with a bold border. Probabilities are indicated in %. Further

definitions of this network are as given in the text.

5 Notice that, by an extension of argument on the basis of the stated assumptions,

as noted above, this low probability for the proposition at the source level also

transposes to the proposition at the crime level (which relates to the suspect’s

guilt).

A. Biedermann et al. / Forensic Science International 212 (2011) 51–6058

makers in their liberty of appraisal. In particular, it amounts toexplicitly ignore not only prior beliefs about the mainpropositions, but also the respective posterior beliefs. Graphi-cally speaking, the implication of this ought to be a model inwhich the event of a conviction is not modeled as a descendantof E and g (Fig. 1), but as an unconditioned root node (i.e., it hasno entering arcs from other nodes). This is made explicit inFig. 3. In that figure, the event of a conviction is modeled interms of the Boolean node D. Together, the event of a convictionD and the true state of nature about the source of the crimestain, node H, allow one to define the event of a false convictionof the suspect. This latter event is modeled here in terms of theBoolean node C. The probability table of this node completeslogically as follows:

PrðC ¼ truejD; HÞ ¼ 0; D ¼ true; H ¼ H1;1; otherwise:

�

We recognise, at this point, that this is an exclusivelyprobabilistic modelling of the relevant target events. In reality,a conviction amounts to a decision and the consequence ofsuch a decision is to be measured in terms of utilities. Bothaspects can be modeled in so-called Bayesian decision networks[21].

Fig. 3 provides a summary of these viewpoints and shows howto set a database search setting into context. This network is basedon the previously discussed model shown in Figs. 1 and 2, with thedifference that the node E (the event of observing exactly onematch among the n individuals in the database) has beeneliminated. The event of a conviction, node D, is now modeledas an unconditioned variable which feeds a new node C,representing the false conviction of the suspect. Strictly speaking,however, the node D should only be interpreted as ‘considering thesuspect as the source of the crime stain’, because the node H isdefined at the source level. Interpreting D as a conviction requiresan assumption of relevance for the crime stain, as well as nopossibility for leaving the stain for innocent reasons. Although,computationally, the probability of a false decision is not affectedby this, it seems nevertheless beneficial to keep in sight thedifferences in the semantics.

A further aspect shown in Fig. 3 is the calculation of theposterior probability distribution over the states of the source level

proposition H, based on the observed match with the individualindexed as 1 in the database (node M1) and the absence of acorrespondence among the other n � 1 database members (node# M in (n � 1)). Referring again to the scenario discussed byFimmers et al. [10], let us assume a reference prior probabilityof 1/N for each of the N = 108 potential sources, a database of size106 and a profile with probability g = 10�6. The posteriorprobability for the suspect being the source of the crime stainmay then be obtained as follows [9, e.g.]:

PrðH1jM1; M in ðn � 1Þ ¼ 0Þ ¼ PrðH1Þ

PrðH1Þ þ gXN

i¼nþ1

PrðHiÞ

¼ 1=N

1=N þ gN � n=N¼ 1

1 þ gðN � nÞ ¼ 0:01:

With such a low posterior probability for the propositionaccording to which the suspect is the source of the crime stain (H1),a conviction (node D) of the suspect thus leads to a false convictionwith a probability 1–0.01 = 0.99 (node C).5 According to thisanalysis, the probability of falsely considering the suspect as thesource of the crime stain is given by the sum of the probabilities ofall the propositions other than H1. In the particular settingconsidered here, it appears to be obviously advisable not to decideagainst the suspect. This would exclude the possibility of a falseconviction altogether. In particular, node C would show aprobability of one for the state ‘false’ when setting the node D

to ‘false’ (not shown here).The analysis pursued up to this point thus suggests that one

should not be concerned with the probability of finding a match inN comparisons of known innocent individuals and the subsequent,necessarily false conviction of some unspecified individual that isselected during such a search process. What is of actual interest isthe posterior probability for a given individual, selected during adatabase search, to be the source of the crime stain. The probabilityto falsely consider this individual as the source of the crime stain is

A. Biedermann et al. / Forensic Science International 212 (2011) 51–60 59

given by the probability of this person not being the source of thecrime stain. Using the numerical assignments of the hypotheticalscenario of Fimmers et al. [10] as discussed above, their categoricdecision rule would actually lead to a false conviction of amatching suspect with a probability 0.99. This is considerablylarger than the probability of finding an adventitious match amongN individuals. Using again the numerical assignments discussed sofar, the latter probability is 0.3678 (see also Fig. 2). Statedotherwise, a suspect runs a risk for false ‘identification’ equal toone minus the probability of being the source of the crime stain,rather than the probability of encountering an adventitious match.

6. Discussion and conclusions

‘‘What is the value of the evidence against a suspect who isfound to have the same analytical characteristics as those found ina sample from a crime scene, when that suspect is found as a resultof a database search?’’ In both theory and practice, this questionhas been debated at length. The most agreed upon viewpoint isthat in such a database search setting, the evidence against amatching suspect is strengthened compared to a situation in whichno database search was conducted [6–9,14,22–25, e.g.]. Althoughthere is substantial mathematical development available insupport of this position, it appears to remain unnoticed in somediscussions of the topic. Recent recommendations in Rechtsme-dizin [1] and the subsequent discussion [10] provide an examplefor this. These papers draw upon superseded positions (e.g.,adjustments in likelihood ratio calculations that tend to reduceevidential support), but also open new fields of discussion, such asthe probability of false convictions. This argument has beenanalysed and discussed in this paper through the use of Bayesiannetworks. It pointed out that the recent recommendations in [1], aswell as subsequent argument in their support [10], lackconsistency and are accompanied with implications that are inconflict with a logical analysis.

From the analysis presented here, three main points emerge:

� The fact of observing database members with different analyticalcharacteristics, which ‘excludes’ them as potential sources, tendsto strengthen the evidence against a non-excluded suspect.� The choice of a pair of propositions of the kind ‘the suspect (some

other) person is the source of the crime stain’ is not compromisedby the actual DNA evidence to be evaluated.� The probability of falsely ‘identifying’ (and, by extension,

convicting) a person ought not to be be equated with theprobability of encountering an adventitious match amongindividuals known to be innocent.

The first two points reaffirm opinions established in earlierdiscussion on the topic. The last point is a conclusion of an analysisof—according to the authors here—a misconceived argumentdeveloped in a more recent writing by Fimmers et al. [10].

Diverging views on how to set the database search issueappropriately in context presents a cause of concern because of thediscomforting effect on recipients of expert information. This effectpresents two main facets. One aspect is that the disagreementrelates to the fundamental question of whether the probative valueincreases or decreases. Another aspect is that divergent views onthis topic deploy their effect not only within the context of DNAevidence (where the debate has mostly been led so far), butpotentially also in other fields in which database searches areconducted. This clearly illustrates the continuing need for researchin this area.

More generally, the discussion here suggests that, in the future,there is also a need to pursue directions of research that are

different from those that this debate pursues. In fact, as noted by areviewer of this paper, one may possibly draw the conclusion thatthe non-scientifically suggested use of a ‘correction factor’ (the np-rule) emerges from the fear that the judicial system cannot handlethe logical approach to evidence evaluation, that is the reportedrarity of the characteristics will as a value of evidence against asuspect never be combined with any prior odds for that suspect tobe the source. However, such a standpoint does not excuse thefaulty introduction of a ‘safeguard’ like the np-correction and it iseven worse to try to motivate such an introduction by unsoundscientific argumentation (like the replacement of the pair ofpropositions). Nevertheless, while the recipients of expert witnessstatements do not understand how they shall treat the reportedevidentiary strength in conjunction with information obtainedprior to the evidence evaluation, there is need for moreargumentation, perhaps using another language than that ofmathematics.

Acknowledgment

The authors wish to thank the two anonymous reviewers fortheir insightful and constructive comments.

References

[1] P.M. Schneider, et al., Recommendations of the german stain commission regard-ing the statistical evaluation of matches following searches in the national DNAdatabase, Rechtsmedizin 20 (2010) 111–115.

[2] F. Taroni, A. Biedermann, R. Coquoz, T. Hicks, C. Champod, Brief an den Heraus-geber zum Beitrag von Schneider, et al., Allgemeine Empfehlungen der Spuren-kommission zur statistischen Bewertung von DNA-Datenbank-Treffern (Letter tothe Editor with reference to Schneider et al. Recommendations of the GermanStain Commission regarding the statistical evaluation of matches followingsearches in the national DNA database), Rechtsmedizin 21 (2011) 55–57.

[3] National Research Council, NRC I—National Research Council Committee on DNATechnology in Forensic Science, National Academy Press, Washington, D.C., 1992.

[4] National Research Council, NRC II—The Evaluation of Forensic DNA Evidence,National Academy Press, Washington, D.C., 1992.

[5] A. Stockmarr, Likelihood ratios for evaluating DNA evidence when the suspect isfound through a database search, Biometrics 55 (1999) 671–677.

[6] D.J. Balding, P. Donnelly, Inference in forensic identification, Journal of the RoyalStatistical Society, Series A 158 (1995) 21–53.

[7] I.W. Evett, B.S. Weir, Interpreting DNA Evidence, Sinauer Associates Inc., Sunder-land, 1998.

[8] A.P. Dawid, Comment on Stockmarr’s likelihood ratios for evaluating DNA evi-dence, when the suspect is found through a database search, Biometrics 57 (2001)976–978.

[9] D.H Kaye, Rounding up the usual suspects: a logical and legal analysis of DNAtrawling cases, North Carolina Law Review 87 (2009) 425–503.

[10] R. Fimmers, H. Schneider, M.P. Baur, P.M. Schneider, Erwiderung zum Brief vonTaroni, et al., an den Herausgeber zum Beitrag von Schneider et al. ‘‘AllgemeineEmpfehlungen der Spurenkommission zur statistischen Bewertung von DNA-Datenbank-Treffern’’ (Reply to the letter of Taroni et al. to the Editor withreference to Schneider et al. ‘‘Recommendations of the German Stain Commissionregarding the statistical evaluation of matches following searches in the nationalDNA database’’), Rechtsmedizin 21 (2011) 57–60.

[11] F. Taroni, C.G.G. Aitken, G. Garbolino, A. Biedermann, Bayesian Networks andProbabilistic Inference in Forensic Science, John Wiley & Sons, Chichester, 2006.

[12] C. Howson, The old evidence problem, British Journal for the Philosophy ofScience 42 (1991) 547–555.

[13] C. Howson, P. Urbach, Scientific Reasoning: The Bayesian Approach, 2nd ed.,Open Court, La Salle, 1993.

[14] I.W. Evett, L.A. Foreman, B.S. Weir, Letter to the editor, Biometrics 56 (2000)1274–1277.

[15] C.G.G. Aitken, F. Taroni, Statistics and the Evaluation of Evidence for ForensicScientists, 2nd ed., John Wiley & Sons, New York, 2004.

[16] D.J. Balding, When can a DNA profile be regarded as unique? Science & Justice 39(1999) 257–260.

[17] R. Cook, I.W. Evett, G. Jackson, P.J. Jones, J.A. Lambert, A hierarchy of propositions:deciding which level to address in casework, Science & Justice 38 (1998) 231–239.

[18] I.W. Evett, Establishing the evidential value of a small quantity of material foundat a crime scene, Journal of the Forensic Science Society 33 (1993) 83–86.

[19] R. Bender, A. Nack, W.-D. Treuer, Tatsachenfeststellung vor Gericht, C.H. Beck,Munchen, 2007.

[20] A.D.N. force probante, Bulletin de Jurisprudence Pe´nale 4 (1997) 103–105.[21] A. Biedermann, S. Bozza, F. Taroni, Decision theoretic properties of forensic

identification: underlying logic and argumentative implications, Forensic ScienceInternational 177 (2008) 120–132.

A. Biedermann et al. / Forensic Science International 212 (2011) 51–6060

[22] D.J. Balding, et al., Some causes for concern about DNA profiles, Statistical Science9 (1994) 248–251 (Comment).

[23] D.J. Balding, P. Donnelly, Evaluating DNA profile evidence when the suspect isidentified through a database search, Journal of Forensic Sciences 41 (1996) 603–607.

[24] D.A. Berry, DNA fingerprinting: a review of the controversy, Statistical Science 9(1994) 252–255 (Comment).

[25] P. Donnelly, R.D. Friedman, DNA database searches and the legal consumption ofscientific evidence, Michigan Law Review 97 (1999) 931–984.