Popper on Trial: A Brief History of a Big Muddle Susan ... · Popper purports to offer a throughly fallibilist, but still fully cognitivist, picture. But what he really offers is

Draft, March 1 st 2009. Do not cite or quote without permission.

1

Popper on Trial: A Brief History of a Big Muddle 1

Susan Haack

Rubbish is rubbish, but the history of rubbish is scholarship.— Saul Lieberman2

1. A Tangled Tale

When they feel the need for a criterion to distinguish genuine science, the real thing, from

pretenders, or for an understanding of the “scientific method,” U.S. courts have quite

often relied on Karl Popper’s philosophy – in particular, on his conception of falsifiability

as the hallmark of the genuinely scientific. Sometimes the legal issue has to do with

science and religion;3 but here I will focus exclusively on questions of evidence law,

where courts have called on Popper’s philosophy of science to help them determine the

reliability required for proffered expert scientific testimony to be admissible.

That courts should have turned to Popper on this question is, to put it bluntly,

disastrous; for Popper’s philosophy of science – which might be described as a kind of

Logical Negativism, since it stresses that scientific claims cannot be shown to be true or

1 © 2009 Susan Haack. 2 When I first heard this, it was attributed to Burton Dreben; but I have since learned that

it is not original to him, but is due to Talmudic scholar Saul Lieberman. Need details. 3 In McLean v. Arkansas, for example, the court reasoned that, unlike the theory of

evolution, creation “science” is unfalsifiable, and so is not really science at all but a religious doctrine; and hence that a statute mandating equal time for evolution and creation science in public high-school biology classes was unconstitutional – in violation of the Establishment Clause of the First Amendment to the Constitution, which provides that “Congress shall make no law ... respecting the establishment of religion.” McLean v. Arkansas Bd. of Educ., 529 F. Supp. 1255 (1982). Popper’s philosophy of science was invoked (first by Michael Ruse as expert witness for the plaintiffs, and then (but without Popper’s name) by Judge Overton in his ruling) to argue that the theory of evolution is science, but “creation science” is not. BUT IS IT SCIENCE?

THE PHILOSOPHICAL QUESTION IN THE CREATION/EVOLUTION CONTROVERSY (1996), Michael Ruse, ed., includes Judge Overton’s ruling, a transcript of Ruse’s testimony, and an article by


2

even probable – quite expressly eschews any interest in reliability. As we shall see, courts

(and legal scholars) who have imagined otherwise have systematically misunderstood

how radical Popper’s ideas really are, and how unsuitable for their purposes.

I will first sketch the relevant themes in Popper’s philosophy of science, to make

clear exactly how impossible it is to make it serve as a criterion of reliability. Next, I will

turn to the mess that’s made of Popper’s ideas when they are pressed into just such

service in the Supreme Court’s landmark ruling on scientific testimony, Daubert v.

Merrell Dow Pharmaceuticals, Inc. (1993). Then I will identify some of the sources of

the Supreme Court’s misunderstanding of Popper, in the amicus briefs in Daubert and in

the legal literature. And then, turning to the legal world after Daubert, I will look at what

federal judges have made of the Supreme Court’s allusions to Popper as Daubert has

played out in subsequent rulings on the admissibility of scientific testimony.

My ultimate purpose, however, is not simply exegetical; I don’t want just to show

that legal players have systematically misunderstood Popper. Nor do I want simply to

argue that Popper’s philosophy of science is no help in determining reliability. What is

most interesting, as I will suggest in conclusion, is that courts’ concern with reliability is

both legally essential and philosophically legitimate; and that, ironically enough, the

picture that legal players have mistakenly attributed to Popper is closer to the mark than

the account Popper actually offers.

2. Will the Real Karl Popper Please Stand Up?

Still, as J. L. Austin somewhere observes, in every significant philosophical thinker

“there’s the part where he says it and the part where he takes it back.” So I should

acknowledge right away that, besides what I take to be the authentic, tough-minded

falsificationist Popper, there is also a kind of shadow Popper – a kinder and gentler

Larry Laudan criticizing that testimony.


3

Popper, perhaps, but a kinder, gentler, and feebler Popper who doesn’t really offer

anything that one could dignify by calling it a “theory of science.” I’ll get to this shadow

Popper later; but let me begin with what I take to be the core themes of Popper’s official

account:

(i) What makes a statement scientific is that it is falsifiable, and what makes a field of inquiry scientific is that it traffics in falsifiable statements.4 This “criterion of demarcation” distinguishes science from non-science – a large and heterogeneous category which includes legitimate fields of inquiry such as history and metaphysics; mythology, religion, and “pre-science”; and allegedly illegitimate fields, “pseudo-sciences” as Popper calls them, such as Freudian psychology and Marxist “scientific socialism.”

(ii) A statement is falsifiable, and hence scientific, iff it is incompatible with some basic statement; 5 i.e., a statement reporting the occurrence of an observable event at a specified place and time.6

(iii) A scientific statement is falsified when a basic statement with which it is incompatible is accepted.7

(iv) The acceptance of basic statements can be neither justified nor impugned, but is strictly a matter of convention, a decision on the part of the relevant scientific community. Scientists may be causally prompted to accept a basic statement by what they observe; but no amount of observation can ever justify such a statement.8 For (Popper argues),

4 KARL R. POPPER, THE LOGIC OF SCIENTIFIC DISCOVERY 40 (English ed. 1959) (1934)

(“the falsifiability of a system is to be taken as a criterion of demarcation”). 5 Id. at 86. 6 Id. at 103. 7 Id. at 86. (Popper goes on to add that “[w]e shall take [a claim] as falsified only if we

discover a reproducible effect which refutes the theory”; but note that this addendum (i) is not about what falsifies a theory, but about what we take to falsify a theory, and (ii) runs together basic statements and the events they describe.)

8 Id. at 105 (“Experiences can motivate a decision, and hence an acceptance or rejection of a statement, but a basic statement cannot be justified by them – no more than by thumping the table.”)


4

justification is a logical relation; logical relations hold only among statements; and the only logical relations are deductive ones. Since even basic statements (he continues) are imbued with theory, if observations could stand in a logical relation to basic statements, that relation would have to be an inductive one; and in any case, since observations are not statements but events, they cannot stand in any kind of logical relation to a statement.9

(v) There is no inductive logic. The method of science is purely deductive, a matter of conjecture (making an informed guess about the possible explanation of some puzzling phenomenon) and attempted refutation (falsification).10 The proper procedure is to make bold, falsifiable conjectures; to test them as severely as possible; and, should they be falsified when they are tested, to drop them and start again rather than making ad hoc adjustments to save them.11

(vi) Theories which have been tested but not (yet) falsified have been, in Popper’s technical sense, “corroborated,” to a degree depending on the severity of the tests passed.12 But that a theory is corroborated, to however high a degree, doesn’t show that it is true, that it is probable,13 that there is reason to believe it – or that it is reliable.14

9 This involves some rational reconstruction of Popper’s text, disentangling two strands

of argument that he runs together; for details, see SUSAN HAACK, EVIDENCE AND INQUIRY (1993), 98-102.

10 POPPER, THE LOGIC OF SCIENTIFIC DISCOVERY, supra note 000, 30. (“The theory to be developed in the following pages stands directly opposed to all attempts to operate with the idea of inductive logic.”)

11 POPPER, THE LOGIC OF SCIENTIFIC DISCOVERY, supra note 000, 82. (But see also note

000 infra.) 12 POPPER, THE LOGIC OF SCIENTIFIC DISCOVERY, supra note 000, 265-9. (The English

translation of the heading of this section of the book speaks of “How a Hypothesis May ‘Prove its Mettle’”; but in footnote *1, p.53, added to the English edition, Popper suggests that this phrase – like Carnap’s translation of “Bewährung” as “confirmed” – may be misleading. See also note 000 infra.)

13 POPPER, THE LOGIC OF SCIENTIFIC DISCOVERY, supra note 000, 269; KARL R. POPPER, OBJECTIVE KNOWLEDGE: AN EVOLUTIONARY APPROACH (1972), 108 (the degree of testability of a theory is inversely related to its probability)

14 KARL R. POPPER, OBJECTIVE KNOWLEDGE, supra note 000, 18, 22. (The degree of


5

Obviously this core Popperian philosophy, eschewing verifiability, probability,

inductive logic, supportive evidence, etc., is thoroughly negative. In fact, it’s far more

negative even than Popper acknowledges. Presenting himself as a champion of science,

Popper purports to offer a throughly fallibilist, but still fully cognitivist, picture. But what

he really offers is a kind of covert skepticism. For if, as he maintains, the acceptance of

basic statements is not justified by scientists’ observations but is purely a matter of

convention, there is no guarantee that a scientific statement that has been “falsified” is

actually false; and this implies that scientific claims can no more be shown to be false

than they can be shown to be true.

So perhaps it is no wonder that Popper sometimes seems to suggest a very

different picture, incompatible with the official one. For example, there’s the shift from

his logical criterion of demarcation, based on the falsifiability of statements, to a

methodological criterion based on recommendations about the best procedures tor

scientists to follow.15 Then there’s his description of science as like a medieval cathedral,

gradually erected over many generations.16 This suggests a more or less cumulative

picture of scientific progress; but this can’t be squared with the falsificationist

conception, according to which science would be more like an endless building site in

which, each day, the previous day’s work is demolished and building begins anew – until

the next day. There is, also, Popper’s description of the “empirical basis” of science as

like piles driven into a swamp.17 This suggests a fallibilist picture of basic statements as

corroboration of a scientific theory represents its past performance only, and “says nothing about future performance, or about the ‘reliability’ of a theory.”)

15 POPPER, THE LOGIC OF SCIENTIFIC DISCOVERY, supra note 000, 82 (suggesting that it is willingness to accept falsification, rather than simply falsifiability, that makes a theory scientific).

16 Citation needed. 17 Id. at 111.


6

partially but not fully justified by what scientists observe; but this is quite at odds with

the official, hard-nosed “objectivist” conception, according to which knowing subjects

and their experiences are simply epistemologically irrelevant.18 Then there are Popper’s

hints that the process of conjecture and refutation might yield, if not evidence of truth,

probability, or reliability, at least increment of verisimilitude – but no, Popper writes,

corroboration is only a measure of what the verisimilitude of a theory appears to be,

relative to other theories, at a given time.19 And then there’s Popper’s “concession” that it

is rational to prefer a corroborated theory as the basis for action:20 a concession that is

wholly groundless if, as he insists, we can never have reason to think a corroborated

theory true, probable, or closer to the truth than other theories.

The potential for confusion is made even worse by some verbal quirks in Popper’s

writing. First, as David Stove long ago pointed out,21 Popper has a habit of using words

like “knowledge” and “discovery” without their usual connotation of truth; and, as I

argued above,22 he also has a habit of using “falsified” without its usual connotation of

falsehood. Perhaps even more consequentially, before he realized how misleading this

was and adopted the word “corroboration” instead, Popper went along with Carnap’s

(mis-)translation of his word “Bewährung” as “confirmation,” and for a while even used

the word “confirmation” himself.23

18 Karl R. Popper, Epistemology Without a Knowing Subject (date), in KARL R,. POPPER,

OBJECTIVE KNOWLEDGE, supra note 000 (1972), 000. 19 POPPER, OBJECTIVE KNOWLEDGE, supra note 000, 22. (“Verisimilitude” is Popper’s

term for (as he puts it, somewhat misleadingly) “nearness to the truth”; roughly, it refers to the proportion of the truth-content of a theory to its falsity-content.)

20 Id. 21 DAVID STOVE, POPPER AND AFTER: FOUR MODERN IRRATIONALISTS (1982; reprinted

under the title ANYTHING GOES, 1999). 22 And earlier: see SUSAN HAACK, EVIDENCE AND INQUIRY (1993), 101-2. See also Alan

Olding, Popper for Afters, 43.2 QUADRANT 19 (Dec. 1999). 23 POPPER, THE LOGIC OF SCIENTIFIC DISCOVERY, supra note 000, 251-2, note *1

(“Carnap translated my term ‘degree of corroboration’ ... as ‘degree of confirmation.’ ... I fell in


7

Of course, though Popper’s official story is about as ill-suited as it could be to

discriminate reliable from unreliable scientific testimony, the parts where he takes it back

make it only too easy to misconstrue what his official story really is. So it should come as

no surprise to find that the idea that Popper’s philosophy of science will be helpful to

courts needing to determine whether scientific testimony is reliable turns out to rest on a

systematic misunderstanding of his ideas: specifically, on mistakenly taking him to hold

that a claim that has been tested but not falsified is thereby confirmed, i.e., shown to be

probable, warranted, valid, or reliable.

3. Daubert’s Pseudo-Popper

In contrast to the principle of the “free evaluation of proof” that prevails in civil-law

systems, in the United States the law of evidence, both federal and state, is exclusionary,

in the sense that certain kinds of evidence are legally inadmissible.24 Rules governing the

admissibility of evidence are the same in criminal and in civil cases; but states’ rules may

differ from federal law, and from each other. Because expert witnesses (including

scientific witnesses) have knowledge not available to the average juror, they are given

more latitude than lay witnesses in expressing their opinion; but, correspondingly, the

admissibility of expert testimony is subject to special restrictions. I shall focus here

specifically on federal evidence law regarding expert scientific testimony.

Daubert v. Merrell Dow Pharmaceuticals, Inc (1993) was a toxic-tort case

involving the morning-sickness drug Bendectin, alleged to cause birth defects. It was also

with his usage, thinking that words do not matter. .. I myself used the term ‘confirmation’ for a time ... . Yet it turned out that I was mistaken: the associations of the word ‘confirmation’ did matter ... ‘degree of confirmation’ was soon used as a synonym ... of ‘probability’ I have therefore now abandoned it ... .”)

24 Inadmissible evidence may be excluded by the judge and never presented to the jury; or, if it is presented to the jury but successfully challenged by the other party as inadmissible, the court will instruct the jury to disregard it.


8

the first case in its then-204-year history in which the U.S. Supreme Court had addressed

the issue of the standards of admissibility of expert scientific testimony. Daubert – alone

among the many cases brought against the manufacturers of Bendectin25 – made it to the

Supreme Court because of a peculiarity of its legal history: in excluding the Dauberts’

proffered scientific testimony, the lower court had relied on the Frye Rule,26 under which

novel scientific testimony is admissible only if “generally accepted in the field to which it

belongs.” This rule arose out of a murder case and, in line with its origin, had hitherto

been used almost exclusively in criminal cases; and the Supreme Court granted certiorari

to determine whether it had been superseded by the 1975 Federal Rules of Evidence,

specifically by FRE 702, which provided that testimony based on “scientific, technical or

other specialized knowledge” was admissible provided only that it was relevant and not

otherwise excluded by law.

A unanimous Court ruled that Frye was superseded, but that this didn’t mean that

federal judges no longer had a responsibility to screen expert testimony. Federal Rule of

Evidence 702, Justice Blackmun continued on behalf of the majority (as we shall see

later, in this he no longer had the unanimous agreement of his colleagues), itself required

judges to screen such testimony not only for relevance but also for reliability. For the

Rule refers to “scientific ... knowledge”; so courts must determine whether proffered

scientific evidence really is scientific knowledge. What “scientific knowledge” means,

Justice Blackmun went on, is knowledge arrived at by the scientific method.

What is the mark of genuine science? Falsifiability, testability, Justice Blackmun

replies. What is the method of science? Essentially, it is the method of conjecture and

25 On the history of Bendectin litigation, see MICHAEL D GREEN, BENDECTIN AND BIRTH

DEFECTS (1996); JOSEPH SANDERS, BENDECTIN ON TRIAL: A STUDY OF MASS TORT LITIGATION (1998).

26 Frye v. United States 54 App. D.C. 46, 47 (1923). Frye remains the law today in a number of states, including Florida, New York, Pennsylvania, and California.


9

attempted refutation. And so the first consideration on the “flexible list” of indicia of

reliability he goes on to provide is whether proffered scientific testimony “can be (and

has been) tested.”

[A] key question in determining whether a theory or technique is scientific knowledge ... is whether it can be (and has been) tested. “Scientific knowledge today is based on generating hypotheses and testing them to see if they can be falsified; indeed, this methodology is what distinguishes science from other fields of human inquiry.”27

The Popperian flavor of all this is unmistakable; and indeed Justice Blackmun

goes on to quote Popper (“the criterion of the scientific status of a theory is its

falsifiability, or refutability, or testability”).28 However, in the very same paragraph –

indeed, in the very same sentence – Justice Blackmun also quotes Carl Hempel (“the

statements constituting a scientific explanation must be capable of empirical test”),29 as if

Popper and Hempel were saying the same thing. Apparently he didn’t understand Popper

(or Hempel); and in particular didn’t realize that Popper’s understanding of “testable”

(“potentially falsifiable”) and Hempel’s (“potentially confirmable or falsifiable”) are

quite different. Nor, apparently, was Justice Blackmun aware that Popper specifically

identifies Hempel as a proponent of the inductivist philosophy of science he repudiates;30

that he insists that his concept of corroboration not be confused with the inductivist idea

of confirmation;31 and that he expressly disavows any interest in the reliability of

scientific theories, and indeed avoids even using the word “reliable” without

precautionary scare quotes.32

27 Daubert v. Merrell Dow Pharms, Inc., 509 U.S. 579, 591 (1993) (citations omitted). 28 KARL R. POPPER, CONJECTURES AND REFUTATIONS 37 (5th ed. 1989). 29

CARL G. HEMPEL, PHILOSOPHY OF NATURAL SCIENCE 49 (1966). 30 POPPER, OBJECTIVE KNOWLEDGE, supra note 000, 20, n.29. 31 See note 000 supra. 32 POPPER, OBJECTIVE KNOWLEDGE, supra note 000, 20 (he puts the emphasis “on

negative arguments, such as negative instances or counter-examples, refutations, and attempted


10

It looks as if – perhaps unduly influenced by that honorific use of “science,”

“scientific,” etc., as all-purpose terms of epistemological praise, and perhaps forgetting

that not all scientific expert testimony is reliable, and not all reliable expert testimony is

scientific33– Justice Blackmun ran “reliable” and “scientific” together. Then, casting

about for some criterion to distinguish the genuinely scientific from pretenders, he

fastened on Popper’s criterion of falsifiability but – not realizing that Popper’s criterion

of demarcation is part of a throughly negative philosophical package that is less than no

help on the question of reliability – he ran together Popper’s and Hempel’s

understandings of “testable.”34

4. Where Did the Muddle Come From?

Where did the Daubert Court get the idea of looking to Popper? And how did it get

Popper so utterly backward? The confusion in Justice Blackmun’s ruling was hardly his

own creation; it was ubiquitous in the literature with which he would have been familiar.

Several of the amicus briefs submitted in Daubert refer to Popper – and all of

these, more or less explicitly, perpetrate the same distortions we have found in Justice

Blackmun’s ruling. For example, a brief submitted by the American Medical Association,

which avers that “[a]n opinion is only based upon scientific knowledge if it is developed

in accordance with the scientific method,” goes on to say – citing Popper! – that “[i]f a

hypothesis is repeatedly corroborated by empirical testing, it is ... generally accepted as

refutations – in short, criticism – while the inductivist lays stress on ‘positive instances’ from which he draws ‘non-deductive inferences’, and which he hopes will guarantee the ‘reliability’ of the conclusions of these inferences”).

33 Notice the ellipses in Justice Blackmun’s reference to “scientific ... knowledge”; and remember that FRE 702 speaks of “scientific, technical, or other specialized knowledge” (emphasis added).

34 It may also be relevant that the usual meaning of “corroborated” in legal contexts is something like “confirmed by other witnesses.”


11

valid.” In principle, these amici admit, “no scientific theory is ever definitively

confirmed”; however, they continue, “[a]s a practical matter ..., some theories are so

thoroughly tested that they become virtually incontrovertible.35 Given how close being

incontrovertible is to being not falsifiable, this is about as un-Popperian as it could be.

Another amicus brief, from the Product Liability Council, describes the scientific

method as follows:

(1) first set forth a hypothesis, (2) design an experiment ... or experiments, to test the hypothesis, (3) conduct the experiment, collect the data, and then analyze those data, (4) publish the results so that they may ... be subject to external scrutiny, and (5) ensure that those results are replicable and verifiable.

Each of these clauses has its own citation (omitted here); the last – attached to the bit

about the results being “replicable and verifiable”! – is to The Logic of Scientific

Discovery.36 Again, given Popper’s repudiation of verificationism (and his

conventionalism about basic statements) this is an exegetical travesty.

And an amicus brief from the Carnegie Commission on Science, Technology, and

Government, after arguing, citing Popper, that “opinions based on clams that are not

capable of being tested should not be admitted into evidence,” immediately adds, citing a

report from a Panel of the National Academy of Sciences, that results that cannot be

replicated should also be excluded, and that “scientists have the responsibility to replicate

and reconfirm their results.”37 Fair enough; except that, because these amici fail to note

that Popper expressly eschews the notion of confirmation, this too conveys the false

impression that claims which have been subjected to replicable tests, but not falsified,

have been confirmed. But of course results can’t be reconfirmed unless they have first

been confirmed.

35 AMA brief at 9 (emphasis added) 36 Product Liability Council brief at *22 and note 20 (emphasis added). 37 Berger brief at *14, and notes 12 (citing Popper) and 13 (citing NAS panel).


12

The crucial misunderstanding of Popper was also circulating in the law reviews,

and is found specifically in an article by Michael Green38 from which Justice Blackmun

quotes in his ruling in Daubert, at the end of the passage I quoted earlier. The key

passage of Green’s paper (a very small part of a very long article most of which is taken

up with a complex discussion of issues about epidemiological evidence in toxic-tort

litigation) reads as follows:

Hume criticized the inductive, rather than the deductive, methodology. From that criticism emerged the idea that while induction could never conclusively prove a proposition, it could falsify one. Thus, based on the framework provided by Karl Popper, knowledge is gained by attempting to falsify a hypothesis based on empirical investigation. Scientific methodology today is based on generating hypotheses and testing them to see if they can be falsified; indeed, this methodology is what distinguishes science from other fields of human inquiry. Of course, if a hypothesis repeated withstands falsification, we may tend to accept it, even if conditionally, as true.39

Skating over the bizarre idea that induction is needed for, or even capable of, falsifying a

statement, I turn to the sentence Justice Blackmun quotes, the one beginning “Scientific

methodology today ... .” This vaguely suggests that Popper’s ideas are now the consensus

position in contemporary philosophy of science, or among scientists themselves –

perhaps, even, that scientists have now adopted the methodology Popper recommended.

This is misleading, to say the least: while Popper’s work has been very influential, and

while there are still enthusiastic Popperians about, his philosophy of science is by no

means “generally accepted in the field to which it belongs.” And the last sentence of this

passage is a real doozy; for here Green manages to suggest, without ever actually saying,

that a claim that has been tested but not falsified is thereby shown to be (“conditionally”)

true. This is, once again, completely un-Popperian.

38 Michael D. Green, Expert Witnesses and Sufficiency of Evidence in Toxic Substances

Litigation: The Legacy of Agent Orange and Bendectin Litigation, 86 NW. L. REV. 643 (1991-2).


13

In a footnote to the passage I quoted, Green cites The Logic of Scientific Discovery

– the whole thing, giving no page numbers. In the same footnote, he cites another law

review article, by David Faigman, published shortly before his own; it seems possible

that he hadn’t actually read Popper, but was simply relying on Faigman’s account. If so,

it was not an entirely happy choice of source. Here is Faigman on Popper’s philosophy of

science:

Falsifiability or testability represents the line of demarcation between science and pseudo-science, and the strength of particular scientific statements depends on the extent to which they have been tested appropriately. ... Popper devoted much of his philosophical efforts to articulating a criterion by which scientific statements could be distinguished from nonscientific statements, especially pseudo-scientific, prescientific, and metaphysical statements. ... Empirical research might corroborate [a] hypothesis by finding evidence supporting it ... . The scientific method is neither static nor easily encapsulated ... [and] [n]o one method can be assumed to be more scientific than another.40

Faigman apparently did read (some) Popper, but only partly understood him. The word

“strength” in his first sentence is a fudge. There is slippage between this first sentence,

which talks about science versus pseudo-science, and the next, which also includes pre-

science, etc., under non-science; but there is no indication that Faigman realizes that the

fact that Popper’s criterion is intended to do several different jobs itself presents

problems. It is doubtful whether, given his stress on conjecture and refutation as the

method of science, Popper would agree with Faigman’s concluding claim, that there is no

unique scientific method. But most importantly, Faigman’s penultimate sentence,

according to which corroboration is a matter of finding supportive evidence, is

completely un-Popperian.

39.Green, supra note 000, 645-6 (emphasis added). 40 David Faigman, To Have and Have Not: Assessing the Value of Social Science to Law


14

5. What Have the Consequences of the Muddle Been?

Only two of Justice Blackmun’s colleagues on the Daubert Court – then-Chief Justice

Rehnquist, and Justice Stevens, who joined in Justice Rehnquist’s partial dissent – seem

to have been even half-aware of how muddled the philosophy of science built into the

majority ruling was. Yes, Frye had been superseded, Justice Rehnquist wrote; and yes,

nevertheless courts have a responsibility for screening expert testimony. But, he

continued, there is no explicit reference in FRE 702 to reliability, and Justice Blackmun’s

observations about “scientific ... knowledge” are too vague and too general to be helpful.

These observations are bound to raise difficulties down the road, when courts have to

determine how to handle non-scientific expert testimony;41 and, frankly, he was baffled –

as well he might be! – by Justice Blackmun’s allusions to falsifiability:

I defer to no one in my confidence in federal judges, but I am at a loss to know what is meant when it is said that the scientific status of a theory depends on its “falsifiability,” and I suspect some of them will be, too.42

Subsequent rulings in which federal judges refer to Popper, falsifiability, or

testability suggest that Justice Rehnquist was right to suspect that they wouldn’t know

what to make of Justice Blackmun’s foray into philosophy of science. Quite often, after

quoting the passage in Daubert presenting the Court’s “flexible list” of indicia of

reliability, courts simply ignore the Popperian rhetoric,43 and rely instead on some other,

and (apparently) easier, Daubert factor, such as whether the work on which testimony is

based has been subject to peer-review and publication.44 But when courts actually try to

as Science and Policy, 38 EMORY L. J. 1005 (1989), 1014-5, 1016, 1018, 1024 (emphasis added).

41 As indeed there were – difficulties resolved in 1999 when the Supreme Court ruled that Daubert applies to all expert testimony, not only the scientific. Kumho Tire Company, Ltd. v. Carmichael, 526 U.S. 137 (1999).

42 Daubert, 509 U.S. 579, 598 (Justice Rehnquist, dissenting in part). 43 Citations needed. 44 Not that this Daubert factor is really as simple as courts often suppose; see Susan


15

use the Popperian clause, they tie themselves in knots.

Very shortly after Daubert, faced with a defense challenge to the FBI laboratory’s

DNA analyses, the court in U.S. v. Bonds (1993) read its new gatekeeping obligations

strictly au pied de la lettre. In this case the court excluded the defendants’ proffered

evidence that DNA testing conducted by the FBI laboratory was unreliable; this

testimony was inadmissible under Daubert, the court reasoned, since in proffering

evidence about the deficiencies of the FBI lab, “the defendants have conceded that the

theory and methods can be and have been tested.”45 In other words, the fact that the FBI

lab’s DNA work had been tested and shown to be unreliable showed that the FBI’s

testimony could be and had been tested; and hence was grounds for admitting it – as

reliable!

Two fingerprint cases misinterpret Daubert’s reference to “testability” as meaning,

not scientific, empirical testing, but legal, dialectical testing in court. In U.S. v. Havvard,

the court reasons that “[t[he claim of uniqueness and permanence of [fingerprints] is a

scientific claim in the sense that it can be falsified. ... In the roughly 100 years since

fingerprints have been used for identification purposes, no one has managed to falsify the

claim of uniqueness.”46 Of course, the crucial issue is not whether fingerprints are unique,

but whether accurate identifications can be made on the basis of latent prints representing

on average 20% of a full fingerprint; but set that aside. The more interesting point is that

the court goes on to construe “testing” in terms, not of empirical testing in a laboratory or

in the field, but of in-court “testing” through cross-examination:

[T]he methods of latent print identification can be and have been tested. They have been tested for roughly 100 years. They have been tested in adversarial

Haack, Peer Review and Publication: Lessons for Lawyers, STETSON L. REV. (2007).

45 U.S. v. Bonds et al., 12 F.3d 540, 550 (3rd. Cir. 1993). 46 U.S. v. Havvard, 117 F.Supp. 2d 848, 852 (S.D. Ind. 2000).


16

proceedings with the highest possible stakes – liberty and sometimes life.47

I think we can be sure this wasn’t what Popper had in mind.

And in U. S v. Mitchell, ruling that the lower court did not abuse its discretion in

admitting latent-fingerprint identification testimony, the court first looks to Webster’s

Dictionary for a definition of “falsifiable”: “capable of being proved false, defeasible”;

then glosses this in a way that starts out sounding entirely Popperian: “for instance, the

hypothesis ‘all crows are black’ is falsifiable (because an albino crow could be found

tomorrow”),” but then wanders into more comfortable legal territory: “a clairvoyant’s

statement that he receives messages from dead relatives is not [falsifiable] (because there

is no way for the departed to deny this).” Then, after fudging “falsify” and “deny,” the

court argues that:

In this case the relevant premises were posed as explicit questions to many of the government experts: (1) Are human friction ridge arrangements unique and permanent? And (2) Can a positive identification be made from fingerprints containing sufficient quantity and quality of detail? The governments experts responded in the affirmative.

But then, apparently realizing that relying the FBI’s experts’ ipse dixit won’t quite do, the

court reverts to the same strictly-literal interpretation of the first Daubert factor we

encountered in Bonds: “We must consider not whether we agree as a factual matter ... but

rather whether these hypotheses are testable (or tested). We conclude that they are.”48

In U.S. v. Hines (1999) the court observes in a footnote that some commentators

had criticized the Daubert ruling for having “simply taken the definition of science from

Karl Popper, a definition that others have criticized as deriving from a culturally defined,

time-bound paradigm.”49 Perhaps this vaguely Kuhnian talk signals that Green’s

47 Id. at 854. 48 U.S. v. Mitchell, 365 F.2d 215, 235-6 (3rd Cir. 2004) 49 U.S. v. Hines, 55 F.Supp. 2d 62, 65 n.2 (U.S. District Ct., D. Mass, 1999) (emphasis


17

reference to “scientific methodology today” had not gone unnoticed.

Judge Flaum, who observes in Bradley v. Brown (1994) that “the court must weed

out the speculative hypothesis from the tested theory,”50 is evidently unaware that

according to Popper all scientific theories are speculative hypotheses. And, in line with

this, most courts that use the first, allegedly Popperian, Daubert factor simply assume

that a claim that has been tested but not falsified has thereby been shown to be valid or

reliable. For example, in U.S. v. Starzecpyzel (1995) – after holding a Daubert hearing on

forensic document identification testimony at the end of which the court ruled that such

testimony is not scientific, and hence falls outside the scope of Daubert! – the court

explained that “[t]he Daubert test is grounded in the scientific process and directs the

judge to evaluate the quality of the testing supporting the scientific conclusion.”51 We

also see the assumption that testing can not only falsify but also support a scientific claim

in Moore v. Ashland Chemical (1997), where the court relies on that fatal sentence from

Michael Green’s article, that while “[t]heoretically .... hypotheses are not affirmatively

proved, only falsified, .... [o]f course, if a hypothesis repeatedly withstands falsification,

one may tend to accept it ... as true.”52

And we see it again in Downs v. Prestorp. Components, Inc. (1999), where the

court reasons that:

[t]he scientific method must be an objective one. This is the essence of what the Supreme Court referred to as scientific validity, also known as falsifiability. It follows that if an expert’s methodology ... is not subject to be proven correct by objective standards, then than methodology is presumptively unreliable.53

added).

50 Bradley v. Brown, 852 F. Supp. 690, 700 (1994). 51 U.S. v. Starzecpyzel, 880 F.Supp. 1027 (S.D.N.Y. 1995), 1040 (emphasis added). 52 Moore v. Ashland Chemical,126 F.3d 679, 685 (5th Cir. 1997). 53 Downs v. Prestorp Components, Inc., 126 F.Supp. 2d 1090, 1127 (E.D. Tex. 1999)

(emphasis added).


18

Scientific validity (which Daubert had identified with evidentiary reliability, and defined

in a footnote as the testimony’s being trustworthy, i.e., showing what it purports to

show)54 is equated with falsifiability (which, however, is entirely consistent with the

testimony’s being plain false); then – adding insult to injury with that wholly

unwarranted “[i]t follows that” – the court throws in the radically un-Popperian idea of a

methodology’s being “proven to be correct.”

Again, in Rogers v. Secretary of Health and Human Services (1999), citing a law

review article by Black et al., the court reasons that “for scientists, a new idea or

explanation is not valid unless there is a possibility that empirical testing can prove it

false and until it has withstood thoughtful efforts at falsification.”55

Then there’s the gloss on the first Daubert factor given in Caraker v. Sandoz

Pharmaceuticals (2001) “The hallmark of [Daubert’s] reliability prong is the scientific

method, i.e., the generation of testable hypotheses which are then subjected to the real-

world crucible of experimentation, falsification/validation, and replication.”56 This

passage, and especially that fused phrase “falsification/validation,” encapsulates the

crucial misunderstanding in a nutshell; it is cited verbatim in several subsequent cases.57

54 Daubert, 509 U.S. 579, 590, n.9. 55 Rogers v. Secretary of Health and Human Services, 19999 WL 809824 Fed. C1

(emphasis added) (citing Bert Black , Francisco J. Ayala, and Carol Saffran-Brinks, Science and Law in the Wake of Daubert: A New Search for Scientific Knowledge, 72.4 TEX. L. REV. 715 (1994)). Black et al. write that “[t]esting either establishes or fails to establish falsehood; it never establishes absolute truth.” Id at 762 (emphasis added). Their word “absolute” hints that corroboration might establish provisional truth; and indeed, they continue, “[b]ecause the truth of scientific hypotheses can never be established conclusively, they can only be accepted contingently,” and “scientists do not have the same degree of confidence in all hypotheses that have survived falsification.”

56 Caraker v. Soldo Pharm. Corp, 188 F.Supp. 2d (S.D.Ill. 2001) at1030 (emphasis added).

57 See Krutsinger v. Pharmacia Corp., 2004 WL 5508617 (S.D. Ill.) at *4; Bickel v. Pfizer, Inc., 431 F.Supp. 2d 918 (N.D. Ind., 2006) at 922; Hardiman v. Davida, Inc., 2007 WL 1395568 (N.D. Ind.). at 000.; Bauer v. Bayer, A.G., 564 F.Supp. 2d 365 (M.D. Pa. 2008); Perry


19

In one of those cases, Soldo v.Sandoz Pharmaceuticals (2003), the court struggles

to explain what this amounts to in the case at hand:

To “falsify” a hypothesis in this context means to prove that the “null hypothesis” – that Parlodel has no effect on the risk of postpartum stroke – is false, i.e, that Parlodel in fact significantly increases the risk of postpartum stroke. The failure of plaintiff’s experts to show any study proving that the null hypothesis has been falsified demonstrates that their causal hypothesis has not been tested or verified by the means of science.58

But the allusions to falsifiability here are worse than useless. To begin with, how to apply

the criterion of falsifiability to complex statistical statements is a vexed question in itself.

Then there’s that characteristic fudge-phrase, “tested or verified.” But the deep point is

that all this passage really says is that the plaintiff’s experts have produced no evidence

supporting the claim that Parlodel increases the risk of postpartum stroke. Shoehorning

this into Popperian vocabulary does no good at all.

And in Fuesting v. Zimmer, Inc. (2005), supposedly applying Daubert’s Popperian

clause, the court writes that:

Pugh did not conduct any scientific tests or studies to bolster his theory relating polyethylene delamination to gamma irradiation in air, not did he produce or rely on any studies to verify his conclusions.59

Probably I should also mention the dog that didn’t bark. For the record: my search

of post-Daubert federal cases referring to Popper or to falsifiability turned up no instance

in which expert psychiatric testimony was excluded on grounds of unfalsifiability – a

significant omission, surely, given that Popper’s criterion of demarcation seems to have

been motivated, in large part, by a sense that neither Freud’s nor Jung’s psychological

theorizing could legitimately be counted as science.

v. Novartis Pharm., Inc., 564 F.Supp. 2d 452, 459 (E.D. Pa. 2008).

58 Soldo v. Sandoz Pharm. Corp., 244 F.Supp. 2d 000, 457 (W.D. Pa. 2003). 59 Fuesting v. Zimmer, Inc., 421 Fed. Reptr. 3d, 528 (2005), 536 (emphasis added).


20

Justice Rehnquist’s fear that federal judges wouldn’t know what to make of

Daubert’s allusions to Popper, falsifiability, etc., was certainly well-founded. More

importantly, though, as I will suggest in conclusion, the conception of science implicit in

many of these rulings – albeit in an inarticulate and half-baked form – is more plausible

than the account Popper actually offers.

6. The Sting in the Tale

To be sure, it is a mistake to fudge “scientific” together with “reliable,” as it seems

Justice Blackmun did. And arguably, at least, in requiring courts to assess the reliability

of expert testimony, Daubert fudged the legal line between questions of the admissibility

of evidence (a matter for courts to determine) and questions of its weight (a matter for

juries to decide). Nevertheless, these issues aside, in any case involving scientific

testimony the question of reliability is bound to arise, and must be determined somehow;

and unless there is such a thing as (objectively) supportive evidence, such determinations

could only be arbitrary. If Popper’s account of what science is and how it works were

true, the legal system’s interest in the question of the reliability of scientific testimony

would be simply misconceived.60

I don’t believe that concern is misconceived; and I don’t believe you do, either.

Think of Kerry Kotler, who was freed from prison in 1993 after DNA analysis revealed

that he was innocent of the rape for which he had served eleven years of a twenty-five to

fifty-year sentence; and three years later was convicted of another rape – of which DNA

evidence showed he was guilty. If you believe, as I do, that there is such a thing as

objectively better and worse evidence, it will seem to you that in this instance justice was

60 Indeed, if, as I have argued, Popper’s account is really a kind of closet skepticism, the

legal system’s interest in the reliability of testimony generally would be misconceived. But I can’t pursue this more general issue here. See also EVIDENCE AND INQUIRY 000, where I made


21

(probably)61 well-served by science; if you really believe, as Popper professes to, that

there is no such thing as objectively supportive evidence, this – and every legal case

turning on scientific evidence – you will be obliged to conclude that the hope that science

could contribute to justice is vain. This is not by itself an argument that Popper is

mistaken in his epistemology, only a way to make vivid how radical the consequences of

that epistemology would be for our – or any – legal system. Nevertheless, I believe,

courts’ misinterpretation of Popper results in an epistemology which is more plausible

than Popper’s official story. To see why, however, we need an understanding of science

which is both more adequate philosophically, and more adequate to the kinds of

evidentiary issues courts must determine.

Perhaps it will be suggested that we can readily arrive at that better

epistemological account simply by adding, to Popper’s official story, an acknowledgment

that a claim that has been tested but not falsified is thereby shown to be to some degree

reliable. But it should by now be clear that such an acknowledgment would change the

character of the falsificationist account so radically that the result should not be called

Popperian at all. Not only does Popper’s conventionalist account of the acceptance of

basic statements make it impossible to show that a scientific claim that has been, in his

sense, “falsified,” is actually false; but his repudiation of even the slightest whiff of

inductivism precludes him from acknowledging that we have grounds to believe even that

a claim that passed a certain test yesterday would pass the same test tomorrow, let alone

that a claim that has passed a variety of demanding tests thus far would likely pass future,

different tests.

Now, perhaps, it will be suggested that, since Popper’s account can’t be fixed

the same point with respect to Rorty’s repudiation of epistemology:

61 “Probably” because, without knowing a lot more than I do, I can’t say with absolute confidence that the DNA analyses in question were well-conducted, the chain of evidence


22

without radically changing its character, Justice Blackmun would have done better to

have relied on Hempel – who, unlike Popper, allows that scientific claims can be

confirmed as well as disconfirmed. And, indeed, Hempel’s inductivist approach is less

grossly unsuitable for the Supreme Courts’ purpose than Popper’s. Less grossly

unsuitable; but still not good enough. For while Hempel believes, what Popper denies:

that there is a logic of confirmation, he nevertheless joins Popper in the assumption that if

there is such a thing as supportive but not deductively conclusive evidence, this must be

representable in a formal, syntactically characterizable, inductive logic. While Hempel

believes, what Popper denies: that scientists’ observations have some bearing on the

epistemological status of an observation statement, he has no account to offer of how they

do so. And, just like Popper, Hempel assumes an atomistic approach, in the sense that he

focuses on the relation between scientific generalizations, such as “All ravens are black,”

and observation statements describing instances of such generalizations, such as “This is

a raven and this is black.” Even if Hempel’s “logic of confirmation” were viable – which

I don’t believe it is – it would be hopelessly inadequate to cope with the enormously

complex congeries of scientific evidence now routinely proffered in, for example, toxic-

tort litigation.

So we need, first, to go beyond the artificially simple generalizations and their

instances which for so long dominated mainstream philosophy of science. We need,

second, to acknowledge the articulated, ramifying complexity of the evidence with

respect to any serious scientific claim. We need, third, to recognize – as Hempel himself

eventually did, albeit obliquely, in response to the “grue” paradox62 – that the

supportiveness of evidence with respect to a claim depends, not solely on form, but on

impeccable, etc.

62 Carl G. Hempel, Postscript (1964) on Confirmation, in CARL G. HEMPEL, ASPECTS OF

SCIENTIFIC EXPLANATION 47 (1965), 51.


23

content, the relation of scientific vocabulary to the world (or, to put it another way, that

while there is such a thing as genuinely supportive evidence, the relation of

supportiveness is not purely formal, not a matter of logic in the narrow sense). And we

need, fourth, to take the role of experience seriously, to acknowledge that scientific

knowledge, like all empirical knowledge, ultimately depends on our interactions with the

world; and in consequence, since it is individuals who see, hear, etc., things and events

around them, we would need to start with an individual conception of warrant – the

degree to which a claim is warranted for an individual – and proceed from there to a

social and eventually to an impersonal conception.

And, eschewing the honorific use of “science” and its cognates, we should focus

on the difference between well- and poorly-conducted inquiry rather than remaining

obsessed with rooting out “pseudo-science.” Willingness to take negative evidence

seriously would then be seen to be – not a mark of the honest scientist specifically, much

less the mark of the scientist as such – but a mark of the honest inquirer in whatever field;

and that a proposed explanation rules out some possible upshots would be seen to be, not

a sign that it is scientific, but a sign that it is genuinely explanatory. This more adequate

picture would, I believe, look much like the epistemological account I developed in

Defending Science 63 – which acknowledges the true complexities of evidence,64 and,

unlike Popper’s or Hempel’s narrowly logical approaches, offers a conception of

scientific rationality in which the world and scientists’ interactions with it play an

essential role.65

63 SUSAN HAACK, DEFENDING SCIENCE – WITHIN REASON: BETWEEN SCIENTISM AND

CYNICISM (2003). 64 And which I have subsequently deployed to answer some serious questions about

compex causation evidence; see Susan Haack, Proving Causation: The Holism of Warrant and the Atomism of Daubert, J. OF BIOMEDICAL AND HEALTH LAW 2008 IV 253 (2008),

65 My thanks to Pamela Lucken for research assistance, and to Mark Migotti for helpful comments on a draft.

Documents

Popper on Trial: A Brief History of a Big Muddle Susan ... · Popper purports to offer a throughly fallibilist, but still fully cognitivist, picture. But what he really offers is