In Defense of the Likelihood Principle

James O. Berger, Vı́ctor Peña

Department of Statistical ScienceDuke University

ISBA 2016

1 / 21

Birnbaum’s Theorem

Birnbaum (1962) proves that theLikelihood Principle (LP) isimplied by the weak sufficiencyprinciple (WSP) and weakconditionality principle (WCP).

Huge result! WSP and WCPwould seem appealing to moststatisticians, but LP has strongimplications.

2 / 21

Today’s Talk

LP doesn’t allow p-values,reference priors, dependenceon stopping rules, etc.

There is a vast literature thatdiscusses the theorem and itsimplications (cf. Berger andWolpert (1988))

Recently, Evans (2013) andMayo (2014) have receivedconsiderable attention.

Our goal is reviewing anddiscussing their content.

3 / 21

Statistical Principles

4 / 21

WCP, AP, WSP

Weak Conditionality Principle (WCP): If you choose yourexperiment by flipping a coin, your inferences should onlydepend on the observed experiment.

Ancillarity Principle (AP): Your inferences shouldn’t changeafter observing (conditioning on) the value of a statistic whosedistribution doesn’t depend on unknown parameters.

Weak Sufficiency Principle (WSP): If T is a sufficientstatistic and x1 and x2 are outcomes such that T (x1) = T (x2),your inferences upon observing x1 or x2 should be the same.

5 / 21

LP and Birnbaum’s Theorem

Likelihood Principle (LP): If two experiments for the sameparameter have proportional likelihoods (given the observeddata), the inferences to be made from the experiments shouldbe the same.

Birnbaum’s theorem

WCP and WSP imply LP.

6 / 21

Evans (2013): Review and Discussion

7 / 21

Evans (2013): Main Result

Evans (2013) considers AP instead of WCP, but his mainobjection carries over if we consider WCP.

Define WSP, WCP, and LP as set relations S, C, and L,respectively (for example, (E1, x1) ∼S (E2, x2) whenever theyare “equivalent” according to WSP).

Then, S ∪ C 6= L.

8 / 21

Interpreting the Main Result

What does S ∪ C = L mean?If S ∪ C = L, the following statement would be true. Let E1 and E2be experiments about the same θ, and assume that x∗1 and x

∗2 are

such that f 1θ (x∗1 ) ∝ f 2θ (x∗2 ). Then, Ev(E1, x∗1 ) = Ev(E2, x∗2 ) by an

application of either WSP or WCP alone.

Clearly, S ∪ C = L is not implied by Birnbaum’s proof (and it’sactually false!).

Ev(E1, x∗1 )WCP= Ev(E , (1, x∗1 ))

WSP= Ev(E , (2, x∗2 ))

WCP= Ev(E2, x∗2 )

9 / 21

Conclusion in Evans (2013)

Evans (2013)

We have shown that the proof in [Birnbaum (1962)] did not provethat S and C lead to L. [...] The statement of Birnbaum’s theoremin prose should have been: if we accept the relation S and weaccept the relation C and we accept all the equivalencesgenerated by S and C together, then this is equivalent to acceptingL. The essential flaw in Birnbaum’s theorem lies in excluding thislast hypothesis from the statement of the theorem.

What are “all the equivalences generated by S and Ctogether”?

10 / 21

Equivalences and “Transitivity”

The equivalences generated by S and C are simply “chains”of applications of S and C.

Is this assumption really left out of the proof? To us, thenotation “Ev” and the “=” in the definitions of the principlesimply the “key assumption”.

And if it seems ambiguous, the proof itself makes it clear.

11 / 21

Is transitivity a strong condition?

Evans (2013) argues against the extension of AP to anequivalence relation because it is “essentially equivalent tosaying that it doesn’t matter which maximal ancillary wecondition on and it is unlikely that this is acceptable to mostfrequentist statisticians”.

If you truly believe in your principles, why shouldn’t you beable to apply them in any order you want, and as many timesas you want?

We wonder what Evans thinks of the proof of “WCP + WSPimplies LP.” Is combining applications of WCP and WSPreasonable?

12 / 21

Mayo (2014): Review and Discussion

13 / 21

Basic Notation: Methods and Inference

Mayo (2014) distinguishes between “methods” and“informative inference”.

New Notation: Methods and “Inferences”

M(E , x): Result of a applying a a statistical method when Eand x are taken as inputs.

I(E ′, x ′): Informative inference (conclusion/decision, etc.)that is made after seeing data X ′ = x ′ from E ′.

NB: I(E ′, x ′) may not be equal toM(E , x).

14 / 21

Principles in Mayo (2014)

WCP and WSP as in Mayo (2014), with our notation

WCP2: Given (Emix , (j, xj)), I(Emix , (j, xj)) =M(Ej , xj).

WSP2: If there exists a sufficient statistic T for θ andT (x) = T (x ′), thenM(E , x) =M(E , x ′).

With these definitions, we can construct examples whereWCP2 and WSP2 are respected, but LP isn’t.

15 / 21

Counterexample: Binomial vs Negative Binomial

Example in Mayo (2010)

Let E1 and E2 be Bin(n, θ) and NB(r , θ) experiments for thesame θ.

Let Emix denote the “coin-flip mixture experiment” between E1and E2.

Suppose both E1 and E2 yield x = (n− r , r), where n− r andr are the number of successes and failures observed afterperforming Ej .

16 / 21

Example in Mayo (2010) (cont.)

M(E , x) is the (usual) one-sided p-value:

M(E1, x) = P(Bin(n, θ0) ≥ n − r)M(E2, x) = P(NB(r , θ0) ≥ n − r)

M(Emix, x) =12[P(Bin(n, θ0) ≥ n − r) + P(NB(r , θ0) ≥ n − r)],

Inference is made using the rule I(Ej , x) =M(Ej , x) andI(Emix, (j, x)) =M(Ej , x).

For any given experiment, the number of successes is suff.for θ and the p-value doesn’t change for any value of x withthe same suff. stat.

It follows that WCP2 and WSP2 do not imply LP.

17 / 21

Example: Some Comments (I)

WSP2 only applies to the method (given some inputs), not thethe final inferences.

As a result, reporting the conditional p-value in the exampleabove isn’t a violation of WSP2.

If WSP2 is defined solely in terms of I, WSP2 and WCP2imply LP (in the Example we discussed, the existence of asuff. statistic requires that the inferences from Emix and Ej beequal).

The same happens if WCP2 were defined in terms ofM(p-values as a “method” would be precluded).

18 / 21

Example: Some Comments (II)

The notation “Ev(E , x)” is supposed to denote our “inferenceafter seeing E from x”, so the definitions should probably bewritten in terms of I alone.

In any case, that distinction is not made in Birnbaum (1962).

19 / 21

Conclusions

Evans (2013) claims that a notion of “transitivity of inferences”is a strong condition, but it seems that his criticisms apply toAP as a principle (and not “transitivity”).

Mayo (2010) makes a distinction between “methods” and“informative inferences”. This distinction is not made inBirnbaum (1962).

Conditional inference from a “p-value” perspective is hard.

20 / 21

References I

Berger, J. O. and Wolpert, R. L. (1988). The likelihood principle.Lecture notes-Monograph series.

Birnbaum, A. (1962). On the foundations of statistical inference.Journal of the American Statistical Association,57(298):269–306.

Evans, M. (2013). What does the proof of Birnbaum’s theoremprove? Electronic Journal of Statistics, 7:2645–2655.

Mayo, D. G. (2010). An error in the argument from conditionalityand sufficiency to the likelihood principle. Error and Inference:Recent Exchanges on Experimental Reasoning, Reliability, andthe Objectivity and Rationality of Science, page 305.

Mayo, D. G. (2014). On the Birnbaum argument for the stronglikelihood principle. Statistical Science, 29(2):227–239.

21 / 21