Confidence is epistemic probability · Probability from the Latin probabilitas, a measure of the authority \worthy of approbation" (Hacking 1975). Wahrscheinlichkeit: Gewissheit von

$Page 1: Confidence is epistemic probability · Probability from the Latin probabilitas, a measure of the authority \worthy of approbation" (Hacking 1975). Wahrscheinlichkeit: Gewissheit von$
Confidence is epistemic probability

Tore Schweder

Dept of EconomicsUniversity of Oslo

Oslo Workshop, 11. May 2015

Tore Schweder (Dept of Economics) Confidence is epistemic probability Oslo Workshop, 11. May 2015 1 / 17

Overview

1662

Confidence is epistemic probability

Confidence distributions

Confidence distributions do not obey the laws of probability

Neyman-Pearson for confidence distributions

Example: Fieller confidence

Example: Combining two independent posteriors for climate sensitivity

Example: What to add to nothing?

A mathematical theory of confidence?

Confidence distribution - the Fisherian synthesis


1662

Probability from the Latin probabilitas, a measure of the authority “worthyof approbation” (Hacking 1975). Wahrscheinlichkeit: Gewissheit vonVorhersagen, Sannsynlig: what seems true, earlier from what theauthorities claim.

By authority: before 1662

By (mathematical) reason after 1662


Epistemic and aleatory probability

Liber de ludo aleae (Book on games of chance, Girolamo Cardano 1663)

An aleatory probability is a property of frequency in nature or society

Epistemic: according to knowledge.

The 95% confidence interval for the Newtonian gravitational constant Gbased on the CODATA 2010 is (6.6723, 6.6754) in appropriate units(Milyukov and Fan, 2012).

P(6.6723 ≤ G ≤ 6.6754) = 0.95

is an epistemic probability.


Confidence distributions (CD)

Definition (Dimension 1)

C (θ; X ) is cdf of a CD when C (θ; x) is a cdf in θ for all x andC (θ0; X ) ∼ U(0, 1) when X ∼ fθ0

confidence curve cc(θ) = |2C (θ)− 1|: cc(θ0,X ) ∼ U

Definition (Dimension ≥ 1)

cc(θ; X ) is a confidence curve of a CD when min(cc(θ)) = 0,cc(θ; x) : θ → [0, 1] for all x and cc(θ0; X ) ∼ U(0, 1) when X ∼ fθ0

the level curve at β, {θ : cc(θ) ≤ β} is a confidence region of level β

the mutinormal CD has many different confidence curves: ellipsoidicKp((µ− µ)tΣ−1(µ− µ)), rectangular in eigen-directions, ...


Confidence curves for the probability of baby being small. Smokingmothers full line, non-smoking red dashed.


Loss, risk and optimality of CDsThe less dispersed a CD is the more informative it is. Measure dispersionof C (ψ, y) by convex functions Γ(ψ) ≥ 0, Γ(0) = 0:

loss: lo(ψ,C , x) =

∫Γ(ψ

′ − ψ)C (dψ′, x)

risk: R(ψ,C ) = E[lo(ψ,C ,X )]

Theorem (Neyman-Pearson for confidence distributions)

A confidence distribution C based on a sufficient statistic in which thelikelihood ratio is everywhere increasing, is uniformly optimal:P(lo(ψ,C ,X ) ≤ lo(psi ,C

′,X )) = 1 for all C

′and all Γ

Lemma (Otimality of conditional CD in exponential families)

When `(θ) = θts + k(θ), θt = (ψ, θ2, · · · , θd) the conditional CD basedon S1 given S2, · · · ,Sd is uniformly optimal.


Confidence distributions do not in general obeyKolmogorov’s laws of probability

Length problem, d = 1: X ∼ N(µ, 1), C (µ) = Φ(µ− X ). Forψ = |µ|, the derived distribution F (ψ; X ) = Φ(ψ − X )− Φ(−ψ − X )is not a CD since e.g. F (1; X ) is not uniformly distributed whenX ∼ N(1, 1).

length problem d > 1: X ∼ Nd(µ, I ). Then Nd(X , I ) is theconfidence distribution for µ and Kd(ψ; ncp = |X |2) is the deriveddistribution for ψ = |µ|2, the non-central chi-sq.Kd(ψ, |X |2) < U(0, 1) stochastically.

Distributions for derived parameters ψ = g(µ, σ) are obtained fromthe natural confidence distribution from a normal sample for (µ, σ) byintegration. Only for linear parameters ψ = aµ+ bσ are theseconfidence distributions (Pedersen 1978).


Example: The Fieller confidence distribution for a ratioa ∼ N(a, σ2

1), b ∼ N(b, σ22). The profile deviance for the ratio ψ = a/b is

D =(a− ψb)2

V (ψ), V (ψ) = σ2

1 + ψ2σ22 :

cc(ψ) = K (D(ψ); df = 1)

For a = 1.333, b = .333, ψ = 4.003, σ1 = σ2 = 1


Example: Combining two independent posteriordistributions of the climate sensitivityNicholas Lewis 2015

The climate sensitivity ψ is the increase in global surface temperatureresulting from doubling the amount of carbon dioxide in the atmosphere.IPCC Fifth Assessment Report gives Bayesian posteriors, p1(ψ) based onpaleo-data and p2(ψ) based on direct measurements. Independent!ψ is really a ratio a/b. If estimated from independent normals, theposterior (flat priors for a, b):

p(ψ) =bσ2

1 + aσ22ψ

V (ψ)3/2φ(

a− ψb

V (ψ)1/2)

Lewis found this to fit both the paleo posterior and the instrumentposterior (4 parameters).

Tore Schweder (Dept of Economics) Confidence is epistemic probabilityOslo Workshop, 11. May 2015 10 /

17

Figure 10-20b in IPCC-AR5, posteriors for climate sensitivity


17

Profile deviances and combined confidence curve

The recovered profile deviances are

Di (ψ) =(ai − ψbi )

2

Vi (ψ), i = paleo, instr .

This is also the implied likelihood of Efron (1993),L = exp(−1

2 (Φ−1(C (ψ)))2) .

Combined deviance

Dcomb(ψ) = Dpaleo + Dinstr −min (Dpaleo + Dinstr )

resulting in the confidence curve

cc(ψ) = K1(Dcomb(ψ)).


17

Example: More heart attacks when using an antidiabeticdrug?Nissen and Wolski (2007) considered 48 independent trials with Yi ,treated

deaths out of mi ,treated and Yi ,control deaths out ofmi ,control , i = 1, · · · , 48. Binomial model,

θi = log(pi ,control

1− pi ,control), θi + ψ = log(

pi ,treated

1− pi ,treated)

Exponential family model with

S1 =∑

Yi ,treated , Si+1 = Yi ,treated + Yi ,control .

Optimal CD

C (ψ) = Pψ(S1 > s1,obs |s2,obs , · · · , s49,obs)+1

2Pψ(S1 = s1,obs |s2,obs , · · · , s49,obs)

8 of the 48 triasl had Ytreated = Ycontrol = 0. They drop out of the CD.You should add nothing to nothing (Sweeting et. al (2004))!Tore Schweder (Dept of Economics) Confidence is epistemic probability

Oslo Workshop, 11. May 2015 13 /17

40 individual CDs for ψ and an optimally combined CD


17

A mathematical theory of confidence (fiducial probability)?Two attempts

Pitman (1939,1957) suggested to restrict the sets over which aconfidence distribution can be integrated for the result to be aconfidence

Hacking (1965) suggested two principlesI The Frequency Principle: support for a statement is the probability of

the statement being true.I The Principle of Irrelevance: irrelevant information should not alter the

support for a proposition. Implying that only models that can betransformed to location form can yield a confidence distribution.


17

Conclusion

Confidence distribution is the appropriate concept of epistemic probabilityfor empirical science. It is the Fisherian compromise between Bayesianismeand frequentisme.

Efron (1998): “My actual guess is that the old Fisher will have a verygood 21st century. The world of applied statistics seems to need aneffective compromise between Bayesian and frequentist ideas, and rightnow there is no substitute in sight for the Fisherian synthesis.”

But a mathematical theory for epistemic probability calculus for empiricalscience is needed. The math of epistemic probability has been neglected!(Fine 1977 when reviewing Shafer 1977).


17

Conclusion





17

Conclusion





17

ReferencesHacking, I. M.(1975, 2006). The Emergence of Probability: A Philosophical Study of Early Ideas About Probability,

Induction and Statistical Inference. Cambridge University Press, Cambridge. This is the third edition of the book, with anextended preface.

Efron, B. (1998). R.A. Fisher in the 21st century [with discussion and a rejoinder]. Statistical Science, 13:95-122.

Fine, T. L. (1977). Book review of Shafer: A mathematical theory of evidence. Bulletin of the American Statistical

Society, 83:667672.

Hacking, I. M. (1965). Logic of Statistical Inference. Cambridge University Press, Cambridge.

Lewis, N. (2015). Objectively combining instrumental period and paleoclimate sensitivity evidence. Unpublished

Milyukov, V. and Fan, S.-H. (2012). The Newtonian gravitational constant: Modern status of measurement and the new

CODATA value. Gravitation and Cosmology, 18:216-224.

Nissen, S. E. and Wolski, K. (2007). Effect of rosiglitazone on the risk of myocardial infarction and death from

cardiovascular causes. The New England Journal of Medicine, 356:2457-2471.

Pedersen, J. G. (1978). Fiducial inference. International Statistical Review, 146:147-170.

Pitman, E. J. G. (1939). The estimation of location and scale parameters of a continuous population of any given form.

Biometrika, 30:391-421.

Pitman, E. J. G. (1957). Statistics and science. Journal of the American Statistical Association, 52:322-330.

Schweder, T. and Hjort, N. L. (2015). Confidence, Likelihood, Probability: Statistical inference with confidence

distributions. Cambridge University Press. Forthcoming

Sweeting, M. J., Sutton, A. J., and Lambert, P. C. (2004). What to add to nothing? Use and avoidance of continuity

corrections in meta-analysis of sparse data. Statistics in Medicine, 23:1351-1375.


17

Documents

Confidence is epistemic probability · Probability from the Latin probabilitas, a measure of the authority \worthy of approbation" (Hacking 1975). Wahrscheinlichkeit: Gewissheit von