129
How approximate is Approximate Bayesian Computation? Christian P. Robert ISBA IWCBTA, Varanasi, Jan. 9, 2013 Joint work with J.-M. Cornuet, J.-M. Marin, K.L. Mengersen, N. Pillai, P. Pudlo and J. Rousseau

ABC in Varanasi

Embed Size (px)

Citation preview

Page 1: ABC in Varanasi

How approximate is Approximate BayesianComputation?

Christian P. RobertISBA IWCBTA, Varanasi, Jan. 9, 2013

Joint work with J.-M. Cornuet, J.-M. Marin,K.L. Mengersen, N. Pillai, P. Pudlo and J. Rousseau

Page 2: ABC in Varanasi

Advertisment

MCMSki IV to be held in Chamonix Mt Blanc, France, fromMonday, Jan. 6 to Wed., Jan. 8, 2014All aspects of MCMC++ theory and methodologyParallel (invited and contributed) sessions: call for proposals onwebsite http://www.pages.drexel.edu/ mwl25/mcmski/

Page 3: ABC in Varanasi

Outline

Unavailable likelihoods

ABC methods

ABC as an inference machine

ABCel

Page 4: ABC in Varanasi

Intractable likelihood

Case of a well-defined statistical model where the likelihoodfunction

`(θ|y) = f (y1, . . . , yn|θ)

I is (really!) not available in closed form

I can (easily!) be neither completed nor demarginalised

I cannot be estimated by an unbiased estimator

c© Prohibits direct implementation of a generic MCMC algorithmlike Metropolis–Hastings

Page 5: ABC in Varanasi

Intractable likelihood

Case of a well-defined statistical model where the likelihoodfunction

`(θ|y) = f (y1, . . . , yn|θ)

I is (really!) not available in closed form

I can (easily!) be neither completed nor demarginalised

I cannot be estimated by an unbiased estimator

c© Prohibits direct implementation of a generic MCMC algorithmlike Metropolis–Hastings

Page 6: ABC in Varanasi

The abc alternative

Approximations to the original B problem

I Degrading the precision down to a tolerance ε

I Replacing the likelihood with a non-parametric approximation

I Summarising the data with insufficient statistics

Page 7: ABC in Varanasi

The abc alternative

Approximations to the original B problem

I Degrading the precision down to a tolerance ε

I Replacing the likelihood with a non-parametric approximation

I Summarising the data with insufficient statistics

Page 8: ABC in Varanasi

The abc alternative

Approximations to the original B problem

I Degrading the precision down to a tolerance ε

I Replacing the likelihood with a non-parametric approximation

I Summarising the data with insufficient statistics

Page 9: ABC in Varanasi

Different worries about abc

Impact on B inference

I a mere computational issue (that will eventually end up beingsolved by more powerful computers, &tc, even if too costly inthe short term, as for regular Monte Carlo methods)

I an inferential issue (opening opportunities for new inferencemachine, with legitimity different than for classical Bapproach)

I a Bayesian conundrum (how closely related to the/a Bapproach?)

Page 10: ABC in Varanasi

Different worries about abc

Impact on B inference

I a mere computational issue (that will eventually end up beingsolved by more powerful computers, &tc, even if too costly inthe short term, as for regular Monte Carlo methods)

I an inferential issue (opening opportunities for new inferencemachine, with legitimity different than for classical Bapproach)

I a Bayesian conundrum (how closely related to the/a Bapproach?)

Page 11: ABC in Varanasi

Different worries about abc

Impact on B inference

I a mere computational issue (that will eventually end up beingsolved by more powerful computers, &tc, even if too costly inthe short term, as for regular Monte Carlo methods)

I an inferential issue (opening opportunities for new inferencemachine, with legitimity different than for classical Bapproach)

I a Bayesian conundrum (how closely related to the/a Bapproach?)

Page 12: ABC in Varanasi

Econom’ections

Similar exploration of simulation-based and approximationtechniques in Econometrics

I Simulated method of moments

I Method of simulated moments

I Simulated pseudo-maximum-likelihood

I Indirect inference

[Gourieroux & Monfort, 1996]

even though motivation is partly-defined models rather thancomplex likelihoods

Page 13: ABC in Varanasi

Econom’ections

Similar exploration of simulation-based and approximationtechniques in Econometrics

I Simulated method of moments

I Method of simulated moments

I Simulated pseudo-maximum-likelihood

I Indirect inference

[Gourieroux & Monfort, 1996]

even though motivation is partly-defined models rather thancomplex likelihoods

Page 14: ABC in Varanasi

Indirect inference

Minimise [in θ] a distance between estimators β based on apseudo-model for genuine observations and for observationssimulated under the true model and the parameter θ.

[Gourieroux, Monfort, & Renault, 1993;Smith, 1993; Gallant & Tauchen, 1996]

Page 15: ABC in Varanasi

Indirect inference (PML vs. PSE)

Example of the pseudo-maximum-likelihood (PML)

β(y) = arg maxβ

∑t

log f ?(yt |β, y1:(t−1))

leading to

arg minθ

||β(yo) − β(y1(θ), . . . ,yS(θ))||2

whenys(θ) ∼ f (y|θ) s = 1, . . . ,S

Page 16: ABC in Varanasi

Indirect inference (PML vs. PSE)

Example of the pseudo-score-estimator (PSE)

β(y) = arg minβ

{∑t

∂ log f ?

∂β(yt |β, y1:(t−1))

}2

leading to

arg minθ

||β(yo) − β(y1(θ), . . . ,yS(θ))||2

whenys(θ) ∼ f (y|θ) s = 1, . . . ,S

Page 17: ABC in Varanasi

Consistent indirect inference

“...in order to get a unique solution the dimension ofthe auxiliary parameter β must be larger than or equal tothe dimension of the initial parameter θ. If the problem isjust identified the different methods become easier...”

Consistency depending on the criterion and on the asymptoticidentifiability of θ

[Gourieroux & Monfort, 1996, p. 66]

Which connection [if any] with the B perspective?

Page 18: ABC in Varanasi

Consistent indirect inference

“...in order to get a unique solution the dimension ofthe auxiliary parameter β must be larger than or equal tothe dimension of the initial parameter θ. If the problem isjust identified the different methods become easier...”

Consistency depending on the criterion and on the asymptoticidentifiability of θ

[Gourieroux & Monfort, 1996, p. 66]

Which connection [if any] with the B perspective?

Page 19: ABC in Varanasi

Consistent indirect inference

“...in order to get a unique solution the dimension ofthe auxiliary parameter β must be larger than or equal tothe dimension of the initial parameter θ. If the problem isjust identified the different methods become easier...”

Consistency depending on the criterion and on the asymptoticidentifiability of θ

[Gourieroux & Monfort, 1996, p. 66]

Which connection [if any] with the B perspective?

Page 20: ABC in Varanasi

Approximate Bayesian computation

Unavailable likelihoods

ABC methodsGenesis of ABCABC basicsAdvances and interpretationsABC as knn

ABC as an inference machine

ABCel

Page 21: ABC in Varanasi

Genetic background of ABC

skip genetics

ABC is a recent computational technique that only requires beingable to sample from the likelihood f (·|θ)

This technique stemmed from population genetics models, about15 years ago, and population geneticists still contributesignificantly to methodological developments of ABC.

[Griffith & al., 1997; Tavare & al., 1999]

Page 22: ABC in Varanasi

Demo-genetic inference

Each model is characterized by a set of parameters θ that coverhistorical (time divergence, admixture time ...), demographics(population sizes, admixture rates, migration rates, ...) and genetic(mutation rate, ...) factors

The goal is to estimate these parameters from a dataset ofpolymorphism (DNA sample) y observed at the present time

Problem:

most of the time, we cannot calculate the likelihood of thepolymorphism data f (y|θ)...

Page 23: ABC in Varanasi

Demo-genetic inference

Each model is characterized by a set of parameters θ that coverhistorical (time divergence, admixture time ...), demographics(population sizes, admixture rates, migration rates, ...) and genetic(mutation rate, ...) factors

The goal is to estimate these parameters from a dataset ofpolymorphism (DNA sample) y observed at the present time

Problem:

most of the time, we cannot calculate the likelihood of thepolymorphism data f (y|θ)...

Page 24: ABC in Varanasi

Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium

Sample of 8 genes

Mutations according tothe Simple stepwiseMutation Model(SMM)• date of the mutations ∼

Poisson process withintensity θ/2 over thebranches• MRCA = 100• independent mutations:±1 with pr. 1/2

Page 25: ABC in Varanasi

Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium

Kingman’s genealogyWhen time axis isnormalized,T (k) ∼ Exp(k(k − 1)/2)

Mutations according tothe Simple stepwiseMutation Model(SMM)• date of the mutations ∼

Poisson process withintensity θ/2 over thebranches• MRCA = 100• independent mutations:±1 with pr. 1/2

Page 26: ABC in Varanasi

Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium

Kingman’s genealogyWhen time axis isnormalized,T (k) ∼ Exp(k(k − 1)/2)

Mutations according tothe Simple stepwiseMutation Model(SMM)• date of the mutations ∼

Poisson process withintensity θ/2 over thebranches• MRCA = 100• independent mutations:±1 with pr. 1/2

Page 27: ABC in Varanasi

Neutral model at a given microsatellite locus, in a closedpanmictic population at equilibrium

Observations: leafs of the treeθ = ?

Kingman’s genealogyWhen time axis isnormalized,T (k) ∼ Exp(k(k − 1)/2)

Mutations according tothe Simple stepwiseMutation Model(SMM)• date of the mutations ∼

Poisson process withintensity θ/2 over thebranches• MRCA = 100• independent mutations:±1 with pr. 1/2

Page 28: ABC in Varanasi

Much more interesting models. . .

I several independent locusIndependent gene genealogies and mutations

I different populationslinked by an evolutionary scenario made of divergences,admixtures, migrations between populations, etc.

I larger sample sizeusually between 50 and 100 genes

A typical evolutionary scenario:

MRCA

POP 0 POP 1 POP 2

τ1

τ2

Page 29: ABC in Varanasi

Intractable likelihood

Missing (too missing!) data structure:

f (y|θ) =

∫Gf (y|G ,θ)f (G |θ)dG

cannot be computed in a manageable way...

The genealogies are considered as nuisance parameters

This modelling clearly differs from the phylogenetic perspective

where the tree is the parameter of interest.

Page 30: ABC in Varanasi

Intractable likelihood

Missing (too missing!) data structure:

f (y|θ) =

∫Gf (y|G ,θ)f (G |θ)dG

cannot be computed in a manageable way...

The genealogies are considered as nuisance parameters

This modelling clearly differs from the phylogenetic perspective

where the tree is the parameter of interest.

Page 31: ABC in Varanasi

A?B?C?

I A stands for approximate[wrong likelihood /picture]

I B stands for Bayesian

I C stands for computation[producing a parametersample]

Page 32: ABC in Varanasi

A?B?C?

I A stands for approximate[wrong likelihood /picture]

I B stands for Bayesian

I C stands for computation[producing a parametersample]

Page 33: ABC in Varanasi

A?B?C?

I A stands for approximate[wrong likelihood /picture]

I B stands for Bayesian

I C stands for computation[producing a parametersample]

ESS=155.6

θ

Den

sity

−0.5 0.0 0.5 1.0

0.0

1.0

ESS=75.93

θ

Den

sity

−0.4 −0.2 0.0 0.2 0.4

0.0

1.0

2.0

ESS=76.87

θ

Den

sity

−0.4 −0.2 0.0 0.2

01

23

4

ESS=91.54

θ

Den

sity

−0.6 −0.4 −0.2 0.0 0.2

01

23

4

ESS=108.4

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.0

2.0

3.0

ESS=85.13

θ

Den

sity

−0.2 0.0 0.2 0.4 0.6

0.0

1.0

2.0

3.0

ESS=149.1

θ

Den

sity

−0.5 0.0 0.5 1.0

0.0

1.0

2.0

ESS=96.31

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.0

2.0

ESS=83.77

θ

Den

sity

−0.6 −0.4 −0.2 0.0 0.2 0.4

01

23

4

ESS=155.7

θ

Den

sity

−0.5 0.0 0.5

0.0

1.0

2.0

ESS=92.42

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.0

2.0

3.0 ESS=95.01

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.5

3.0

ESS=139.2

Den

sity

−0.6 −0.2 0.2 0.6

0.0

1.0

2.0

ESS=99.33

Den

sity

−0.4 −0.2 0.0 0.2 0.4

0.0

1.0

2.0

3.0

ESS=87.28

Den

sity

−0.2 0.0 0.2 0.4 0.6

01

23

Page 34: ABC in Varanasi

How Bayesian is aBc?

Could we turn the resolution into a Bayesian answer?

I ideally so (not meaningfull: requires ∞-ly powerful computer

I approximation error unknown (w/o costly simulation)

I true Bayes for wrong model (formal and artificial)

I true Bayes for noisy model (much more convincing)

I true Bayes for estimated likelihood (back to econometrics?)

I illuminating the tension between information and precision

Page 35: ABC in Varanasi

Untractable likelihood

Back to stage zero: what can we dowhen a likelihood function f (y|θ) iswell-defined but impossible / toocostly to compute...?

I MCMC cannot be implemented!

I shall we give up Bayesianinference altogether?!

I

Page 36: ABC in Varanasi

Untractable likelihood

Back to stage zero: what can we dowhen a likelihood function f (y|θ) iswell-defined but impossible / toocostly to compute...?

I MCMC cannot be implemented!

I shall we give up Bayesianinference altogether?!

I or settle for an almost Bayesianinference/picture...?

Page 37: ABC in Varanasi

ABC methodology

Bayesian setting: target is π(θ)f (x |θ)When likelihood f (x |θ) not in closed form, likelihood-free rejectiontechnique:

Foundation

For an observation y ∼ f (y|θ), under the prior π(θ), if one keepsjointly simulating

θ′ ∼ π(θ) , z ∼ f (z|θ′) ,

until the auxiliary variable z is equal to the observed value, z = y,then the selected

θ′ ∼ π(θ|y)

[Rubin, 1984; Diggle & Gratton, 1984; Tavare et al., 1997]

Page 38: ABC in Varanasi

ABC methodology

Bayesian setting: target is π(θ)f (x |θ)When likelihood f (x |θ) not in closed form, likelihood-free rejectiontechnique:

Foundation

For an observation y ∼ f (y|θ), under the prior π(θ), if one keepsjointly simulating

θ′ ∼ π(θ) , z ∼ f (z|θ′) ,

until the auxiliary variable z is equal to the observed value, z = y,then the selected

θ′ ∼ π(θ|y)

[Rubin, 1984; Diggle & Gratton, 1984; Tavare et al., 1997]

Page 39: ABC in Varanasi

ABC methodology

Bayesian setting: target is π(θ)f (x |θ)When likelihood f (x |θ) not in closed form, likelihood-free rejectiontechnique:

Foundation

For an observation y ∼ f (y|θ), under the prior π(θ), if one keepsjointly simulating

θ′ ∼ π(θ) , z ∼ f (z|θ′) ,

until the auxiliary variable z is equal to the observed value, z = y,then the selected

θ′ ∼ π(θ|y)

[Rubin, 1984; Diggle & Gratton, 1984; Tavare et al., 1997]

Page 40: ABC in Varanasi

A as A...pproximative

When y is a continuous random variable, strict equality z = y isreplaced with a tolerance zone

ρ(y, z) 6 ε

where ρ is a distanceOutput distributed from

π(θ)Pθ{ρ(y, z) < ε}def∝ π(θ|ρ(y, z) < ε)

[Pritchard et al., 1999]

Page 41: ABC in Varanasi

A as A...pproximative

When y is a continuous random variable, strict equality z = y isreplaced with a tolerance zone

ρ(y, z) 6 ε

where ρ is a distanceOutput distributed from

π(θ)Pθ{ρ(y, z) < ε}def∝ π(θ|ρ(y, z) < ε)

[Pritchard et al., 1999]

Page 42: ABC in Varanasi

ABC algorithm

In most implementations, further degree of A...pproximation:

Algorithm 1 Likelihood-free rejection sampler

for i = 1 to N dorepeat

generate θ ′ from the prior distribution π(·)generate z from the likelihood f (·|θ ′)

until ρ{η(z),η(y)} 6 εset θi = θ

end for

where η(y) defines a (not necessarily sufficient) statistic

Page 43: ABC in Varanasi

Output

The likelihood-free algorithm samples from the marginal in z of:

πε(θ, z|y) =π(θ)f (z|θ)IAε,y(z)∫

Aε,y×Θ π(θ)f (z|θ)dzdθ,

where Aε,y = {z ∈ D|ρ(η(z),η(y)) < ε}.

The idea behind ABC is that the summary statistics coupled with asmall tolerance should provide a good approximation of theposterior distribution:

πε(θ|y) =

∫πε(θ, z|y)dz ≈ π(θ|y) .

...does it?!

Page 44: ABC in Varanasi

Output

The likelihood-free algorithm samples from the marginal in z of:

πε(θ, z|y) =π(θ)f (z|θ)IAε,y(z)∫

Aε,y×Θ π(θ)f (z|θ)dzdθ,

where Aε,y = {z ∈ D|ρ(η(z),η(y)) < ε}.

The idea behind ABC is that the summary statistics coupled with asmall tolerance should provide a good approximation of theposterior distribution:

πε(θ|y) =

∫πε(θ, z|y)dz ≈ π(θ|y) .

...does it?!

Page 45: ABC in Varanasi

Output

The likelihood-free algorithm samples from the marginal in z of:

πε(θ, z|y) =π(θ)f (z|θ)IAε,y(z)∫

Aε,y×Θ π(θ)f (z|θ)dzdθ,

where Aε,y = {z ∈ D|ρ(η(z),η(y)) < ε}.

The idea behind ABC is that the summary statistics coupled with asmall tolerance should provide a good approximation of theposterior distribution:

πε(θ|y) =

∫πε(θ, z|y)dz ≈ π(θ|y) .

...does it?!

Page 46: ABC in Varanasi

Output

The likelihood-free algorithm samples from the marginal in z of:

πε(θ, z|y) =π(θ)f (z|θ)IAε,y(z)∫

Aε,y×Θ π(θ)f (z|θ)dzdθ,

where Aε,y = {z ∈ D|ρ(η(z),η(y)) < ε}.

The idea behind ABC is that the summary statistics coupled with asmall tolerance should provide a good approximation of therestricted posterior distribution:

πε(θ|y) =

∫πε(θ, z|y)dz ≈ π(θ|η(y)) .

Not so good..!skip convergence details!

Page 47: ABC in Varanasi

Convergence of ABC

What happens when ε→ 0?

For B ⊂ Θ, we have∫B

∫Aε,y

f (z|θ)dz∫Aε,y×Θ π(θ)f (z|θ)dzdθ

π(θ)dθ =

∫Aε,y

∫B f (z|θ)π(θ)dθ∫

Aε,y×Θ π(θ)f (z|θ)dzdθdz

=

∫Aε,y

∫B f (z|θ)π(θ)dθ

m(z)

m(z)∫Aε,y×Θ π(θ)f (z|θ)dzdθ

dz

=

∫Aε,y

π(B |z)m(z)∫

Aε,y×Θ π(θ)f (z|θ)dzdθdz

which indicates convergence for a continuous π(B |z).

Page 48: ABC in Varanasi

Convergence of ABC

What happens when ε→ 0?

For B ⊂ Θ, we have∫B

∫Aε,y

f (z|θ)dz∫Aε,y×Θ π(θ)f (z|θ)dzdθ

π(θ)dθ =

∫Aε,y

∫B f (z|θ)π(θ)dθ∫

Aε,y×Θ π(θ)f (z|θ)dzdθdz

=

∫Aε,y

∫B f (z|θ)π(θ)dθ

m(z)

m(z)∫Aε,y×Θ π(θ)f (z|θ)dzdθ

dz

=

∫Aε,y

π(B |z)m(z)∫

Aε,y×Θ π(θ)f (z|θ)dzdθdz

which indicates convergence for a continuous π(B |z).

Page 49: ABC in Varanasi

Convergence (do not attempt!)

...and the above does not apply to insufficient statistics:

If η(y) is not a sufficient statistics, the best one can hope for is

π(θ|η(y)) , not π(θ|y)

If η(y) is an ancillary statistic, the whole information contained iny is lost!, the “best” one can “hope” for is

π(θ|η(y)) = π(θ)

Bummer!!!

Page 50: ABC in Varanasi

Convergence (do not attempt!)

...and the above does not apply to insufficient statistics:

If η(y) is not a sufficient statistics, the best one can hope for is

π(θ|η(y)) , not π(θ|y)

If η(y) is an ancillary statistic, the whole information contained iny is lost!, the “best” one can “hope” for is

π(θ|η(y)) = π(θ)

Bummer!!!

Page 51: ABC in Varanasi

Convergence (do not attempt!)

...and the above does not apply to insufficient statistics:

If η(y) is not a sufficient statistics, the best one can hope for is

π(θ|η(y)) , not π(θ|y)

If η(y) is an ancillary statistic, the whole information contained iny is lost!, the “best” one can “hope” for is

π(θ|η(y)) = π(θ)

Bummer!!!

Page 52: ABC in Varanasi

Convergence (do not attempt!)

...and the above does not apply to insufficient statistics:

If η(y) is not a sufficient statistics, the best one can hope for is

π(θ|η(y)) , not π(θ|y)

If η(y) is an ancillary statistic, the whole information contained iny is lost!, the “best” one can “hope” for is

π(θ|η(y)) = π(θ)

Bummer!!!

Page 53: ABC in Varanasi

Comments

I Role of distance paramount (because ε 6= 0)

I Scaling of components of η(y) is also determinant

I ε matters little if “small enough”

I representative of “curse of dimensionality”

I small is beautiful!

I the data as a whole may be paradoxically weakly informativefor ABC

Page 54: ABC in Varanasi

ABC (simul’) advances

how approximative is ABC? ABC as knn

Simulating from the prior is often poor in efficiencyEither modify the proposal distribution on θ to increase the densityof x ’s within the vicinity of y ...

[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]

...or by viewing the problem as a conditional density estimationand by developing techniques to allow for larger ε

[Beaumont et al., 2002]

.....or even by including ε in the inferential framework [ABCµ][Ratmann et al., 2009]

Page 55: ABC in Varanasi

ABC (simul’) advances

how approximative is ABC? ABC as knn

Simulating from the prior is often poor in efficiencyEither modify the proposal distribution on θ to increase the densityof x ’s within the vicinity of y ...

[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]

...or by viewing the problem as a conditional density estimationand by developing techniques to allow for larger ε

[Beaumont et al., 2002]

.....or even by including ε in the inferential framework [ABCµ][Ratmann et al., 2009]

Page 56: ABC in Varanasi

ABC (simul’) advances

how approximative is ABC? ABC as knn

Simulating from the prior is often poor in efficiencyEither modify the proposal distribution on θ to increase the densityof x ’s within the vicinity of y ...

[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]

...or by viewing the problem as a conditional density estimationand by developing techniques to allow for larger ε

[Beaumont et al., 2002]

.....or even by including ε in the inferential framework [ABCµ][Ratmann et al., 2009]

Page 57: ABC in Varanasi

ABC (simul’) advances

how approximative is ABC? ABC as knn

Simulating from the prior is often poor in efficiencyEither modify the proposal distribution on θ to increase the densityof x ’s within the vicinity of y ...

[Marjoram et al, 2003; Bortot et al., 2007, Sisson et al., 2007]

...or by viewing the problem as a conditional density estimationand by developing techniques to allow for larger ε

[Beaumont et al., 2002]

.....or even by including ε in the inferential framework [ABCµ][Ratmann et al., 2009]

Page 58: ABC in Varanasi

ABC-NP

Better usage of [prior] simulations byadjustement: instead of throwing awayθ ′ such that ρ(η(z),η(y)) > ε, replaceθ’s with locally regressed transforms

θ∗ = θ− {η(z) − η(y)}Tβ[Csillery et al., TEE, 2010]

where β is obtained by [NP] weighted least square regression on(η(z) − η(y)) with weights

Kδ {ρ(η(z),η(y))}

[Beaumont et al., 2002, Genetics]

Page 59: ABC in Varanasi

ABC-NP (regression)

Also found in the subsequent literature, e.g. in Fearnhead-Prangle (2012) :weight directly simulation by

Kδ {ρ(η(z(θ)),η(y))}

or

1

S

S∑s=1

Kδ {ρ(η(zs(θ)),η(y))}

[consistent estimate of f (η|θ)]Curse of dimensionality: poor estimate when d = dim(η) is large...

Page 60: ABC in Varanasi

ABC-NP (regression)

Also found in the subsequent literature, e.g. in Fearnhead-Prangle (2012) :weight directly simulation by

Kδ {ρ(η(z(θ)),η(y))}

or

1

S

S∑s=1

Kδ {ρ(η(zs(θ)),η(y))}

[consistent estimate of f (η|θ)]Curse of dimensionality: poor estimate when d = dim(η) is large...

Page 61: ABC in Varanasi

ABC-NP (density estimation)

Use of the kernel weights

Kδ {ρ(η(z(θ)),η(y))}

leads to the NP estimate of the posterior expectation∑i θiKδ {ρ(η(z(θi )),η(y))}∑i Kδ {ρ(η(z(θi )),η(y))}

[Blum, JASA, 2010]

Page 62: ABC in Varanasi

ABC-NP (density estimation)

Use of the kernel weights

Kδ {ρ(η(z(θ)),η(y))}

leads to the NP estimate of the posterior conditional density∑i Kb(θi − θ)Kδ {ρ(η(z(θi )),η(y))}∑

i Kδ {ρ(η(z(θi )),η(y))}

[Blum, JASA, 2010]

Page 63: ABC in Varanasi

ABC-NP (density estimations)

Other versions incorporating regression adjustments∑i Kb(θ

∗i − θ)Kδ {ρ(η(z(θi )),η(y))}∑i Kδ {ρ(η(z(θi )),η(y))}

In all cases, error

E[g(θ|y)] − g(θ|y) = cb2 + cδ2 + OP(b2 + δ2) + OP(1/nδ

d)

var(g(θ|y)) =c

nbδd(1 + oP(1))

Page 64: ABC in Varanasi

ABC-NP (density estimations)

Other versions incorporating regression adjustments∑i Kb(θ

∗i − θ)Kδ {ρ(η(z(θi )),η(y))}∑i Kδ {ρ(η(z(θi )),η(y))}

In all cases, error

E[g(θ|y)] − g(θ|y) = cb2 + cδ2 + OP(b2 + δ2) + OP(1/nδ

d)

var(g(θ|y)) =c

nbδd(1 + oP(1))

[Blum, JASA, 2010]

Page 65: ABC in Varanasi

ABC-NP (density estimations)

Other versions incorporating regression adjustments∑i Kb(θ

∗i − θ)Kδ {ρ(η(z(θi )),η(y))}∑i Kδ {ρ(η(z(θi )),η(y))}

In all cases, error

E[g(θ|y)] − g(θ|y) = cb2 + cδ2 + OP(b2 + δ2) + OP(1/nδ

d)

var(g(θ|y)) =c

nbδd(1 + oP(1))

[standard NP calculations]

Page 66: ABC in Varanasi

ABC as knn

[Biau et al., 2012, arxiv:1207.6461]

Practice of ABC: determine tolerance ε as a quantile on observeddistances, say 10% or 1% quantile,

ε = εN = qα(d1, . . . , dN)

I Interpretation of ε as nonparametric bandwidth onlyapproximation of the actual practice

[Blum & Francois, 2010]

I ABC is a k-nearest neighbour (knn) method with kN = NεN[Loftsgaarden & Quesenberry, 1965]

Page 67: ABC in Varanasi

ABC as knn

[Biau et al., 2012, arxiv:1207.6461]

Practice of ABC: determine tolerance ε as a quantile on observeddistances, say 10% or 1% quantile,

ε = εN = qα(d1, . . . , dN)

I Interpretation of ε as nonparametric bandwidth onlyapproximation of the actual practice

[Blum & Francois, 2010]

I ABC is a k-nearest neighbour (knn) method with kN = NεN[Loftsgaarden & Quesenberry, 1965]

Page 68: ABC in Varanasi

ABC as knn

[Biau et al., 2012, arxiv:1207.6461]

Practice of ABC: determine tolerance ε as a quantile on observeddistances, say 10% or 1% quantile,

ε = εN = qα(d1, . . . , dN)

I Interpretation of ε as nonparametric bandwidth onlyapproximation of the actual practice

[Blum & Francois, 2010]

I ABC is a k-nearest neighbour (knn) method with kN = NεN[Loftsgaarden & Quesenberry, 1965]

Page 69: ABC in Varanasi

ABC consistency

Provided

kN/ log logN −→∞ and kN/N −→ 0

as N →∞, for almost all s0 (with respect to the distribution ofS), with probability 1,

1

kN

kN∑j=1

ϕ(θj) −→ E[ϕ(θj)|S = s0]

[Devroye, 1982]

Biau et al. (2012) also recall pointwise and integrated mean square errorconsistency results on the corresponding kernel estimate of theconditional posterior distribution, under constraints

kN →∞, kN/N → 0, hN → 0 and hpNkN →∞,

Page 70: ABC in Varanasi

ABC consistency

Provided

kN/ log logN −→∞ and kN/N −→ 0

as N →∞, for almost all s0 (with respect to the distribution ofS), with probability 1,

1

kN

kN∑j=1

ϕ(θj) −→ E[ϕ(θj)|S = s0]

[Devroye, 1982]

Biau et al. (2012) also recall pointwise and integrated mean square errorconsistency results on the corresponding kernel estimate of theconditional posterior distribution, under constraints

kN →∞, kN/N → 0, hN → 0 and hpNkN →∞,

Page 71: ABC in Varanasi

Rates of convergence

Further assumptions (on target and kernel) allow for precise(integrated mean square) convergence rates (as a power of thesample size N), derived from classical k-nearest neighbourregression, like

I when m = 1, 2, 3, kN ≈ N(p+4)/(p+8) and rate N− 4p+8

I when m = 4, kN ≈ N(p+4)/(p+8) and rate N− 4p+8 logN

I when m > 4, kN ≈ N(p+4)/(m+p+4) and rate N− 4m+p+4

[Biau et al., 2012, arxiv:1207.6461]

Drag: Only applies to sufficient summary statistics

Page 72: ABC in Varanasi

Rates of convergence

Further assumptions (on target and kernel) allow for precise(integrated mean square) convergence rates (as a power of thesample size N), derived from classical k-nearest neighbourregression, like

I when m = 1, 2, 3, kN ≈ N(p+4)/(p+8) and rate N− 4p+8

I when m = 4, kN ≈ N(p+4)/(p+8) and rate N− 4p+8 logN

I when m > 4, kN ≈ N(p+4)/(m+p+4) and rate N− 4m+p+4

[Biau et al., 2012, arxiv:1207.6461]

Drag: Only applies to sufficient summary statistics

Page 73: ABC in Varanasi

ABC inference machine

Unavailable likelihoods

ABC methods

ABC as an inference machineError inc.Exact BC and approximatetargetssummary statistic

ABCel

Page 74: ABC in Varanasi

How much Bayesian aBc is..?

I maybe a convergent method of inference (meaningful?sufficient? foreign?)

I approximation error unknown (w/o simulation)

I pragmatic Bayes (there is no other solution!)

I many calibration issues (tolerance, distance, statistics)

I the NP side should be incorporated into the whole B picture

to ABCel

Page 75: ABC in Varanasi

ABCµ

Idea Infer about the error as well as about the parameter:Use of a joint density

f (θ, ε|y) ∝ ξ(ε|y, θ)× πθ(θ)× πε(ε)

where y is the data, and ξ(ε|y, θ) is the prior predictive density ofρ(η(z),η(y)) given θ and y when z ∼ f (z|θ)Warning! Replacement of ξ(ε|y, θ) with a non-parametric kernelapproximation.

[Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]

Page 76: ABC in Varanasi

ABCµ

Idea Infer about the error as well as about the parameter:Use of a joint density

f (θ, ε|y) ∝ ξ(ε|y, θ)× πθ(θ)× πε(ε)

where y is the data, and ξ(ε|y, θ) is the prior predictive density ofρ(η(z),η(y)) given θ and y when z ∼ f (z|θ)Warning! Replacement of ξ(ε|y, θ) with a non-parametric kernelapproximation.

[Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]

Page 77: ABC in Varanasi

ABCµ

Idea Infer about the error as well as about the parameter:Use of a joint density

f (θ, ε|y) ∝ ξ(ε|y, θ)× πθ(θ)× πε(ε)

where y is the data, and ξ(ε|y, θ) is the prior predictive density ofρ(η(z),η(y)) given θ and y when z ∼ f (z|θ)Warning! Replacement of ξ(ε|y, θ) with a non-parametric kernelapproximation.

[Ratmann, Andrieu, Wiuf and Richardson, 2009, PNAS]

Page 78: ABC in Varanasi

ABCµ details

Multidimensional distances ρk (k = 1, . . . ,K ) and errorsεk = ρk(ηk(z),ηk(y)), with

εk ∼ ξk(ε|y, θ) ≈ ξk(ε|y, θ) =1

Bhk

∑b

K [{εk−ρk(ηk(zb),ηk(y))}/hk ]

then used in replacing ξ(ε|y, θ) with mink ξk(ε|y, θ)ABCµ involves acceptance probability

π(θ ′, ε ′)

π(θ, ε)

q(θ ′, θ)q(ε ′, ε)

q(θ, θ ′)q(ε, ε ′)

mink ξk(ε′|y, θ ′)

mink ξk(ε|y, θ)

Page 79: ABC in Varanasi

ABCµ details

Multidimensional distances ρk (k = 1, . . . ,K ) and errorsεk = ρk(ηk(z),ηk(y)), with

εk ∼ ξk(ε|y, θ) ≈ ξk(ε|y, θ) =1

Bhk

∑b

K [{εk−ρk(ηk(zb),ηk(y))}/hk ]

then used in replacing ξ(ε|y, θ) with mink ξk(ε|y, θ)ABCµ involves acceptance probability

π(θ ′, ε ′)

π(θ, ε)

q(θ ′, θ)q(ε ′, ε)

q(θ, θ ′)q(ε, ε ′)

mink ξk(ε′|y, θ ′)

mink ξk(ε|y, θ)

Page 80: ABC in Varanasi

Wilkinson’s exact BC (not exactly!)

ABC approximation error (i.e. non-zero tolerance) replaced withexact simulation from a controlled approximation to the target,convolution of true posterior with kernel function

πε(θ, z|y) =π(θ)f (z|θ)Kε(y− z)∫π(θ)f (z|θ)Kε(y− z)dzdθ

,

with Kε kernel parameterised by bandwidth ε.[Wilkinson, 2008]

Theorem

The ABC algorithm based on the assumption of a randomisedobservation y = y+ ξ, ξ ∼ Kε, and an acceptance probability of

Kε(y− z)/M

gives draws from the posterior distribution π(θ|y).

Page 81: ABC in Varanasi

Wilkinson’s exact BC (not exactly!)

ABC approximation error (i.e. non-zero tolerance) replaced withexact simulation from a controlled approximation to the target,convolution of true posterior with kernel function

πε(θ, z|y) =π(θ)f (z|θ)Kε(y− z)∫π(θ)f (z|θ)Kε(y− z)dzdθ

,

with Kε kernel parameterised by bandwidth ε.[Wilkinson, 2008]

Theorem

The ABC algorithm based on the assumption of a randomisedobservation y = y+ ξ, ξ ∼ Kε, and an acceptance probability of

Kε(y− z)/M

gives draws from the posterior distribution π(θ|y).

Page 82: ABC in Varanasi

How exact a BC?

Pros

I Pseudo-data from true model and observed data from noisymodel

I Interesting perspective in that outcome is completelycontrolled

I Link with ABCµ and assuming y is observed with ameasurement error with density Kε

I Relates to the theory of model approximation[Kennedy & O’Hagan, 2001]

Cons

I Requires Kε to be bounded by M

I True approximation error never assessed

Page 83: ABC in Varanasi

How exact a BC?

Pros

I Pseudo-data from true model and observed data from noisymodel

I Interesting perspective in that outcome is completelycontrolled

I Link with ABCµ and assuming y is observed with ameasurement error with density Kε

I Relates to the theory of model approximation[Kennedy & O’Hagan, 2001]

Cons

I Requires Kε to be bounded by M

I True approximation error never assessed

Page 84: ABC in Varanasi

Noisy ABC

Specific case of a hidden Markov model

Xt+1 ∼ Qθ(Xt , ·)Yt+1 ∼ gθ(·|xt)

where only y01:n is observed.

[Dean, Singh, Jasra, & Peters, 2011]

Use of specific constraints, adapted to the Markov structure:{y1 ∈ B(y01 , ε)

}× · · · ×

{yn ∈ B(y0n , ε)

}

Page 85: ABC in Varanasi

Noisy ABC

Specific case of a hidden Markov model

Xt+1 ∼ Qθ(Xt , ·)Yt+1 ∼ gθ(·|xt)

where only y01:n is observed.

[Dean, Singh, Jasra, & Peters, 2011]

Use of specific constraints, adapted to the Markov structure:{y1 ∈ B(y01 , ε)

}× · · · ×

{yn ∈ B(y0n , ε)

}

Page 86: ABC in Varanasi

Noisy ABC-MLE

Idea: Modify instead the data from the start

(y01 + εζ1, . . . , yn + εζn)

[ see Fearnhead-Prangle ] andnoisy ABC-MLE estimate

arg maxθ

Pθ(Y1 ∈ B(y01 + εζ1, ε), . . . ,Yn ∈ B(y0n + εζn, ε)

)[Dean, Singh, Jasra, & Peters, 2011]

Page 87: ABC in Varanasi

Consistent noisy ABC-MLE

I Degrading the data improves the estimation performances:I Noisy ABC-MLE is asymptotically (in n) consistentI under further assumptions, the noisy ABC-MLE is

asymptotically normalI increase in variance of order ε−2

I likely degradation in precision or computing time due to thelack of summary statistic [curse of dimensionality]

Page 88: ABC in Varanasi

Which summary?

Fundamental difficulty of the choice of the summary statistic whenthere is no non-trivial sufficient statistics

I Loss of statistical information balanced against gain in dataroughening

I Approximation error remains unknown

I Choice of statistics induces choice of distance functiontowards standardisation

Page 89: ABC in Varanasi

Which summary?

Fundamental difficulty of the choice of the summary statistic whenthere is no non-trivial sufficient statistics

I Loss of statistical information balanced against gain in dataroughening

I Approximation error remains unknown

I Choice of statistics induces choice of distance functiontowards standardisation

Page 90: ABC in Varanasi

Which summary for model choice?

Depending on the choice of η(·), the Bayes factor based on thisinsufficient statistic,

Bη12(y) =

∫π1(θ1)f

η1 (η(y)|θ1) dθ1∫

π2(θ2)fη2 (η(y)|θ2) dθ2

,

is consistent or not.[X, Cornuet, Marin, & Pillai, 2012]

Consistency only depends on the range of Ei [η(y)] under bothmodels.

[Marin, Pillai, X, & Rousseau, 2012]

Page 91: ABC in Varanasi

Which summary for model choice?

Depending on the choice of η(·), the Bayes factor based on thisinsufficient statistic,

Bη12(y) =

∫π1(θ1)f

η1 (η(y)|θ1) dθ1∫

π2(θ2)fη2 (η(y)|θ2) dθ2

,

is consistent or not.[X, Cornuet, Marin, & Pillai, 2012]

Consistency only depends on the range of Ei [η(y)] under bothmodels.

[Marin, Pillai, X, & Rousseau, 2012]

Page 92: ABC in Varanasi

Semi-automatic ABC

Fearnhead and Prangle (2012) study ABC and the selection of thesummary statistic in close proximity to Wilkinson’s proposal

I ABC considered as inferential method and calibrated as such

I randomised (or ‘noisy’) version of the summary statistics

η(y) = η(y) + τε

I derivation of a well-calibrated version of ABC, i.e. analgorithm that gives proper predictions for the distributionassociated with this randomised summary statistic

Page 93: ABC in Varanasi

Summary [of F&P/statistics)

I optimality of the posterior expectation

E[θ|y]

of the parameter of interest as summary statistics η(y)![requires iterative process]

I use of the standard quadratic loss function

(θ− θ0)TA(θ− θ0)

I recent extension to model choice, optimality of Bayes factor

B12(y)

[F&P, ISBA 2012, Kyoto]

Page 94: ABC in Varanasi

Summary [of F&P/statistics)

I optimality of the posterior expectation

E[θ|y]

of the parameter of interest as summary statistics η(y)![requires iterative process]

I use of the standard quadratic loss function

(θ− θ0)TA(θ− θ0)

I recent extension to model choice, optimality of Bayes factor

B12(y)

[F&P, ISBA 2012, Kyoto]

Page 95: ABC in Varanasi

ummary [about summaries]

I Choice of summary statistics is paramount for ABCvalidation/performances

I At best, ABC approximates π(. | η(y))

I Model selection feasible with ABC [with caution!]

I For estimation, consistency if {θ;µ(θ) = µ0} = θ0 whenEθ[η(y)] = µ(θ)

I For testing consistency if{µ1(θ1), θ1 ∈ Θ1} ∩ {µ2(θ2), θ2 ∈ Θ2} = ∅

[Marin et al., 2011]

Page 96: ABC in Varanasi

ummary [about summaries]

I Choice of summary statistics is paramount for ABCvalidation/performances

I At best, ABC approximates π(. | η(y))

I Model selection feasible with ABC [with caution!]

I For estimation, consistency if {θ;µ(θ) = µ0} = θ0 whenEθ[η(y)] = µ(θ)

I For testing consistency if{µ1(θ1), θ1 ∈ Θ1} ∩ {µ2(θ2), θ2 ∈ Θ2} = ∅

[Marin et al., 2011]

Page 97: ABC in Varanasi

ummary [about summaries]

I Choice of summary statistics is paramount for ABCvalidation/performances

I At best, ABC approximates π(. | η(y))

I Model selection feasible with ABC [with caution!]

I For estimation, consistency if {θ;µ(θ) = µ0} = θ0 whenEθ[η(y)] = µ(θ)

I For testing consistency if{µ1(θ1), θ1 ∈ Θ1} ∩ {µ2(θ2), θ2 ∈ Θ2} = ∅

[Marin et al., 2011]

Page 98: ABC in Varanasi

ummary [about summaries]

I Choice of summary statistics is paramount for ABCvalidation/performances

I At best, ABC approximates π(. | η(y))

I Model selection feasible with ABC [with caution!]

I For estimation, consistency if {θ;µ(θ) = µ0} = θ0 whenEθ[η(y)] = µ(θ)

I For testing consistency if{µ1(θ1), θ1 ∈ Θ1} ∩ {µ2(θ2), θ2 ∈ Θ2} = ∅

[Marin et al., 2011]

Page 99: ABC in Varanasi

ummary [about summaries]

I Choice of summary statistics is paramount for ABCvalidation/performances

I At best, ABC approximates π(. | η(y))

I Model selection feasible with ABC [with caution!]

I For estimation, consistency if {θ;µ(θ) = µ0} = θ0 whenEθ[η(y)] = µ(θ)

I For testing consistency if{µ1(θ1), θ1 ∈ Θ1} ∩ {µ2(θ2), θ2 ∈ Θ2} = ∅

[Marin et al., 2011]

Page 100: ABC in Varanasi

Empirical likelihood (EL)

Unavailable likelihoods

ABC methods

ABC as an inference machine

ABCel

ABC and ELComposite likelihoodIllustrations

Page 101: ABC in Varanasi

Empirical likelihood (EL)

Dataset x made of n independent replicates x = (x1, . . . , xn) ofsome X ∼ F

Generalized moment condition model

EF

[h(X ,φ)

]= 0,

where h is a known function, and φ an unknown parameter

Corresponding empirical likelihood

Lel(φ|x) = maxp

n∏i=1

pi

for all p such that 0 6 pi 6 1,∑

i pi = 1,∑

i pih(xi ,φ) = 0.

[Owen, 1988, Bio’ka, & Empirical Likelihood, 2001]

Page 102: ABC in Varanasi

Empirical likelihood (EL)

Dataset x made of n independent replicates x = (x1, . . . , xn) ofsome X ∼ F

Generalized moment condition model

EF

[h(X ,φ)

]= 0,

where h is a known function, and φ an unknown parameter

Corresponding empirical likelihood

Lel(φ|x) = maxp

n∏i=1

pi

for all p such that 0 6 pi 6 1,∑

i pi = 1,∑

i pih(xi ,φ) = 0.

[Owen, 1988, Bio’ka, & Empirical Likelihood, 2001]

Page 103: ABC in Varanasi

Convergence of EL [3.4]

Theorem 3.4 Let X ,Y1, . . . ,Yn be independent rv’s with commondistribution F0. For θ ∈ Θ, and the function h(X , θ) ∈ Rs , letθ0 ∈ Θ be such that

Var(h(Yi , θ0))

is finite and has rank q > 0. If θ0 satisfies

E(h(X , θ0)) = 0,

then

−2 log

(Lel(θ0|Y1, . . . ,Yn)

n−n

)→ χ2(q)

in distribution when n→∞.[Owen, 2001]

Page 104: ABC in Varanasi

Convergence of EL [3.4]

“...The interesting thing about Theorem 3.4 is what is not there. Itincludes no conditions to make θ a good estimate of θ0, nor evenconditions to ensure a unique value for θ0, nor even that any solution θ0exists. Theorem 3.4 applies in the just determined, over-determined, andunder-determined cases. When we can prove that our estimatingequations uniquely define θ0, and provide a consistent estimator θ of it,then confidence regions and tests follow almost automatically throughTheorem 3.4.”.

[Owen, 2001]

Page 105: ABC in Varanasi

Raw ABCelsampler

Act as if EL was an exact likelihood[Lazar, 2003]

for i = 1→ N do

generate φi from the prior distribution π(·)set the weight ωi = Lel(φi |xobs)

end for

return (φi ,ωi ), i = 1, . . . ,N

I Output weighted sample of size N

Page 106: ABC in Varanasi

Raw ABCelsampler

Act as if EL was an exact likelihood[Lazar, 2003]

for i = 1→ N do

generate φi from the prior distribution π(·)set the weight ωi = Lel(φi |xobs)

end for

return (φi ,ωi ), i = 1, . . . ,N

I Performance evaluated through effective sample size

ESS = 1/ N∑

i=1

ωi

/ N∑j=1

ωj

2

Page 107: ABC in Varanasi

Raw ABCelsampler

Act as if EL was an exact likelihood[Lazar, 2003]

for i = 1→ N do

generate φi from the prior distribution π(·)set the weight ωi = Lel(φi |xobs)

end for

return (φi ,ωi ), i = 1, . . . ,N

I More advanced algorithms can be adapted to EL:E.g., adaptive multiple importance sampling (AMIS) ofCornuet et al. to speed up computations

[Cornuet et al., 2012]

Page 108: ABC in Varanasi

Moment condition in population genetics?

EL does not require a fully defined and often complex (hencedebatable) parametric model

Main difficulty

Derive a constraintEF

[h(X ,φ)

]= 0,

on the parameters of interest φ when X is made of the genotypesof the sample of individuals at a given locus

E.g., in phylogeography, φ is composed of

I dates of divergence between populations,

I ratio of population sizes,

I mutation rates, etc.

None of them are moments of the distribution of the allelic statesof the sample

Page 109: ABC in Varanasi

Moment condition in population genetics?

EL does not require a fully defined and often complex (hencedebatable) parametric model

Main difficulty

Derive a constraintEF

[h(X ,φ)

]= 0,

on the parameters of interest φ when X is made of the genotypesof the sample of individuals at a given locus

c© h made of pairwise composite scores (whose zero is the pairwisemaximum likelihood estimator)

Page 110: ABC in Varanasi

Pairwise composite likelihood

The intra-locus pairwise likelihood

`2(xk|φ) =∏i<j

`2(xik , x

jk |φ)

with x1k , . . . , xnk : allelic states of the gene sample at the k-th locus

The pairwise score function

∇φ log `2(xk|φ) =∑i<j

∇φ log `2(xik , x

jk |φ)

� Composite likelihoods are often much narrower than theoriginal likelihood of the model

Safe with EL because we only use position of its mode

Page 111: ABC in Varanasi

Pairwise likelihood: a simple case

Assumptions

I sample ⊂ closed, panmicticpopulation at equilibrium

I marker: microsatellite

I mutation rate: θ/2

if x ik et x jk are two genes of thesample,

`2(xik , x

jk |θ) depends only on

δ = x ik − x jk

`2(δ|θ) =1√

1 + 2θρ (θ)|δ|

with

ρ(θ) =θ

1 + θ+√

1 + 2θ

Pairwise score function

∂θ log `2(δ|θ) =

−1

1 + 2θ+

|δ|

θ√

1 + 2θ

Page 112: ABC in Varanasi

Pairwise likelihood: a simple case

Assumptions

I sample ⊂ closed, panmicticpopulation at equilibrium

I marker: microsatellite

I mutation rate: θ/2

if x ik et x jk are two genes of thesample,

`2(xik , x

jk |θ) depends only on

δ = x ik − x jk

`2(δ|θ) =1√

1 + 2θρ (θ)|δ|

with

ρ(θ) =θ

1 + θ+√

1 + 2θ

Pairwise score function

∂θ log `2(δ|θ) =

−1

1 + 2θ+

|δ|

θ√

1 + 2θ

Page 113: ABC in Varanasi

Pairwise likelihood: a simple case

Assumptions

I sample ⊂ closed, panmicticpopulation at equilibrium

I marker: microsatellite

I mutation rate: θ/2

if x ik et x jk are two genes of thesample,

`2(xik , x

jk |θ) depends only on

δ = x ik − x jk

`2(δ|θ) =1√

1 + 2θρ (θ)|δ|

with

ρ(θ) =θ

1 + θ+√

1 + 2θ

Pairwise score function

∂θ log `2(δ|θ) =

−1

1 + 2θ+

|δ|

θ√

1 + 2θ

Page 114: ABC in Varanasi

Pairwise likelihood: 2 diverging populations

MRCA

POP a POP b

τ

Assumptions

I τ: divergence date ofpop. a and b

I θ/2: mutation rate

Let x ik and x jk be two genescoming resp. from pop. a andbSet δ = x ik − x jk .

Then `2(δ|θ, τ) =

e−τθ√1 + 2θ

+∞∑k=−∞ ρ(θ)

|k |Iδ−k(τθ).

whereIn(z) nth-order modifiedBessel function of the firstkind

Page 115: ABC in Varanasi

Pairwise likelihood: 2 diverging populations

MRCA

POP a POP b

τ

Assumptions

I τ: divergence date ofpop. a and b

I θ/2: mutation rate

Let x ik and x jk be two genescoming resp. from pop. a andbSet δ = x ik − x jk .

A 2-dim score function∂τ log `2(δ|θ, τ) = −θ+θ

2

`2(δ− 1|θ, τ) + `2(δ+ 1|θ, τ)

`2(δ|θ, τ)

∂θ log `2(δ|θ, τ) =

−τ−1

1+ 2θ+

q(δ|θ, τ)

`2(δ|θ, τ)+

τ

2

`2(δ− 1|θ, τ) + `2(δ+ 1|θ, τ)

`2(δ|θ, τ)

where

q(δ|θ, τ) :=

e−τθ

√1+ 2θ

ρ ′(θ)

ρ(θ)

∞∑k=−∞ |k |ρ(θ)|k|Iδ−k(τθ)

Page 116: ABC in Varanasi

Example: normal posterior

ABCel with two constraintsESS=155.6

θ

Den

sity

−0.5 0.0 0.5 1.0

0.0

1.0

ESS=75.93

θD

ensi

ty

−0.4 −0.2 0.0 0.2 0.4

0.0

1.0

2.0

ESS=76.87

θ

Den

sity

−0.4 −0.2 0.0 0.2

01

23

4

ESS=91.54

θ

Den

sity

−0.6 −0.4 −0.2 0.0 0.2

01

23

4

ESS=108.4

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.0

2.0

3.0

ESS=85.13

θ

Den

sity

−0.2 0.0 0.2 0.4 0.6

0.0

1.0

2.0

3.0

ESS=149.1

θ

Den

sity

−0.5 0.0 0.5 1.0

0.0

1.0

2.0

ESS=96.31

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.0

2.0

ESS=83.77

θ

Den

sity

−0.6 −0.4 −0.2 0.0 0.2 0.40

12

34

ESS=155.7

θ

Den

sity

−0.5 0.0 0.5

0.0

1.0

2.0

ESS=92.42

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.0

2.0

3.0 ESS=95.01

θ

Den

sity

−0.4 0.0 0.2 0.4 0.6

0.0

1.5

3.0

ESS=139.2

Den

sity

−0.6 −0.2 0.2 0.6

0.0

1.0

2.0

ESS=99.33

Den

sity

−0.4 −0.2 0.0 0.2 0.4

0.0

1.0

2.0

3.0

ESS=87.28

Den

sity

−0.2 0.0 0.2 0.4 0.6

01

23

Sample sizes are of 25 (column 1), 50 (column 2) and 75 (column 3)

observations

Page 117: ABC in Varanasi

Example: normal posterior

ABCel with three constraintsESS=300.1

θ

Den

sity

−0.4 0.0 0.4 0.8

0.0

1.0

ESS=205.5

θD

ensi

ty

−0.6 −0.2 0.0 0.2 0.4

0.0

1.0

2.0

ESS=179.4

θ

Den

sity

−0.2 0.0 0.2 0.4

0.0

1.5

3.0

ESS=265.1

θ

Den

sity

−0.3 −0.2 −0.1 0.0 0.1

01

23

4

ESS=250.3

θ

Den

sity

−0.6 −0.4 −0.2 0.0 0.2

0.0

1.0

2.0

ESS=134.8

θ

Den

sity

−0.4 −0.2 0.0 0.1

01

23

4

ESS=331.5

θ

Den

sity

−0.8 −0.4 0.0 0.4

0.0

1.0

2.0 ESS=167.4

θ

Den

sity

−0.9 −0.7 −0.5 −0.3

01

23

ESS=136.5

θ

Den

sity

−0.4 −0.2 0.0 0.20

12

34

ESS=322.4

θ

Den

sity

−0.2 0.0 0.2 0.4 0.6 0.8

0.0

1.0

2.0

ESS=202.7

θ

Den

sity

−0.4 −0.2 0.0 0.2 0.4

0.0

1.0

2.0

3.0 ESS=166

θ

Den

sity

−0.4 −0.2 0.0 0.2

01

23

4

ESS=263.7

Den

sity

−1.0 −0.6 −0.2

0.0

1.0

2.0

ESS=190.9

Den

sity

−0.4 −0.2 0.0 0.2 0.4 0.6

01

23

ESS=165.3

Den

sity

−0.5 −0.3 −0.1 0.1

0.0

1.5

3.0

Sample sizes are of 25 (column 1), 50 (column 2) and 75 (column 3)

observations

Page 118: ABC in Varanasi

Example: Superposition of gamma processes

Example of superposition of N renewal processes with waitingtimes τij (i = 1, . . . ,M, j = 1, . . .) ∼ G(α,β), when N is unknown.Renewal processes

ζi1 = τi1, ζi2 = ζi1 + τi2, . . .

with observations made of first n values of the ζij ’s,

z1 = min{ζij }, z2 = min{ζij ; ζij > z1}, . . .

ending withzn = min{ζij ; ζij > zn−1} .

[Cox & Kartsonaki, B’ka, 2012]

Page 119: ABC in Varanasi

Example: Superposition of gamma processes (ABC)

Interesting testing ground for ABCel since data (zt) neither iid norMarkov

Recovery of an iid structure by

1. simulating a pseudo-dataset,(z?1 , . . . , z

?n ), as in regular

ABC,

2. deriving sequence ofindicators (ν1, . . . ,νn), as

z?1 = ζν11, z?2 = ζν2j2 , . . .

3. exploiting that thoseindicators are distributedfrom the prior distributionon the νt ’s leading to an iidsample of G(α,β) variables

Comparison of ABC andABCel posteriors

α

Den

sity

0 1 2 3 4

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

β

Den

sity

0 1 2 3 4

0.0

0.5

1.0

1.5

N

Den

sity

0 5 10 15 20

0.00

0.02

0.04

0.06

0.08

αD

ensi

ty

0 1 2 3 4

0.0

0.5

1.0

1.5

β

Den

sity

0 1 2 3 4

0.0

0.2

0.4

0.6

0.8

1.0

N

Den

sity

0 5 10 15 20

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Top: ABCel

Bottom: regular ABC

Page 120: ABC in Varanasi

Example: Superposition of gamma processes (ABC)

Interesting testing ground for ABCel since data (zt) neither iid norMarkov

Recovery of an iid structure by

1. simulating a pseudo-dataset,(z?1 , . . . , z

?n ), as in regular

ABC,

2. deriving sequence ofindicators (ν1, . . . ,νn), as

z?1 = ζν11, z?2 = ζν2j2 , . . .

3. exploiting that thoseindicators are distributedfrom the prior distributionon the νt ’s leading to an iidsample of G(α,β) variables

Comparison of ABC andABCel posteriors

α

Den

sity

0 1 2 3 4

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

β

Den

sity

0 1 2 3 4

0.0

0.5

1.0

1.5

N

Den

sity

0 5 10 15 20

0.00

0.02

0.04

0.06

0.08

αD

ensi

ty

0 1 2 3 4

0.0

0.5

1.0

1.5

β

Den

sity

0 1 2 3 4

0.0

0.2

0.4

0.6

0.8

1.0

N

Den

sity

0 5 10 15 20

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Top: ABCel

Bottom: regular ABC

Page 121: ABC in Varanasi

Pop’gen’: A first experiment

Evolutionary scenario:MRCA

POP 0 POP 1

τ

Dataset:

I 50 genes per populations,

I 100 microsat. loci

Assumptions:

I Ne identical over allpopulations

I φ = (log10 θ, log10 τ)

I uniform prior over(−1., 1.5)× (−1., 1.)

Comparison of the originalABC with ABCel

ESS=7034

log(theta)

Den

sity

0.00 0.05 0.10 0.15 0.20 0.25

05

1015

log(tau1)

Den

sity

−0.3 −0.2 −0.1 0.0 0.1 0.2 0.3

01

23

45

67

histogram = ABCel

curve = original ABCvertical line = “true”parameter

Page 122: ABC in Varanasi

Pop’gen’: A first experiment

Evolutionary scenario:MRCA

POP 0 POP 1

τ

Dataset:

I 50 genes per populations,

I 100 microsat. loci

Assumptions:

I Ne identical over allpopulations

I φ = (log10 θ, log10 τ)

I uniform prior over(−1., 1.5)× (−1., 1.)

Comparison of the originalABC with ABCel

ESS=7034

log(theta)

Den

sity

0.00 0.05 0.10 0.15 0.20 0.25

05

1015

log(tau1)

Den

sity

−0.3 −0.2 −0.1 0.0 0.1 0.2 0.3

01

23

45

67

histogram = ABCel

curve = original ABCvertical line = “true”parameter

Page 123: ABC in Varanasi

ABC vs. ABCel on 100 replicates of the 1st experiment

Accuracy:log10 θ log10 τ

ABC ABCel ABC ABCel

(1) 0.097 0.094 0.315 0.117

(2) 0.071 0.059 0.272 0.077

(3) 0.68 0.81 1.0 0.80

(1) Root Mean Square Error of the posterior mean

(2) Median Absolute Deviation of the posterior median

(3) Coverage of the credibility interval of probability 0.8

Computation time: on a recent 6-core computer(C++/OpenMP)

I ABC ≈ 4 hours

I ABCel≈ 2 minutes

Page 124: ABC in Varanasi

Pop’gen’: Second experiment

Evolutionary scenario:MRCA

POP 0 POP 1 POP 2

τ1

τ2

Dataset:

I 50 genes per populations,

I 100 microsat. loci

Assumptions:

I Ne identical over allpopulations

I φ =(log10 θ, log10 τ1, log10 τ2)

I non-informative uniformprior

Comparison of the original ABCwith ABCel

histogram = ABCel

curve = original ABCvertical line = “true” parameter

Page 125: ABC in Varanasi

Pop’gen’: Second experiment

Evolutionary scenario:MRCA

POP 0 POP 1 POP 2

τ1

τ2

Dataset:

I 50 genes per populations,

I 100 microsat. loci

Assumptions:

I Ne identical over allpopulations

I φ =(log10 θ, log10 τ1, log10 τ2)

I non-informative uniformprior

Comparison of the original ABCwith ABCel

histogram = ABCel

curve = original ABCvertical line = “true” parameter

Page 126: ABC in Varanasi

Pop’gen’: Second experiment

Evolutionary scenario:MRCA

POP 0 POP 1 POP 2

τ1

τ2

Dataset:

I 50 genes per populations,

I 100 microsat. loci

Assumptions:

I Ne identical over allpopulations

I φ =(log10 θ, log10 τ1, log10 τ2)

I non-informative uniformprior

Comparison of the original ABCwith ABCel

histogram = ABCel

curve = original ABCvertical line = “true” parameter

Page 127: ABC in Varanasi

Pop’gen’: Second experiment

Evolutionary scenario:MRCA

POP 0 POP 1 POP 2

τ1

τ2

Dataset:

I 50 genes per populations,

I 100 microsat. loci

Assumptions:

I Ne identical over allpopulations

I φ =(log10 θ, log10 τ1, log10 τ2)

I non-informative uniformprior

Comparison of the original ABCwith ABCel

histogram = ABCel

curve = original ABCvertical line = “true” parameter

Page 128: ABC in Varanasi

ABC vs. ABCel on 100 replicates of the 2nd experiment

Accuracy:log10 θ log10 τ1 log10 τ2

ABC ABCel ABC ABCel ABC ABCel

(1) .0059 .0794 .472 .483 29.3 4.76

(2) .048 .053 .32 .28 4.13 3.36

(3) .79 .76 .88 .76 .89 .79

(1) Root Mean Square Error of the posterior mean

(2) Median Absolute Deviation of the posterior median

(3) Coverage of the credibility interval of probability 0.8

Computation time: on a recent 6-core computer(C++/OpenMP)

I ABC ≈ 6 hours

I ABCel≈ 8 minutes

Page 129: ABC in Varanasi

Comparison

On large datasets, ABCel gives more accurate results than ABC

ABC simplifies the dataset through summary statisticsDue to the large dimension of x , the original ABC algorithmestimates

π(θ∣∣∣η(xobs)),

where η(xobs) is some (non-linear) projection of the observeddataset on a space with smaller dimension↪→ Some information is lost

ABCel simplifies the model through a generalized momentcondition model.↪→ Here, the moment condition model is based on pairwisecomposition likelihood