Assessing the Adequacy of Epidemic Models Using Hybrid Approaches Gavin J Gibson & George Streftaris Maxwell Institute for Mathematical Sciences Heriot-Watt

Assessing the Adequacy of Epidemic Models Using Hybrid Approaches

Gavin J Gibson & George StreftarisMaxwell Institute for Mathematical Sciences

Heriot-Watt University

Collaborators: Alex Cook, Chris Gilligan, Tim Gottwald, Glenn Marion, Mark Woolhouse, Joao Filipe,..

Funding: BBSRC, USDA

InFER2011 (Inference For Epidemic-related Risk) 28th March - 1st April 2011

Outline

• Modern algorithms allow computation of solutions to complex problems in inference

• Understanding and interpreting solutions is not always so easy!

• True for fitting/testing/comparing epidemic models

• Hybrid approaches & the Freudian Metaphor

• Examples and alternatives

• Questions and challenges

Bayesian inference for epidemics Experiment: yields partial data y. Stochastic model: with parameter specifying (y|). Aim: Express belief re as a probability density (|y).

Bayesian solution: Assign prior distribution (), to yield posterior (|y) ()(y|).

Problem: (y|) is often intractable integral.

Data augmentation: Consider data x from ‘richer’ experiment (i.e. y = f(x)) for which (x|) is tractable. Consider (, x | y) ()(x |)(y|x). Often straightforward to simulate from (, x |y) e.g. using MCMC.

Basic SEIR model

S E: If j is in state S at time t, then Pr(j is exposed in (t, t+dt)) = I(t)dt

E I: (random time in E)I R: (random time in I)

~E

j EET

~I

j IIT

Parameters: = (, E, I)

If times of transitions are observed (x), then likelihood (x | ) tractable.

Data y usually heavily censored/filtered (e.g. only removals are observed, weekly totals of new infections)

Fitting using McMC

(, x |y) ()(x, y|)

•Construct Markov chain with stationary distribution (, x |y)

•Iterate by proposing & accepting/rejecting changes to the current state (i, xi) to obtain (i+1, xi+1).

Updates to can often be carried out by Gibbs steps.

Updates to x, usually require Metropolis-Hastings and Reversible-Jump type approaches.

Iterate chain to produce sample from (, x | y). (See e.g. GJG + ER, 1998,O’Neill & Roberts, 1999, Streftaris & GJG, 2004, Forrester et al, 2007, Gibson et al., 2006, Chis-Ster et al. 2008, Starr et al. 2009,…)

Sensitivity to prior

Removals from smallpox epidemic (Bailey)

Markovian SEIR model: : contact rate,: removal rate, : E → I rate

1000 samples from posterior using uniform prior over cuboid.

Similar difficulties arise e.g. when considering infection processes incorporating primary (a) and secondary infection (b) rates. (Gibson & Renshaw, 2001)

Extension to spatio-temporal SI models

Susceptible j acquires infection at rate:

Rj = f(t; )( + i K(dij, ))

Spatial kernels (examples):

1. K(d, ) = exp(-d)

2. K(d, ) = exp(-d2)

3. K(d, ) = (d+1)-

j

Can be fitted using standard Bayesian/data augmentation/MCMC approach

See GJG (1997), Jamieson (2004), Cook et al (2008) , Chis-Ster & Ferguson (2009)

If models are used to design control strategy – e.g. spatial eradication programmes – then model choice can be crucial.

Example: Miami Citrus Canker epidemic (Gottwald et al., Phytopathology, 2002; Jamieson, PhD Thesis, U. Cambridge, 2004, Cook et al. 2008)

Data: Dade county, Miami

Optimal strategy for eradication sensitive to model choice

6056 susceptibles, 1124 infections after 12 30-day periods

Control strategies can be controversial

Classical-Bayesian Spectrum‘Classical’ model: fixed, model specifies (x |) , where x represents quantities varying between replicate experiments. Predicts frequencies for x given .

‘Bayesian’ model: Uncertainty in modelled as prior () giving (, x) ()(x | ). A framework for both prediction of x and learning about .

How ‘large’ should our space of possible be?

Very large - less need to benchmark against alternatives – but problems of prior representation and sensitivity, computational complexity

Very small – greater need to assess adequacy – sensitivity and complexity of inference reduced

Hybrid approaches to model adequacy

Example: Posterior predictive p-values (e.g. Rubin, 1984, XL Meng, 1994).

To test H0: the model is valid. Observe y, calculate teststatistic T(y), then consider

dTTPp )|)((| yy

Interpretation:

•Posterior probability of more extreme value of T in next experiment.•Posterior expectation of the classical p-value P(T > T(y); ), computed by classical statistician with knowledge of .

*in Handbook for data analysis in the behavioral sciences

EGO: Reason, common sense, translates the appetites if ID into action

SUPEREGO: Conscience, criticism of EGO

ID: Basic Instincts & Drives

The Freudian Metaphor (see e.g. Gigerenzer*, 1993) Co-existence of multiple facets of the statistical personality.






Gigerenzer

Neyman-Pearson

Fisher

Bayesians





Gigerenzer

Neyman-Pearson

Fisher

Bayesians

GJG

Classical

Bayesian






Gigerenzer

Neyman-Pearson

Fisher

Bayesians

GJG

Classical

Bayesian

Physicists?


Observes:

Imputes:

y

S

( |y)

p(T(y), ))

(p |y)

E Asserts model and ()

0 1

Schematic diagram of PP p-value

Large probability of small p-value indicates conflict between E and S

WORLD OF THE EGO

Suppose S applies Likelihood Ratio Test, for example

• T(y, ) = (y | )/1(y) where 1(y) denotes sampling density of y under an alternative model. Problem of intractable likelihood (y | ) arises again!

• Impute S’s response to observation of latent process x?

• So long as both and 1 specify a tractable sampling density for x, then (x|)/1(x) can be imputed (e.g. from MCMC).

PP p-value for model comparison

Observes:

Imputes:

y

x S

(x |y)

p(T(x, ))

(p |y)


0 1

Imputed p-value from a latent process


R solani in radish (GJG, et al., 2006)

18 x 23 grids of plants, daily sampling (approx):High inoculum: 45 randomly chosen sites (13 reps)Low inoculum: 15 randomly chosen sites (13 reps)

•Model: SI with primary infection (a), nearest-neighbour secondary infection (b0, b1, b2) representing max rate, variability and peak timing.

•Fitted using MCMC methods

Results: Replicates fitted jointly (assuming common parameters) and separately.

Model checking with imputed p-values

Symptomatic day 9 X

Missing O, Primary inoc. +

Symptomatic day 9 X

Sample of data (high inoculum)

High inoculum

Low inoculum

Posterior densities (primary infection rate, a)

High inoculum

Low inoculum

Posterior densities (peak secondary rate, b0)

Posterior densities (secondary variability, b1)

High inoculum

Low inoculum

Posterior densities (secondary peak time, b2)

High inoculum

Low inoculum

Posterior predictive envelope of I(t) (joint fit posterior mean parameters)

High

Low

Checking using ‘Sellke’ residuals

If Ri(t) denotes infectious challenge to i at time t,

where xi denotes infection time.

Impute latent ‘Sellke’ thresholds for each site and S’s p-value from K-S test, generating posterior distribution of p-values.

dttRix

ii 0

~ Exp(1)

E

(, x, ) S p()

()

Observation y

(p|y)

mean median LQ UQ Pr(p<5%) rep 0.013454 0.008933 0.003938 0.017715 0.9706 H1 0.000024 0.000006 0.000002 0.000020 1.0000 H2 0.000048 0.000015 0.000005 0.000046 1.0000 H3 0.002916 0.002039 0.001029 0.003757 1.0000 H4 0.136859 0.121301 0.070754 0.189353 0.1508 H5 0.075692 0.061481 0.033481 0.101138 0.4054 H6 0.014030 0.010434 0.005457 0.018893 0.9794 H7 0.000000 0.000000 0.000000 0.000000 1.0000 H8 0.000024 0.000009 0.000003 0.000026 1.0000 H9 0.552893 0.540598 0.388507 0.707525 0.0014 H10 0.000002 0.000000 0.000000 0.000002 0.9844 H11 0.000000 0.000000 0.000000 0.000000 1.0000 H12 0.086550 0.067288 0.035668 0.116220 0.3708 H13

mean median LQ UQ Pr(p<5%) rep 0.085966 0.063765 0.030051 0.118423 0.4036 L1 0.016515 0.011087 0.004239 0.022995 0.9412 L2 0.025164 0.018007 0.008519 0.034758 0.8724 L3 0.001911 0.001254 0.000602 0.002506 1.0000 L4 0.000126 0.000034 0.000009 0.000116 1.0000 L5 0.067819 0.052805 0.024010 0.097047 0.4804 L6 0.000045 0.000017 0.000005 0.000048 1.0000 L7 0.000004 0.000000 0.000000 0.000001 1.0000 L8 0.000030 0.000006 0.000001 0.000022 1.0000 L9 0.003341 0.001897 0.000737 0.004192 0.9998 L10 0.000001 0.000000 0.000000 0.000001 1.0000 L11 0.001243 0.000678 0.000257 0.001549 1.0000 L12 0.051377 0.027275 0.008475 0.069362 0.6560 L13

High inoculum - joint fit p-val. posterior summaries

Low inoculum - joint fit p-val. posterior summaries

Posteriors for p indicate lack of fit…….

Model comparison using imputed p-values

Streftaris &Gibson (PRSB, 2004) implicitly followed this approach

•Analysed data from 2 experiments of FMD in 2 populations of sheep.•2 groups of 32 sheep each subdivided into 4 sub-groups.•Group 1 exposed to FMD, subsequent epidemic observed.

Data: Censored estimates of infectious period for each sheep, and measures of peak viraemic load for each animal.

Question: Does viraemia decline as we go down the infection tree?

Experiment (Hughes et al., J. Gen. Virol. 2002):

32 sheep allocated to 4 groups G1, .., G4. Animals in G1 inoculated with FMD virus (4 at t=0, 4 at t=1 day). Thereafter animals mix according to the following scheme:

Day 1 2 3 4 .

G1 G2 G3 G4

Idea is to ‘force’ higher groups further down the chain of infection.Data: Daily tests on each animal, summarised by:y = (time of 1st +ve test, last +ve test, peak viraemic level)

Model: SEIR-Relationship between infectivity and viraemic load-Weibull distributions for sojourn in E and I classes-Peak viraemia independent of depth-Vague priors for model parameters ().

Depth 0

Depth 2

Depth 1

Depth 4

S conducts 1-way ANOVA on peak viraemic levels, to generate p-value.

E considers (p|y) to identify potential conflict.

Let x denote the infection network – must be imputed using MCMC

x

Group 1

Group 2

Arguably, a little too strongly stated.

From Streftaris & Gibson, PRSB (2004)

If we accept the modelling assumptions we must nevertheless concede that, with high posterior probability, an ANOVA test would provide significant evidence of differences in viraemia with depth in the infection chain.

A general infection process (Streftaris & Gibson, 2011, in preparation)

Assume ‘Sellke’ thresholds drawn from unit mean Weibull with shape parameter . (NB = 1 is exponential).

Is there evidence against the exponential model in favour of this new model for Experiments 1 & 2?

A: Full Bayesian – include as additional parameter and consider (|data)

B: Latent KS-test applied to imputed thresholds for exponential model

C: ‘Latent’ LRT (against Weibull alternative) applied to imputed thresholds for exponential model

Results for Weibull threshold model

A

C

B

EXPERIMENT 1

Imputation ‘reinforces’ the model

Observes:

Imputes:

y

x S

(x |y)

p(T(x, ))

(p |y)


0 1


Power of tests applied to x should be expected to diminish with amount of imputation.

Loss of ‘power’ as the ‘richness’ of x increases

Consider simple hypotheses regarding distribution of x.

E asserts x ~ 0(x) = (x | 0). S checks against alternative 1(x)

Observe y = f(x).

E imputes x ~ 0(x|y) and result of S’s test based on 0(x)/1(x).

Should use 0(x)/(1(y)0(x|y)) = 0(y)/1(y).

If we use z instead of x, where x = f(z), the corresponding mis-match between denominators increases (as measured by K-L divergence).

Comparing models - ‘Symmetric’ Approaches

1. Bayesian Model Choice

• Embed ‘competing’ models i = 1, … , k in an expanded model space equipped with prior for models (p1, …, pk) and parameters (i), i = 1, ..., k.

• Increased complexity makes implementation of MCMC harder.

• Model posterior probabilities sensitive to choice of prior (i).

Power-law decay, a (transformed)

CTV spread by melon aphid: Model: Rj = + i dij-

Gottwald et al., 1996,GJG 1997

1 year 1 year

Posterior contour plots: Melon aphid (3 epidemics) v Brown citrus aphid (3 epidemics)

MELON APHID(B + NN)

BC APHID(Local not NN)

Gottwald, et al (1999)

n1 infections n2 infections

Local parameter a

b

Analysis of such historical data could provide informative priors for comparison of MA and BCA ‘models’ fitted to a new data set.

MA prior

BCA prior

3rd model representingunspecified alternative (characterised by vague prior) may not be favoured in Bayesian model comparison.

Leads to comparisons based on separate fitting of models.

Backgroundinfection

2. Posterior Bayes Factors / DIC

•PBF (Aitken, 1991) compare models on basis of

•DIC (Spiegelhalter et al, 2002) uses D() = -2 log (y|). Formally

where is measure of complexity and expectations are taken over (|y). DIC is then computed across the models to be compared.

22222

11111

||

||

dyy

dyy Ratio of posterior expectations of the likelihood

DpDDIC 2)~

(

~)( DDpD

DIC for epidemic modelling?

We may need to consider augmented parameter vector ′ = (, x) where x are unobserved components so that (y, x|) is tractable.

•No unique choice of x!

•Dimension of imputed x may approach (or exceed) dimension of data set y.

See Celeux at al, 2006, DIC for missing data models for extensive range of alternative ways to define DIC

•Bayesian relevance of comparing DICs across models?

~

DIC

Philosophical difficulties with DIC/PBF?

E1

1S

Observation y

(|y)

E2

2

()

(|y)

2 or more Egos required!“Batesian” rather than Bayesian?

E1

S1

E2

S2

DIC1(y) DIC2(y)

DIC1(y) DIC2(y)

Interpretation 1 Interpretation 2

()

2 or more statisticians: DIC interpreted by some external arbiter.

DIC

Philosophical difficulties with DIC/PBF?

E1

1S

Observation y

(|y)

E2

2

()

(|y)

2 or more Egos required!“Batesian” rather than Bayesian?

E1

S1

E2

S2

DIC1(y) DIC2(y)

DIC1(y) DIC2(y)

Interpretation 1 Interpretation 2

()

2 or more statisticians: DIC interpreted by some external arbiter.

12

(p|y) (p|y)

Summing up

•Many ‘tensions’ in Bayesian methods come to the fore in the context of dynamical epidemic models

•Hybrid approaches may offer a way of addressing these tensions by applying Bayesian methods to low-complexity models checked in a classical approach

•Perhaps we need to underplay the importance of models as predictive tools as opposed to interpretive tools.

•Qualitative conclusions that are robust to model choice may be seen as extremely valuable

Model: SEI with ‘quenching’, primary and secondary infection constant latent period.

3 ‘submodels’:

1. Latent period = 0 (SI)2. SEI with observations recording I3. SEI with observations recording E+I

No trichoderma

Trichoderma

Final example: R solani in radish re-visited

1. SI model 2. SEI model, I observed

PrimaryPrimary

Secondary

Secondary

Latent

Quenching

Quenching

Although quantitative estimatesof parameters changes with model the qualitative conclusion seems robust.

There is consistent evidence that T viride appears to affect the primary infection parameter.

Models are useful ‘lenses’ even if they cannot be used as ‘crystal balls’!

3. SEI model, E+I observed

Primary Secondary

Quenching Latent

Documents

Assessing the Adequacy of Epidemic Models Using Hybrid Approaches Gavin J Gibson & George Streftaris Maxwell Institute for Mathematical Sciences Heriot-Watt