computational science & engineering seminar, 16 oct 2013

De�nitions Statistics Computation Software Conclusions References

Statistics, computation, and software engineering:development and maintenance of mixed modeling

software in R

Ben Bolker

McMaster University, Mathematics & Statistics and Biology

15 October 2013

Ben Bolker

Mixed model software


Outline

1 De�nitions and context

2 Statistical challenges

3 Computational challenges

4 Software engineering

5 Conclusions

Ben Bolker



Outline





5 Conclusions

Ben Bolker



(Generalized) linear mixed models

(G)LMMs: a statistical modeling framework incorporating:

Linear combinations of categorical and continuouspredictors, and interactions

Response distributions in the exponential family

(binomial, Poisson, and extensions)

Any smooth, monotonic link function

(e.g. logistic, exponential models)

Flexible combinations of blocking factors

(clustering; random e�ects)

Applications in ecology, neurobiology, behaviour, epidemiology, realestate, . . .

Ben Bolker













Ben Bolker













Ben Bolker













Ben Bolker



Examples

ecology survival, predation, etc. (experimental plots)

genomics presence/absence of polymorphisms, gene expression(individuals)

educational assessment student scores (students × teachers)

psychology/sensometrics decisions, responses to stimuli(individuals)

epidemiology disease prevalence (postal codes, provinces, countries)

Ben Bolker



Examples






Ben Bolker



Examples






Ben Bolker



Examples






Ben Bolker



Examples






Ben Bolker



Technical de�nition

Yi︸︷︷︸response

∼

conditionaldistribution︷︸︸︷Distr (g−1(ηi )︸︷︷︸

inverselink

function

, φ︸︷︷︸scale

parameter

)

η︸︷︷︸linear

predictor

= Xβ︸︷︷︸�xede�ects

+ Zb︸︷︷︸randome�ects

b︸︷︷︸conditionalmodes

∼ MVN(0, Σ(θ)︸︷︷︸variance-covariancematrix

)

Ben Bolker



Outline





5 Conclusions

Ben Bolker



Estimation

Maximum likelihood estimation

L(Yi |θ,β)︸︷︷︸likelihood

=

∫· · ·

∫L(Yi |θ,β′)︸︷︷︸

data|random e�ects

×L(β′|Σ(θ))︸︷︷︸random e�ects

dβ′

deterministic: precision vs. computational cost:penalized quasi-likelihood, Laplace approximation, adaptiveGauss-Hermite quadrature (Breslow, 2004) . . .

Monte Carlo: frequentist and Bayesian (Booth and Hobert,1999; Ponciano et al., 2009; Sung, 2007)

Ben Bolker



Estimation: example (McKeon et al., 2012)

Log−odds of predation−6 −4 −2 0 2

Symbiont

Crab vs. Shrimp

Added symbiont

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

GLM (fixed)GLM (pooled)PQLLaplaceAGQ

Ben Bolker



Inference

Standard inferential tools:mostly asymptotic oruncontrolled approximations

Solutions are computationaland/or Bayesian: parametricbootstrap, MCMC

Good news: di�erent problemsfor small vs large data

True p value

Infe

rred

p v

alue

0.02

0.04

0.06

0.08

0.02 0.06

Osm Cu

H2S

0.02 0.06

0.02

0.04

0.06

0.08

Anoxia

Ben Bolker



Inference




True p value

Infe

rred

p v

alue

0.02

0.04

0.06

0.08

0.02 0.06

Osm Cu

H2S

0.02 0.06

0.02

0.04

0.06

0.08

Anoxia

Ben Bolker



Inference




True p value

Infe

rred

p v

alue

0.02

0.04

0.06

0.08

0.02 0.06

Osm Cu

H2S

0.02 0.06

0.02

0.04

0.06

0.08

Anoxia

Ben Bolker



Outline





5 Conclusions

Ben Bolker



Problems of big data

How big is big?Airline data: 12G

(G)LMM works on moderately large problems,e.g. student evaluations(≈ 75K total, 3K students, 1K profs)

Fairly clever linear algebra

Possible improvements?

Chunking/parallelizationOut-of-memory operation

Ben Bolker



Sparse matrix algorithms

repeated decomposition oflarge, matrices (especially Z )

�ll-reducing permutation toimprove sparsity pattern

further improvements possible:better matrix representation,parallelization?

Ben Bolker



Bounded optimization

Parameterizevariance-covariance matrix Σ(θ)(Pinheiro and Bates, 1996)

Positive de�nite or onlysemi-de�nite?

Disadvantages of transformingto unconstrain

(Disadvantages of boundarysolutions)

raw log

0

10

20

30

0 1 2 3 −3 −2 −1 0

devi

ance

Ben Bolker



Outline





5 Conclusions

Ben Bolker



Language tradeo�s

high-level/convenience: R

low-level/performance: C++

new wave? Julia

multi-language friction: mostly escaped in R/C++ case, atthe price of complexity

Ben Bolker



Getting it right vs. getting it written

the curse of neophilia: Superiority

many versions: nlme, lme4(a,b,Eigen) . . .

The moral of the story is that if

you want to create a beautiful

language, for god's sake don't

make it useful

(Patrick Burns)

Ben Bolker


http://www.mayofamily.com/RLM/txt_Clarke_Superiority.html

http://www.burns-stat.com/pages/Present/infernoishR_annotated.pdf


Sociological issues

Wide user base:

As usual when software for complicated statistical

inference procedures is broadly disseminated, there is

potential for abuse and misinterpretation.

(Breslow, 2004)

What if there is no good answer?�do no harm� vs. �better me than someone else�

Diagnostics and warning messages

End users vs. downstream developers

Ben Bolker



Outline





5 Conclusions

Ben Bolker



Next steps

Alternative platforms/languages

Flexible correlation structures:spatial, temporal, phylogenetic . . .

Improved MCMC methods?

Simulation tests of inferential tools

Ben Bolker



Is it science?

Science is what we

understand well enough to

explain to a computer. Art

is everything else we do.

(Donald Knuth)

10

20

30

4050

2006 2008 2010 2012Date

artic

les

per

mon

th

key

glmm

lme4

Public Library of Science data

Ben Bolker



Acknowledgments

lme4: Doug Bates, MartinMächler, Steve Walker

Data: Adrian Stier (UBC/OSU),Sea McKeon (Smithsonian),David Julian (UF)

NSERC (Discovery)

SHARCnet

Ben Bolker



Booth, J.G. and Hobert, J.P., 1999. Journal of the Royal Statistical Society. Series B, 61(1):265�285.doi:10.1111/1467-9868.00176.

Breslow, N.E., 2004. In D.Y. Lin and P.J. Heagerty, editors, Proceedings of the second Seattlesymposium in biostatistics: Analysis of correlated data, pages 1�22. Springer. ISBN 0387208623.

McKeon, C.S., Stier, A., et al., 2012. Oecologia, 169(4):1095�1103. ISSN 0029-8549.doi:10.1007/s00442-012-2275-2.

Pinheiro, J.C. and Bates, D.M., 1996. Statistics and Computing, 6(3):289�296.doi:10.1007/BF00140873.

Ponciano, J.M., Taper, M.L., et al., 2009. Ecology, 90(2):356�362. ISSN 0012-9658.

Sung, Y.J., 2007. The Annals of Statistics, 35(3):990�1011. ISSN 0090-5364.doi:10.1214/009053606000001389.

Ben Bolker


Technology

computational science & engineering seminar, 16 oct 2013