Progress on the software developed under E-STAT

Bill Browne and Chris Charlton

Why the slides?• To remind me which templates to demonstrate

• Let’s start with simply demonstrating 1 templateTemplate1lev with the tutorial dataset and a

regression model in E-STAT• Run in E-STAT• Show generated C code • Show java script and explain the server

possibilities• Show the template code (maybe discuss input

functions)

Explanation for following screenshots

• We show how to set up the model and run using the E-STAT engine

• The generated C code can be taken away and run on it’s own or modified by an algorithm writer

• The Java script code is similar however offers the opportunity of hosting the software on a server but running the estimation (via Java script) on the local machine.

Setting up first model

Equations for model and model code

Output from the E-STAT engine

Output of generated C code

View of Java Script source

Java Script output

Template input codeclass Template1Lev(Template): invars = '''y = DataVector('response: ')D = Text('specify distribution: ', ['Normal', 'Binomial', 'Poisson'])if D.name == 'Binomial': n = DataVector('denominator: ') link = Text('specify link function: ', ['logit', 'probit', 'cloglog'])if D.name == 'Poisson': link = Text(value = 'ln') offset = Text('Is there an offset: ', ['yes', 'no']) if offset.name == 'yes': n = DataVector('offset: ')if D.name == 'Normal': tau = ParamScalar() sigma = ParamScalar()x = DataMatrix('explanatory variables: ')beta = ParamVector()beta.ncols = len(x.name) '''

This code matches with the inputs in the web interface

Stringing Templates together

• Use rats dataset and aim to fit random intercepts model:

• Use template split and demonstrate the view and summary buttons

• Form the dataset called ratlong• Choose as template2lev and ratlong dataset.• Again use view to look at the dataset created.• Setup the random intercepts model

Viewing a data set via the view button

Summarising the data (like MLwiN names window)

Converting repeated measures data to a single response (Templatesplit)

Viewing the new dataset

Setting up a random intercepts model (with template2lev)

Output from the model

Rats continued

• Fit the model storing results in ratout• Construct the VPC (= (1/tau_u)/((1/tau_u)

+(1/tau)) !!!) using TemplateEvalute storing in ratsout again

• Then view the chain using TemplateColumndiag

• Finally look at the residuals using Templatecaterpillar

Using TemplateEvaluate to construct the VPC

View the file ratout (including VPC)

Using TemplateColumnDiag to look at the VPC

Set up templateCaterpillar for Caterpillar plot

Caterpillar plot

Other Stuff

• Graphics – show with tutorial (or with output u_1 and tau_u)

TemplateXYlabelTemplateHistogramAnd then show the python code!• Finally large numbers of model templates –

some still to do depends on Bruce’s algebra system.

TemplateXYlabel

All code for templateXYlabel – quite a short template!

from EStat.Templating import *from mako.template import Template as MakoTemplateclass TemplateXYLabel(Template): invars = '''yaxis = DataMatrix('Y values: ')xaxis = DataVector('X values: ')

yaxislabel = Text('Y label: ')xaxislabel = Text('X label: ')''‘ def preparedata(self, data): self.data = data return self.data def resultdata(self, m): return self.data def graphdata(self, data): import numpy from matplotlib.figure import Figure import matplotlib.lines as lines from matplotlib.backends.backend_agg import

FigureCanvasAgg

import subprocess import tempfile import os fig = Figure(figsize=(8,8)) ax = fig.add_subplot(100 + 10 + 1, xlabel = str(self.objects['xaxislabel'].name), ylabel = str(self.objects['yaxislabel'].name)) for n in self.objects['yaxis'].name: ax.plot(self.data[self.objects['xaxis'].name], self.data[n], 'x')

canvas = FigureCanvasAgg(fig) directory = tempfile.mkdtemp() + os.sep canvas.print_figure(directory + 'xyplot.png', dpi=80) return directory + 'xyplot.png'

TemplateHistogram

Classification as index notation

• Tutorial dataset and Template2levindex• Set up model then fire up Bruce’s demo (via

notepad)• Next show the C code• Finally run the model• Camille to demo interoperability later.• Going off piste: Any other models ?• Any suggestions for improvements, wish list etc.

Setting up model in E-Stat

Model Code in detailmodel { for (i in 1:length(normexam)) { normexam[i] ~ dnorm(mu[i], tau) mu[i] <- cons[i] * beta0 + standlrt[i] * beta1 + u[school[i]] * cons[i] } for (j in 1:length(u)) { u[j] ~ dnorm(0, tau_u) } # Priors beta0 ~ dflat() beta1 ~ dflat() tau ~ dgamma(0.001000, 0.001000) tau_u ~ dgamma(0.001000, 0.001000) }

Bruce’s Demo algebra system step for u

Generated C code – note the if statements for u

Model executed in E-Stat

Note code takes longer but generalises easily to cross-classified models.

Interoperability with WinBUGS

Output from WinBUGS with multiple chains