Progress on the software developed under E-STAT Bill Browne and Chris Charlton

Why the slides? To remind me which templates to demonstrate Lets start with simply demonstrating 1 template Template1lev with the tutorial dataset and a regression model in E-STAT Run in E-STAT Show generated C code Show java script and explain the server possibilities Show the template code (maybe discuss input functions)

Explanation for following screenshots We show how to set up the model and run using the E-STAT engine The generated C code can be taken away and run on its own or modified by an algorithm writer The Java script code is similar however offers the opportunity of hosting the software on a server but running the estimation (via Java script) on the local machine.

Setting up first model

Equations for model and model code

Output from the E-STAT engine

Output of generated C code

View of Java Script source

Java Script output

Template input code class Template1Lev(Template): invars = ''' y = DataVector('response: ') D = Text('specify distribution: ', ['Normal', 'Binomial', 'Poisson']) if D.name == 'Binomial': n = DataVector('denominator: ') link = Text('specify link function: ', ['logit', 'probit', 'cloglog']) if D.name == 'Poisson': link = Text(value = 'ln') offset = Text('Is there an offset: ', ['yes', 'no']) if offset.name == 'yes': n = DataVector('offset: ') if D.name == 'Normal': tau = ParamScalar() sigma = ParamScalar() x = DataMatrix('explanatory variables: ') beta = ParamVector() beta.ncols = len(x.name) ''' This code matches with the inputs in the web interface

Stringing Templates together Use rats dataset and aim to fit random intercepts model: Use template split and demonstrate the view and summary buttons Form the dataset called ratlong Choose as template2lev and ratlong dataset. Again use view to look at the dataset created. Setup the random intercepts model

Viewing a data set via the view button

Summarising the data (like MLwiN names window)

Converting repeated measures data to a single response (Templatesplit)

Viewing the new dataset

Setting up a random intercepts model (with template2lev)

Output from the model

Rats continued Fit the model storing results in ratout Construct the VPC (= (1/tau_u)/((1/tau_u)+(1/tau)) !!!) using TemplateEvalute storing in ratsout again Then view the chain using TemplateColumndiag Finally look at the residuals using Templatecaterpillar

Using TemplateEvaluate to construct the VPC

View the file ratout (including VPC)

Using TemplateColumnDiag to look at the VPC

Set up templateCaterpillar for Caterpillar plot

Caterpillar plot

Other Stuff Graphics – show with tutorial (or with output u_1 and tau_u) TemplateXYlabel TemplateHistogram And then show the python code! Finally large numbers of model templates – some still to do depends on Bruces algebra system.

TemplateXYlabel

All code for templateXYlabel – quite a short template! from EStat.Templating import * from mako.template import Template as MakoTemplate class TemplateXYLabel(Template): invars = ''' yaxis = DataMatrix('Y values: ') xaxis = DataVector('X values: ') yaxislabel = Text('Y label: ') xaxislabel = Text('X label: ') '' def preparedata(self, data): self.data = data return self.data def resultdata(self, m): return self.data def graphdata(self, data): import numpy from matplotlib.figure import Figure import matplotlib.lines as lines from matplotlib.backends.backend_agg import FigureCanvasAgg import subprocess import tempfile import os fig = Figure(figsize=(8,8)) ax = fig.add_subplot( , xlabel = str(self.objects['xaxislabel'].name), ylabel = str(self.objects['yaxislabel'].name)) for n in self.objects['yaxis'].name: ax.plot(self.data[self.objects['xaxis'].name], self.data[n], 'x') canvas = FigureCanvasAgg(fig) directory = tempfile.mkdtemp() + os.sep canvas.print_figure(directory + 'xyplot.png', dpi=80) return directory + 'xyplot.png'

TemplateHistogram

Classification as index notation Tutorial dataset and Template2levindex Set up model then fire up Bruces demo (via notepad) Next show the C code Finally run the model Camille to demo interoperability later. Going off piste: Any other models ? Any suggestions for improvements, wish list etc.

Setting up model in E-Stat

Model Code in detail model { for (i in 1:length(normexam)) { normexam[i] ~ dnorm(mu[i], tau) mu[i] <- cons[i] * beta0 + standlrt[i] * beta1 + u[school[i]] * cons[i] } for (j in 1:length(u)) { u[j] ~ dnorm(0, tau_u) } # Priors beta0 ~ dflat() beta1 ~ dflat() tau ~ dgamma( , ) tau_u ~ dgamma( , ) }

Bruces Demo algebra system step for u

Generated C code – note the if statements for u

Model executed in E-Stat Note code takes longer but generalises easily to cross-classified models.

Interoperability with WinBUGS

Output from WinBUGS with multiple chains