7/31/2019 Win Bugs

1/19

WinBUGS

Software for Bayesian Inference

7/31/2019 Win Bugs

2/19

OUTLINE

Brief overview of WinBUGS

Basics of Bayesian Inference

Navigating in WinBUGS Examples

7/31/2019 Win Bugs

3/19

What is WinBUGS?

WinBUGS is software used for analysis of Bayesian StatisticalModels

Uses Markov Chain Monte Carlo (MCMC) techniques to obtainestimates of posterior distributions

WinBUGS is a windows based version of older software called

BUGS. BUGS stands for Bayesian inference Using Gibbs Sampling

WinBUGS is FREE (thats right, free)

WinBUGS handles simple to complex statistical models

WinBUGS is object-oriented. That is, sections of your code can be

written in somewhat random order. You highlight the code you wantto run before running it.

A Help manual comes with the software (available via the helpwindow)

There is a graphical mode of writing WinBUGS code that we wont

cover today, called Doodles.

7/31/2019 Win Bugs

4/19

Review of Bayesian Inference

Recall: Letting represent a parameter of interest, we specify a prior distributionfor . In addition, we specify the Likelihood of using standard inference methods.

Note: The Prior distribution of represents our prior beliefs about . A distributionwith a small variance says you have a lot of prior information about . Largervariance implies little or no prior information about .

Inference about comes from the Posterior distribution of given Y.

The posterior distribution is computed from the prior and likelihood. Symbolicallywe state posterior(|Y) prior() x likelihood(). That is posterior is proportionalto prior times likelihood. The constant of proportionality is a constant required tomake posterior(|Y) a valid probability density (that is, integrate to 1).

Sometimes, the posterior distribution can be calculated simply.

Example: Let have a gamma(a,b) prior, where a and b are known. Let Y be theobserved data, and let Y1, Y2, , Yn be iid from a Poisson() distribution. Then theposterior of is proportional to.

ba

a

i

Yn

ebaY

ei

11

!

7/31/2019 Win Bugs

5/19

Bayesian Inference - continued

Simplifying somewhat reduces this expression to

which is a kernel of a Gamma ( ) distribution.

Often, however, the posterior does not easily reduce into arecognizable form. The mathematics of computing the appropriateposterior was a significant challenge for Bayesians beforecomputing became prevalent.

WinBUGS uses information about the likelihood and prior tosample from the posterior distribution. With enough samples avery good approximation to the posterior distribution is assembled.

1

1

exp)tan(aY

i

nbb

tscons

1,

nb

baY

i

7/31/2019 Win Bugs

6/19

WinBUGS Menus

Like in SAS, you will submit your programs (andmonitor output) through Menus.

Important WinBUGS Menus

Attributes: font attributes (size, color, etc.)

Model: allows you to submit your code; also allowsyou to run MCMC samples to estimate the posterior

Inference: declares which parameter(s) or functions

you want to have monitored through the MCMCprocess. All parameters for which you requireposterior estimates should be entered here.

7/31/2019 Win Bugs

7/19

WinBUGS program syntax

All WinBUGS programs require 3 sections:

Stating the likelihood and prior(s) (called the Model statement)

Entering the observed data

Entering a set of initial values for the parameters. Doing this gives theMCMC algorithm a set of starting values for the parameters.

Syntax:

All defined variables will be either stochastic (random) or deterministic(directly equal to some value or variable).

Specifying a stochastic variable requires the syntax ~

Example: If the variable X is a Poisson random variable with mean m, then you

would write X~dpois(m) in your program to define X.

Specifying a deterministic variable requires the syntax

7/31/2019 Win Bugs

8/19

Program Syntax - continued

In WinBUGS, the symbols used to the start and end of sections of code are { and }.

Do loops are written as

for (j in 1:100){commandline1commandline2

.

.

.last command line}

Arrays are written as [] after a variable name.

For example, given n observations on a variable Y, Yi would be written as Y[i] in the code.

Entering Data or Initial Values starts with a List command. Then, in parentheses,you enter values for all required variables. Example: To enter 5 observations for Y[i]; and the value 9.2 for a constant k, the syntax

would be list(y=c(3, 5, 1, 0, 2),k=9.2)

7/31/2019 Win Bugs

9/19

Example 1

Continuing with our model from earlier, letY~Poisson() distribution.

Consider a Gamma(a,b) prior for

If we observe the following data for Y: Y: 3, 5, 1, 0, 2, 1, 4, 3, 1, 6 We will let a=1 and b=2 for purposes of this example.

In reality these values would be chosen after carefulexamination.

We want to analyze the posterior distribution of . Now lets see how this program would be written

in WinBUGS

7/31/2019 Win Bugs

10/19

Example 1 - continued

Model

#STATE THE LIKELIHOOD

{

for (i in 1:10)

{

y[i]~dpois(lambda)

}

#PRIOR DISTRIBUTION FOR LAMBDA

lambda~dgamma(a,b)

}

#ENTER THE DATA

list(y=c(3, 5, 1, 0, 2, 1, 4, 3, 1, 6),a=1,b=0.5)

#SET INITIAL VALUES FOR LAMBDA

list(lambda=2)

list(lambda=8)

list(lambda=0)

7/31/2019 Win Bugs

11/19

Example 1 - continued

To submit this program:

1. Highlight the part of the code starting with model, through at least the first { symbol.

2. Click the Model menu and select the Specification option.

3. Click Check Model. If the likelihood is written correctly, a message at the bottom of the screen appearssaying Model is syntactically correct.

4. Highlight the data part of the code, starting at the word list.

5. Click the Load Data button. You should see a confirmation message stating Data Loaded at thebottom of the screen.

6. Enter 3 in the Number of Chains field. Click the Compile Button. You should receive the messageModel Compiled.

7. Highlight the first initial value. Here this says list(lambda=2). Click the Load Inits button. Continuesimilarly for the 2nd and 3rdinitial values, clicking Load Inits after highlighting each list one at a time.

8. Next click the Inference Menu, and then the Samples option. In the Node field type lambda. Thenclick set.

9. Finally, click the Model Menu, and then the Update option. Here, you tell WinBUGS how manyMCMC updates (samples) to run to estimate the posterior. You should choose at least 1000 updates ingeneral. Since this code runs quickly, enter 5000 updates. Then click Update.

7/31/2019 Win Bugs

12/19

Example 1 - continued Viewing Posterior Information

In the Sample Monitor Toolwindow, click Lambda as the node.

To view the posterior density ofLambda, click the Density button.

To get mean, std dev, median, and

percentiles for the posterior, clickStats.

We should also check the Historyoption to ensure the 3 MCMC chainshave converged. In this window,each chain has a separate color.Ideally, you should see all the colors

mixing together well.

As a check, we expected ourposterior distribution to beGamma(27, 2/21).

lambda chains 1:3 sample: 15000

0.0 2.0 4.0

0.00.25

0.50.751.0

Output should look like:

node m ean s d M C e rror 2.5% m edian 97.5% s tar t s ample

lambda 2.57 0.4976 0.004184 1.695 2.535 3.642 1 15000

lambda chains 1:3

iteration

1 2000 4000

1.0

2.0

3.0

4.0

5.0

7/31/2019 Win Bugs

13/19

Example 1 - continued

Expected Results: Given what we know about Gamma(27,2/21)

distributions:

Mean: 2.571

Std Deviation: 0.2449

2.5th percentile: 1.695

97.5th percentile: 3.628

Compare this with your WinBUGS posterioractual result.

7/31/2019 Win Bugs

14/19

Example 2

Lets look at an example where the parameter of interest is multivariate.

Let Y~N(mu, tau) where both mu and tau are unknown. In WinBUGS,tau=1/variance is the way WinBUGS specifies the Normaldistribution.

We know tau is is positive, so we pick a gamma prior for sigma.

Lets assume mu is known to be in the range 0 to 10, but that lowervalues are more likely to occur. So we pick an exponential prior formu, and restrict possible values to be

7/31/2019 Win Bugs

15/19

Example 2 Code

#STATE THE LIKELIHOOD

Model{for (i in 1:20)

{y[i]~dnorm(mu,tau)

}

#PRIOR DISTRIBUTIONS FOR TAU&MU#Note: TAU is 1/Variance, also called the precision

tau~dgamma(a,b)mu~dexp(1)I(0,10)var

7/31/2019 Win Bugs

16/19

Example 2

The data came from a N(1.6, Var=4)distribution. Do your posterior distributionssupport this?

Try changing some of the parameters ofyour prior distributions and re-running thecode and see what happens to yourestimates.

7/31/2019 Win Bugs

17/19

Example 3 Logistic Regression

Researchers were interested in p, the probabilityof germination of 2 types of seeds, plantedaccording to 2 root extract methods.

To get the code, go to the Help menu in

WinBUGS. Click Examples Vol I. Then click the Seeds: random effects logistic

regression link. Authors used non-informative priors (high

variance). Run the code to get posterior estimates for

alpha0, alpha1, alpha2, and alpha12.

7/31/2019 Win Bugs

18/19

Burn In Samples

Often users use a burn in period of initialsamples that they discard when estimatingthe posterior distribution. This period

allows the MCMC sampling procedure tostabilize.

To leave out k burn-in samples forestimating the posterior, enter the numberk+1 in the Beg field of the SampleMonitor Tool.

7/31/2019 Win Bugs

19/19

Final Points

The Help manual contains much useful information,including several examples.

You should Always verify how WinBUGS specifies thedistributions. For example, their scale parameter for the

normal distribution is the inverse of WinBUGS can be obtained for FREE from the following

website: http://www.mrc-bsu.cam.ac.uk/bugs/

Click the WinBUGS option on the left side of the screen

To learn more, take Dr. Ghoshs Bayesian Inferenceclass offered this fall.

2

http://www.mrc-bsu.cam.ac.uk/bugs/http://www.mrc-bsu.cam.ac.uk/bugs/http://www.mrc-bsu.cam.ac.uk/bugs/http://www.mrc-bsu.cam.ac.uk/bugs/