MSH3 Generalized linear model Ch. 6 Count data …SydU MSH3 GLM (2015) First semester Dr. J. Chan 208 The plot of variance against mean in log scale for all cells with at least 20

MSH3 Generalized linear model Ch. 6 Count data models

Contents

§6 Count data model 208

§6.1 Introduction: The Children Ever Born Data . . . . . . . 208

§6.2 The Poisson Distribution . . . . . . . . . . . . . . . . . 210

§6.3 Log-Linear Models . . . . . . . . . . . . . . . . . . . . . 211

§6.4 ML estimation . . . . . . . . . . . . . . . . . . . . . . . 212

§6.5 Goodness of fit and hypothesis test . . . . . . . . . . . . 213

§6.6 Children ever born example . . . . . . . . . . . . . . . . 214

§6.6.1 Grouped Data and the Offset . . . . . . . . . . . 214

§6.6.2 The Deviance Table . . . . . . . . . . . . . . . . 215

§6.6.3 The Additive Model . . . . . . . . . . . . . . . . 217

§6.7 Negative binomial distribution . . . . . . . . . . . . . . . 220

§6.8 Model for excess zero . . . . . . . . . . . . . . . . . . . 222

§6.8.1 Zero-inflated Poisson model . . . . . . . . . . . . 222

§6.8.2 Hurdle model . . . . . . . . . . . . . . . . . . . . 224

§6.9 Model selection example . . . . . . . . . . . . . . . . . . 224

SydU MSH3 GLM (2015) First semester Dr. J. Chan 207


§6 Count data model

In this chapter we study log-linear models for count data with a Poisson

error structure and a log link function.

§6.1 Introduction: The Children Ever Born Data

The following table, adapted from Little (1979), comes from the Fiji

Fertility Survey, typical in the reports of the World Fertility Survey.

Each cell in the table shows the mean, the variance and the number of

observations.

The unit of analysis is the individual woman, the response is the number

of children she has borne, and the 3 discrete predictors are the duration

since her first marriage, the type of place of residence (Suva the capital,

other urban and rural), and her educational level (none, lower primary,

upper primary, and secondary or higher).

Data such as these have traditionally been analyzed using ordinary lin-

ear models with normal errors. The key concern, however, is not the

normality of the errors but rather the assumption of constant variance.

0.0 0.5 1.0 1.5 2.0

−0.

50.

00.

51.

01.

52.

02.

5

log(mean)

log(

var)

Figure 6.1: The Mean-variance Relationship for the CEB Data



The plot of variance against mean in log scale for all cells with at least

20 observation shows clearly that, the assumption of constant variance is

not suitable and the relationship is close to proportional. Thus, a Poisson

regression models is more suitable than a ordinary linear models.

Table 6.1: Number of Children Ever Born to Women of Indian RaceBy Marital Duration, Type of Place of Residence and Educational Level(Each cell shows the mean, variance and sample size)

Marr. Suva Urban RuralDur. N LP UP S+ N LP UP S+ N LP UP S+

0-4 0.50 1.14 0.90 0.73 1.17 0.85 1.05 0.69 0.97 0.96 0.97 0.741.14 0.73 0.67 0.48 1.06 1.59 0.73 0.54 0.88 0.81 0.80 0.59

8 21 42 51 12 27 39 51 62 102 107 47

5-9 3.10 2.67 2.04 1.73 4.54 2.65 2.68 2.29 2.44 2.71 2.47 2.241.66 0.99 1.87 0.68 3.44 1.51 0.97 0.81 1.93 1.36 1.30 1.1910 30 24 22 13 37 44 21 70 117 81 21

10-14 4.08 3.67 2.90 2.00 4.17 3.33 3.62 3.33 4.14 4.14 3.94 3.331.72 2.31 1.57 1.82 2.97 2.99 1.96 1.52 3.52 3.31 3.28 2.5012 27 20 12 18 43 29 15 88 132 50 9

15-19 4.21 4.94 3.15 2.75 4.70 5.36 4.60 3.80 5.06 5.59 4.50 2.002.03 1.46 0.81 0.92 7.40 2.97 3.83 0.70 4.91 3.23 3.29 -14 31 13 4 23 42 20 5 114 86 30 1

20-24 5.62 5.06 3.92 2.60 5.36 5.88 5.00 5.33 6.46 6.34 5.74 2.504.15 4.64 4.08 4.30 7.19 4.44 4.33 0.33 8.20 5.72 5.20 0.5021 18 12 5 22 25 13 3 117 68 23 2

25-29 6.60 6.74 5.38 2.00 6.52 7.51 7.54 - 7.48 7.81 5.80 -12.40 11.66 4.27 - 11.45 10.53 12.60 - 11.34 7.57 7.07 -

47 27 8 1 46 45 13 - 195 59 10 -



§6.2 The Poisson Distribution

A random variable Y is said to have a Poisson distribution with param-

eter µ if it takes integer values y = 0, 1, 2, . . . with probability

Pr(Y = y) =e−µµy

y!

for µ > 0. The mean and variance are

E(Y ) = Var(Y ) = µ.

Since the mean is equal to the variance, any factor that affects one will

also affect the other and hence homoscedasticity fails for Poisson data.

Poisson model captures the usual fact with count data that the variance

tends to increase with the mean.

The Poisson distribution can be derived as a limiting form of the bino-

mial distribution. Specifically, if Y ∼ B(n, π), Yapprox∼ P (µ) with fixed

µ = nπ as n→∞ and π → 0. Thus, the Poisson distribution provides

an approximation to the binomial for the analysis of rare events, where

µ is small and n is large.

Alternatively, the Poisson distribution is defined in terms of a stochastic

process described as follows. Suppose events occur randomly in time with

the following conditions:

• The probability of at least one occurrence of the event in a given

time interval is proportional to the length of the interval.

• The probability of two or more occurrences of the event in a very

small time interval is negligible.

• The numbers of occurrences of the event in disjoint time intervals

are mutually independent.



Then the probability distribution of the number of occurrences of the

event in a fixed time interval is Poisson with mean µ = λt, where λ is

the rate of occurrence of the event per unit time and t is the length of

the time interval. The proof will be given in the Appendix. A process

satisfying the three assumptions is called a Poisson process.

A useful property of the Poisson distribution is that the sum of indepen-

dent Poisson random variables is also Poisson. Specifically, if Y1 and Y2

are independent with Yi ∼ P (µi) for i = 1, 2 then Y1 +Y2 ∼ P (µ1 +µ2).

This result generalizes in an obvious way to the sum of more than two

Poisson observations.

For a group of ni individuals with identical covariate values, let Yij denote

the number of events experienced by the j-th unit in the i-th group, and

let Yi denote the total number of events in group i. Then, under the

usual assumption of independence, if Yij ∼ P (µi) for j = 1, 2, . . . ni,

then Yi ∼ P (niµi). In terms of estimation, we obtain exactly the same

likelihood function if we work with the individual counts Yij or the group

counts Yi.

§6.3 Log-Linear Models

Let Yi ∼ P (µi) independently and y1, y2, . . . yn are n observed values.

Let the mean µi (variance as well) depend on a vector of explanatory

variables xi in a simple linear model:

µi = x′iβ.

However, while x′iβ can assume any real value, the Poisson mean µi has

to be non-negative. One solution is to model the logarithm of the µi, a

canonical link for Poisson distribution, using a linear model:

log(µi) = x′iβ



where βj in β represents the expected change in the log of the mean per

unit change in the predictor xij. Exponentiating the equation, we obtain

a multiplicative model for the mean:

µi = exp(x′iβ) = exp (p∑j=1

xijβj) =

p∏j=1

exp(xijβj) =

p∏j=1

[exp(βj)]xij

which agrees with what we observe that the size of effects is proportional

to the count itself. Increasing xij by one unit, the mean µi is multiplied

by exp(βj).

§6.4 ML estimation

The log-likelihood function for n independent Poisson observations is

`(β) =

n∑i=1

yi log(µi)− µi − log(yi!).

Taking derivatives of `(β) with respect to the elements of β, and setting

them zero

∂`

∂βj=

n∑i=1

yiµiµixij − µixij =

n∑i=1

xij(yi − µi),

the maximum likelihood estimates in log-linear Poisson models satisfy

the estimating equations

X ′y = X ′µ

where X is the design matrix, with row xi for each observation and

one column for each predictor, including the constant (if any). This

estimating equation arises in any generalized linear model with canonical

link.

Thus, in models with a constant, one of the estimating equations matches

the sum of observed and fitted values. In models with a discrete factor,



such as the Children Ever Born example, the observed and fitted total

number of children ever born will be matched in each category of marital

duration since first marriage. This result generalizes to higher order

terms. The interaction model A+B+AB would match the totals in each

combination of categories of A and B, or the AB margin.

To estimate the parameters, we will use the iteratively-reweighted least

squares (IRLS) algorithm the working dependent variable z

zi = ηi +yi − µiµi

since ∂ηi∂µi

= ∂ lnµi∂µi

= 1µi

and the diagonal matrix W of iterative weights

has elements

wii = [V ar(zi)]−1

= [V ar(yi) (∂ηi∂µi

)2]−1

= [µi(1

µi)

2]−1

= µi

where µi is the fitted values based on the current parameter estimates.

Initial values can be obtained by applying the link to the data, that is

taking the log of the response, and regressing it on the predictors using

OLS. The procedure usually converges in a few iterations.

§6.5 Goodness of fit and hypothesis test

1. Deviance: It measures the discrepancy between observed and fitted

values for Poisson responses:

D = 2

n∑i=1

[yi log

(yiµi

)− (yi − µi)

]n→∞∼ χ2

n−p

where p the number of parameters. The first term is identical to

the binomial deviance and the second term, a sum of differences

between observed and fitted values, is usually zero for MLE which

march marginal totals.



2. Pearson’s chi-squared statistic: It is the squared difference between

observed and fitted values to the variance of the observed value:

X2 =

n∑i=1

(yi − µi)2

µi

n→∞∼ χ2n−p

which the same for Poisson and binomial data.

3. Likelihood ratio test: It can be constructed in terms the difference

in deviances between two nested model:

−2 lnλ =D(ω1)−D(ω2)

φ

n→∞∼ χ2p2−p1

under the assumption that the smaller model ω1 is correct in H0.

4. Wald test: To test H0 : β = β0, the test statistics is

W0 = (β − β0)′[var(β)]−1(β − β0)n→∞∼ χ2

p

based on the fact that the MLE

β ∼ Np(β,X′WX)

where W is the diagonal matrix of estimation weights wii.

§6.6 Children ever born example

§6.6.1 Grouped Data and the Offset

Let Yijkl denote the number of children from the l-th woman in the

(i, j, k)-th group, where i denotes marital duration, j residence and k

education and with mean µijk. Let Yijk =∑

l Yijkl denote the group

total. The group total is a realization of a Poisson variate with mean

nijkµijk, where nijk is the number of observations in the (i, j, k)-th cell.

Suppose that a log-linear model is postulated for the individual means:

logE(Yijkl) = log(µijk) = x′ijkβ



where xijk is a vector of covariates. Then the log of the expected value

of the group total is

logE(Yijk) = log(nijkµijk) = log(nijk) + x′ijkβ.

Thus the group totals have the same coefficients β as the individual

means, except that the linear predictor includes the term log(nijk) called

an offset which is the log of exposure measures, the number of women

here.

Thus, we can fit log-linear models to the individual counts or group

totals with the log of the number of women in each cell as an offset. The

deviances is different because they measure goodness of fit to different

sets of counts but differences of deviances between nested models are

exactly the same.

§6.6.2 The Deviance Table

Table 7.2 reports the results of fitting a variety of Poisson models. The

null model assuming the same number of children for all the groups is

rejected based (D = 3732 on 69 df). Introducing marital duration

reduces the deviance to 165.8 on 64 df. The drop of 3566 on 5 df shows

that the number of CEB depends on the duration since the first marriage.

To test the effect of education controlling for duration, the additive

model D + E relative to the one-factor D model gives a chi-squared

statistic of 65.8 (165.84-100.01) on 3 (64-61) df and is highly significant.

We can also test the net effect of education controlling for residence &

duration, by comparing the 3-factor additive model D + R + E with the

2-factor model D + R. The difference in deviances of 50.1 (120.68-70.65)

on 3 (62-59) df is highly significant. This smaller Chi-square statistic

indicates that part of the education effect may due to the fact that more

educated women live in Suva or in other urban areas.



Table 6.2: Deviances for Poisson log-linear models fitted to CEB data

Type Model Deviance d.f.

Null 3731.52 69

1-factor Models Duration 165.84 64

Residence 3659.23 67

Education 2661.00 66

2-factor Models D + R 120.68 62

D + E 100.01 61

DR 108.84 52

DE 84.46 46

3-factor Models D + R + E 70.65 59

D + RE 59.89 53

E + DR 57.06 49

R + DE 54.91 44

DR + RE 44.27 43

DE + RE 44.60 38

DR + DE 42.72 34

DR + DE + RE 30.95 28

Does education make more of a difference in rural areas than in

urban areas? To answer this question we compare models D+RE to

D+R+E. The reduction in deviance is 10.8 (70.65-59.89) on 6 (59-53) df

and is not significant, with a P-value of 0.096.

Does the effect of education increase with marital duration? Adding

DE (model R+DE) to model D+R+E reduces the deviance by 15.7

(70.65-54.91) at the cost of 15 df (59-44), hardly a bargain. Thus, we

conclude that the additive model which has a deviance of 70.65 on 59

d.f. (an average of 1.2 per d.f.) is adequate for these data.



§6.6.3 The Additive Model

The associated P-value of the D+R+E model is 0.14 (Pr(χ259 > 70.65) =

0.14), so the model passes the goodness-of-fit test. Table 6.3 shows its

parameter estimates and standard errors.

Duration. The constant represents the log of the mean number of CEB

for the reference cell, which is Suvanese women with no education and

0-4 years marriage duration. Since exp(0.1173) = 0.89, these women

have 0.89 children on average at this time in their lives. The duration

parameters trace the increase in CEB with duration for any residence

and education group. The model is multiplicative in the original scale.

From duration 0-4 to 5-9 years, the number of CEB gets multiplied by

exp(0.9977) = 2.71. By duration 25-29 years, the number of CEB is

exp(1.977) = 7.22 times those at duration 0-4 for any residence type and

education level.

Residence. The effects of residence show that Suvanese women have

the lowest fertility. At any given marriage duration, women living in

other urban areas have 12% larger families (exp(0.1123) = 1.12) than

Suvanese women with the same level of education. Similarly, at any

fixed duration, women who live in rural areas have 16% more children

(exp(0.1512) = 1.16), than Suvanese women with the same level of edu-

cation.

Education. Higher education is associated with smaller family sizes

net of duration and residence. At any given marriage duration, women

with upper primary education have 10% fewer kids, and women with

secondary or higher education have 27% (1 − exp(−0.3096) = 0.27)

fewer kids, than women with no education in the same type of residence.



Table 6.3: Estimates for additive log-linear model of CEB data

Parameter Estimate Std. Error z-ratio

Constant -0.1173 0.0549 2.14

Duration 0-4 -

5-9 0.9977 0.0528 18.91

10-14 1.3705 0.0511 26.83

15-19 1.6142 0.0512 31.52

20-24 1.7855 0.0512 34.86

25-29 1.9768 0.0500 39.50

Residence Suva -

Urban 0.1123 0.0325 3.46

Rural 0.1512 0.0283 5.34

Education None -

Lower 0.0231 0.0227 1.02

Upper -0.1017 0.0310 -3.28

Sec+ -0.3096 0.0552 -5.61

The educational differential of 27% between these two groups translates

into a quarter of a child at durations 0-4, increases to about one child

around duration 15, and reaches almost one and three quarter children

by duration 25+. Thus, the effect of education increases with marital

duration. (5-9, Sec+: exp(−0.1173 + 0.9977− 0.3096) = 1.77)

Table 6.4: Fitted values for Suvanese women with no education and

with secondary or higher Education

Marital Duration 0-4 5-9 10-14 15-19 20-24 25+

No Education 0.89 2.41 3.50 4.47 5.30 6.42

Secondary+ 0.65 1.77 2.57 3.28 3.89 4.71

Difference 0.24 0.64 0.93 1.19 1.41 1.71



> rm(list=ls())

> data=read.table("CEB.txt",skip=1)

> mu=data[,1]

> ni=data[,3]

> ln=log(ni)

> dur=as.factor(data[,4])

> resid=as.factor(data[,5])

> edu=as.factor(data[,6])

> y=round(mu*ni,0)

> d0=glm(y~offset(ln),family=poisson)$dev

> d1=glm(y~offset(ln)+dur,family=poisson)$dev

> d2=glm(y~offset(ln)+resid,family=poisson)$dev

> d3=glm(y~offset(ln)+edu,family=poisson)$dev

> d4=glm(y~offset(ln)+dur+resid,family=poisson)$dev

> d5=glm(y~offset(ln)+dur+edu,family=poisson)$dev

> d6=glm(y~offset(ln)+dur*resid,family=poisson)$dev

> d7=glm(y~offset(ln)+dur*edu,family=poisson)$dev

> d8=glm(y~offset(ln)+dur+edu+resid,family=poisson)$dev

> d9=glm(y~offset(ln)+dur+edu*resid,family=poisson)$dev

> d10=glm(y~offset(ln)+dur*resid+edu,family=poisson)$dev

> d11=glm(y~offset(ln)+dur*edu+resid,family=poisson)$dev

> d12=glm(y~offset(ln)+dur*resid+edu*resid,family=poisson)$dev

> d13=glm(y~offset(ln)+dur*edu+resid*edu,family=poisson)$dev

> d14=glm(y~offset(ln)+dur*resid+dur*edu,family=poisson)$dev

> d15=glm(y~offset(ln)+dur*resid+dur*edu+resid*edu,family=poisson)$dev

> c(d0,d1,d2,d3,d4,d5,d6)

[1] 3731.8516 166.0717 3659.2791 2660.9976 120.6804 100.1917 108.8965

> c(d7,d8,d9,d10,d11,d12,d13)

[1] 84.53047 70.66530 59.92076 57.13495 54.80143 44.31104 44.52327

> summary(glm(y~offset(ln)+dur+resid+edu,family=poisson))

Call:

glm(formula = y ~ offset(ln) + dur + resid + edu, family = poisson)

Deviance Residuals:

Min 1Q Median 3Q Max



-2.2960 -0.6641 0.0725 0.6336 3.6782

Coefficients:

Estimate Std. Error z value Pr(>|z|)

(Intercept) -0.11710 0.05491 -2.132 0.032969 *

dur2 0.99693 0.05274 18.902 < 2e-16 ***

dur3 1.36940 0.05107 26.815 < 2e-16 ***

dur4 1.61376 0.05119 31.522 < 2e-16 ***

dur5 1.78491 0.05121 34.852 < 2e-16 ***

dur6 1.97641 0.05003 39.501 < 2e-16 ***

resid2 0.11242 0.03250 3.459 0.000541 ***

resid3 0.15166 0.02833 5.353 8.63e-08 ***

edu2 0.02297 0.02266 1.014 0.310597

edu3 -0.10127 0.03099 -3.268 0.001082 **

edu4 -0.31015 0.05521 -5.618 1.94e-08 ***

---

Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1

(Dispersion parameter for poisson family taken to be 1)

Null deviance: 3731.852 on 69 degrees of freedom

Residual deviance: 70.665 on 59 degrees of freedom

AIC: 522.14

Number of Fisher Scoring iterations: 4

§6.7 Negative binomial distribution

Negative binomial (NB) distribution is a discrete probability distribution

for the number of successes in a sequence of iid Bernoulli trials before a

specified number of failures, r, occurs. It’s pmf, mean and variance are

f (y) =

(y + r − 1

x

)(1− p)rpy,

E(Y ) =pr

1− p, and V ar(Y ) =

pr

(1− p)2.



As V ar(Y ) > E(Y ), it is overdispersed. Being a Poisson-Gamma mix-

ture,

Y ∼ NB(r, p) ⇔ Y ∼ P (λ), λ ∼ G(r,p

1− p)

it has higher variability than Poisson. The proof is:

f (y; r, p) =

∫ ∞0

λye−λ

y!

λr−1e−λ(1−p)/p

( p1−p)

rΓ(r)dλ

=(1− p)rp−r

y!Γ(r)

∫ ∞0

λr+y−1e−λ/pdλ

=(1− p)rp−r

y!Γ(r)pr+yΓ(r + y)

=Γ(r + y)

y!Γ(r)py(1− p)r =

(y + r − 1

y

)(1− p)rpy.

Poisson and NB distributions are related as:

P (λ) = limr→∞

NB

(r,

λ

λ + r

)since taking expectation on both sides, λ = rp

1−p ⇒ p = λλ+r . Note that

E(Y ) = limr→∞

pr

1− p= lim

r→∞

λλ+r · r

1− λλ+r

= λ

V ar(Y ) = limr→∞

pr

(1− p)2= lim

r→∞

λλ+r · r

(1− λλ+r)

2= lim

r→∞λλ + r

r= λ.

In fitting a negative binomial distribution to the CEB data,

> library(MASS)

> glm.nb(y ~ offset(ln)+dur+resid+edu)

Call: glm.nb(formula = y ~ offset(ln) + dur + resid + edu,

init.theta = 809399.1036, link = log)



Coefficients:

(Intercept) dur2 dur3 dur4 dur5 dur6

-0.11709 0.99694 1.36939 1.61375 1.78489 1.97640

resid2 resid3 edu2 edu3 edu4

0.11243 0.15166 0.02297 -0.10128 -0.31016

Degrees of Freedom: 69 Total (i.e. Null); 59 Residual

Null Deviance: 3730

Residual Deviance: 70.66 AIC: 524.1

There were 27 warnings (use warnings() to see them)

> data=read.table("CEBMeanVar.txt")

> mug=data[,1] #mean by group

> var=data[,2] #var by group

> mean(mug)

[1] 3.727609

> mean(var)

[1] 3.615435

There is no improvement in model fit (Residual Deviance & AIC) as the

data is equi-dispersed across groups.

§6.8 Model for excess zero

The model concerns a random event containing excess zero counts.

§6.8.1 Zero-inflated Poisson model

Proposed by Lambert (1992), the model contains two components for two

zero generating processes: a binary distribution that generates structural

zeros and a Poisson or Negative Binomial distributions that generates

counts, some of which may be zero. Essentially, it is a mixture model is



given by:

Pr(Y = 0) = π + (1− π)e−λ

Pr(Y = y) = (1− π)λye−λ

y!, y ≥ 1

where λ is the expected Poisson count here and π is the probability of

extra zeros. An example is the number of certain insurance claims within

a population that is zero-inflated by those people who have not taken out

insurance against the risk and thus are unable to claim. The mean and

variance are

E(Y ) = (1− π)λ and V ar(Y ) = λ(1− π)(1 + λπ)

since

V ar(Y ) = E(Y 2)− [E(Y )]2 = (1− π)(λ2 + λ)− (1− π)2λ2

= (1− π)[λ2 + λ− (1− π)λ2] = λ(1− π)(1 + λπ).

The method of moments estimators are given by

λ =s2 + y2 − y

yand π =

s2 − ys2 + y2 − y

where y is the sample mean and s2 is the sample variance. The maximum

likelihood estimator can be found by solving the following equation

y(1− e−λ) = λ(

1− n0

n

)(1)

for λ where n0n is the observed proportion of zeros. This can be solved

by iterations or the R module uniroot, and the maximum likelihood

estimator for π is given below (proofs in assignment):

π = 1− y

λ.



§6.8.2 Hurdle model

Proposed by Mullahy (1986), the model contains two components: a

binary model for measuring whether the outcome overcomes the hurdle

(say zero for zero-inflated data) and a truncated model like Poisson or

negative binomial (NB) distributions explaining those outcomes that pass

the hurdle. The model is given by:

Pr(Y = 0) = π

Pr(Y = y) = (1− π)λye−λ

y!(1− e−λ), y ≥ 1

The loglikelihood function is

`(π, λ) = n0 ln π+n1 ln(1−π)+lnλ

n1∑i=1

yi−n1λ−n1∑i=1

yi!−n1 ln(1−e−λ)

where n0 is the count of 0 and n1 is the count of yi > 0. Hence the MLE

are π = n0n and λ is the solution to (1).

§6.9 Model selection example

Maxell (1961) discusses a 5 × 4 table giving the number of boys with

4 different ratings for disturbed dreams in 5 different age groups. The

higher the rating the more the boys suffer from disturbed dreams.

Rating

Age 4 3 2 1

5-7 7 3 4 7

8-9 13 11 15 10

10-11 7 11 9 23

12-13 10 12 9 28

14-15 3 4 5 32



One way of modeling the data is to let the number in the (i, j)-th cell be

Yij and assume that Yij ∼ Poisson(µij).

Given∑5

i=1

∑4j=1 Yij = 223, the joint conditional distribution of the

Yij’s is multinomial. We look for a pattern in the µij relating to the

factors age and rating. We use log link.

The saturated model Ω : lnµij = µ0 + αi + βj + (αβ)ij

where αi is the age group effect, α1 = β1 = 0,

βj is the dream rating effect,

(αβ)ij is the interaction effects and (αβ)1j = 0 = (αβ)i1.

The null model φ : lnµij = µ0 where

µij = y =223

20= 11.15, µ0 = lnµij = ln(11.15) = 2.411

> y=c(7,3,4,7,13,11,15,10,7,11,9,23,10,12,9,28,3,4,5,32)

> age=factor(c(1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4,5,5,5,5))

> rate=factor(c(1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4))

> glm1=glm(y~age+rate,family=poisson)

> summary(glm1)

Call:

glm(formula = y ~ age + rate, family = poisson)

Deviance Residuals:


-2.86383 -0.71932 -0.05721 0.59370 2.53159

Coefficients:


(Intercept) 1.32623 0.26103 5.081 3.76e-07 ***

age2 0.84730 0.26082 3.249 0.00116 **

age3 0.86750 0.26004 3.336 0.00085 ***

age4 1.03302 0.25410 4.065 4.80e-05 ***



age5 0.73967 0.26523 2.789 0.00529 **

rate2 0.02469 0.22224 0.111 0.91153

rate3 0.04879 0.22093 0.221 0.82522

rate4 0.91629 0.18708 4.898 9.69e-07 ***

---

Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1


Null deviance: 94.607 on 19 degrees of freedom #n-p=20-1

Residual deviance: 32.457 on 12 degrees of freedom #n-p=20-1-3-4

AIC: 129.57


> ey=glm1$fitted.values

> ey

1 2 3 4 5 6 7 8

3.766816 3.860987 3.955157 9.417040 8.789238 9.008969 9.228700 21.973094

9 10 11 12 13 14 15 16

8.968610 9.192825 9.417040 22.421525 10.582960 10.847534 11.112108 26.457399

17 18 19 20

7.892377 8.089686 8.286996 19.730942

> n=length(y)

> mu=mean(y)

> mu

[1] 11.15

> D.null=2*sum(y*log(y/mu)-(y-mu))

> D.null

[1] 94.60676

> D.add=2*sum(y*log(y/ey)-(y-ey)) #additive age_rate model

> D.add

[1] 32.4571

> AIC.add=-2*sum(y*log(ey)-ey-log(factorial(y)))+2*(1+3+4)

> AIC.add

[1] 129.5738

To test if β1 = β2 = β3, i.e., rating can be collapsed to 2-levels, the



change in deviance, as compared to the saturated model with 0 deviance

and 0 d.f., is 4.2581 which distributed as χ210. Since the p-value > 0.05,

accept H0.

> r1=factor(c(1,1,1,2,1,1,1,2,1,1,1,2,1,1,1,2,1,1,1,2))

> glm3=glm(y~age*r1,family=poisson)

> summary(glm3)

Call:

glm(formula = y ~ age * r1, family = poisson)

Deviance Residuals:


-0.8260 -0.3434 0.0000 0.1203 1.0049

Coefficients:


(Intercept) 1.5404 0.2673 5.764 8.22e-09 ***

age2 1.0245 0.3116 3.288 0.00101 **

age3 0.6568 0.3293 1.994 0.04613 *

age4 0.7949 0.3220 2.469 0.01356 *

age5 -0.1542 0.3934 -0.392 0.69517

r12 0.4055 0.4629 0.876 0.38108

age2:r12 -0.6678 0.5830 -1.145 0.25203

age3:r12 0.5328 0.5430 0.981 0.32644

age4:r12 0.5914 0.5313 1.113 0.26567

age5:r12 1.6740 0.5735 2.919 0.00351 **

---

Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1


Null deviance: 94.6068 on 19 degrees of freedom

Residual deviance: 4.2581 on 10 degrees of freedom #n-p=20-1-4-1-4

AIC: 105.37




For residual checks in GLM, the mean and variance are functionally

related in interpreting the residuals plot. To help in this problem, we

generally use the scaled residuals of some form:

1. Ordinary residuals: Ri = Yi− µi but it is useful only in normal case.

2. Pearson (or standardized) residuals: Rpi =Yi − µi√a(φ)b′′(θi)

where Var(Yi) = a(φ)b′′(θi). For the Poisson case,

Rpi =Yi − µi√

µi

as Var(Yi) = E(Yi).

> r=residuals.glm(glm3,type="pearson")

> r

1 2 3 4 5

1.080123e+00 -7.715167e-01 -3.086067e-01 -2.014199e-15 -1.478018e-15

6 7 8 9 10

-5.547002e-01 5.547002e-01 -5.617334e-16 -6.666667e-01 6.666667e-01

11 12 13 14 15

-1.776357e-15 -2.222376e-15 -1.036952e-01 5.184758e-01 -4.147807e-01

16 17 18 19 20

6.713998e-16 -5.000000e-01 -1.332268e-15 5.000000e-01 -2.512148e-15

> par=glm3$coeff

> r1=(y[1]-exp(par[1]))/sqrt(exp(par[1]))

> r1 #agree with the first pearson residual

(Intercept)

1.080123

Results of the four models are summarized below:



Model Dev df Diff dev Diff df p-value Conclusion1. Saturated age*rate 0 0 - - - -2. Reduced sat. age*rate 4.2581 10 4.2581 10 0.935 Accept M23. Additive age+rate 32.457 12 32.457 12 0.001 Reject M3, favor M14. Null 94.607 19 62.15 7 0.000 Reject M4, favor M3

Note that M2 and M3 are not nested.

Reference:

1. Little, J.A. (1979) The General Linear Model and Direct Standard-

ization A Comparison. Sociological methods and research, 7, 475-

501.

2. Lambert, D. (1992) Zero-Inflated Poisson Regression, with an Ap-

plication to Defects in Manufacturing. Technometrics, 34: 1-14.

3. Mullahy, J. (1986) Specification and testing of some modified count

data models. Journal of Econometrics, 33, 341-365.



Appendix

Definition: A stochastic process N(t), t ≥ 0 is a counting process

if N(t) represents the total number of events that have occured by time

t. Note: N(t) ≥ 0. If s < t, N(s) ≤ N(t).

Definition: A stochastic process is independent increment the number

of events in disjoint intervals are independent.

Definition: A stochastic process is stationary increment if the distri-

butions of N(t1 + s)−N(t2 + s) and N(t1)−N(t2) are equal.

Lemma: The counting process N(t), t ≥ 0 is a Poisson process with

rate λ if

1. N(0) = 0

2. The process has stationary and independent increment.

3. Pr(N(h) = 1) = λh + o(h)

4. Pr(N(h) ≥ 2) = o(h)

where f (x) = o(x) if f(x)x → 0 as x→ 0 and o(h)± o(h) = o(h).

Proof: Let pn(t) = Pr(N(t) = n). We have

p0(t + h) = Pr(N(t) = 0, N(t + h)−N(t) = 0)

= Pr(N(t) = 0) Pr(N(h) = 0) = p0(t)(1− λh + o(h)) by (2-4)

Hence

p′0(t) = limh→0

p0(t + h)− p0(t)

h= lim

h→0−λp0(t) +

o(h)

hp0(t) = −λp0(t)

p′0(t)

p0(t)= −λ ⇒ ln p0(t) = −λt + c ⇒ p0(t) = ke−λt



Since p0(0) = Pr(N(0) = 0) = 1 ⇒ k = 1, we have p0(t) = e−λt. Then

assuming pn−1(t) =(λt)n−1e−λt

(n− 1)!using mathematical induction, we have

pn(t+ h) = Pr(N(t) = n,N(t+ h)−N(t) = 0)

+ Pr(N(t) = n− 1, N(t+ h)−N(t) = 1)

+n∑k=2

Pr(N(t) = n− k,N(t+ h)−N(t) = k)

= Pr(N(t) = n) Pr(N(h) = 0) + Pr(N(t) = n− 1) Pr(N(h) = 1) + o(h)

= pn(t)(1− λh+ o(h)) + pn−1(t)(λh+ o(h)) + o(h)

pn(t+ h)− pn(t)h

= −pn(t)λ+ pn(t)o(h)

h+ pn−1(t)λ+ pn−1(t)

o(h)

h+o(h)

hp′n(t) = −λpn(t) + λpn−1(t)

Implies

p′n(t) + λpn(t) = λ(λt)n−1e−λt

(n− 1)!=λntn−1e−λt

(n− 1)!

The solution to the differential equation

dy

dt+ h(t)y = g(t)

is

y =

∫e∫h(t)dtg(t)dt + c

e∫h(t)dt

.

Now h(t) = λ and g(t) =λntn−1e−λt

(n− 1)!. We have

pn(t) =

∫(e∫λdt)[λ

ntn−1e−λt

(n−1)! ]dt + c

e∫λdt

=

∫(eλt)[λ

ntn−1e−λt

(n−1)! ]dt + c

eλt

=λn∫

tn−1

(n−1)!dt + c

eλt=λn tn

n(n−1)! + c

eλt

= e−λt(

(λt)n

n!+ c

).



Since pn(0) = Pr(N(0) = n) = 0⇒ c = 0, n ≥ 1, we have

pn(t) =e−λt(λt)n

n!

which is the pmf of Poisson distribution.


Documents

MSH3 Generalized linear model Ch. 6 Count data …SydU MSH3 GLM (2015) First semester Dr. J. Chan 208 The plot of variance against mean in log scale for all cells with at least 20