An Ecological Trap for Ecologists: Zero-Modified Models Western Mensurationists’ Meeting 2009 Tzeng Yih Lam 06.23.2009 Tzeng Yih Lam, OSU Manuela Huso,

An Ecological Trap for Ecologists:

Zero-Modified Models

Western Mensurationists’ Meeting 2009 Tzeng Yih Lam 06.23.2009

Tzeng Yih Lam, OSU

Manuela Huso, OSU

Doug Maguire, OSU

^Possible

Ecological Trap [ē-kə-ˈlä-ji-kəl ˈtrap]

A preference of falsely attractive habitat and a general avoidance of high-quality but less-attractive habitats.


Wikipedia

• Cues for Zero-Modified Models• Possible Trap #1• Possible Trap #2• Discussions


Cues for Using Zero-Modified ModelsCues for Zero-Modified Models

Possible Trap #1Possible Trap #2

DiscussionsDiscussions

Count & Normal TheoryThe Cues & The SolutionsZero-Inflated ModelsHurdle ModelsExpected Count – Observed CountConclusions

• ‘The Cues’:•For rare species data, the marginal count

frequency distribution contains large number of zeros,

•Poisson and/or NB GLM have poor fit.

• ‘The Solutions’:•Zero-modified Models: A general class of

finite mixture models that account for excessive zeros,

•Zero-Inflated Models (ZI)1,•Hurdle Models (H)2.


1Lambert (1992); 2Mullahy (1986)





Zero-Inflated Models (Poisson; ZIP)


Lambert (1992)

1 , 0

Pr1 , 0

!

y

p p e y

Y y ep y

y

Two States: Perfect and Imperfect States, Finite Mixture Model (FMM) with 1 latent structure:

• An observation belongs to either state.

Specify it as Zero-Inflated Negative Binomial (ZINB).

log λ Bβ logit log1

pp Gγ

pProbability of Belonging to Perfect State

1 , 0

Pr, 0

1 !

y

y

Y y ey

e y





Hurdle Models (Poisson; HPOIS)


Mullahy (1986) 2Baughman

(2007)

Under FMM framework comparable to ZI models, Hurdle Models with 2 latent structures2:

• An observation either cross the ‘hurdle’ or not,• All observations are in the Imperfect State.

Specify it as Hurdle Negative Binomial (HNB).

log λ Bβ logit log1

π

π Gγπ

Probability of Crossing the Hurdle




Count & Normal TheoryThe QuestionsThe Simulation StudyThe Bias & AICcOther Preliminary Key FindingsSome Plausible Explanations

Given known data generating process (dgp):

(1) Is there any bias when the data is fitted to different ZI and H model specifications?

(2) Is/Are there any universally best fit models?





Count & Normal TheoryThe QuestionsThe Simulation StudyThe Bias & AICcOther Preliminary Key FindingsSome Plausible Explanation

4 Factors(1) LAMBDA (λ): 0.3 1.5 5.0(2) INFLA (p): 0.0 0.25 0.75 (3) RATIO (Var/Mean): 1.0 1.5 3.0(4) SAMPLE : 25 50 75 100

250


For each of the 27 dgp (LAMBDA × INFLA × RATIO), generate 1000 sets of SAMPLE random count,

Fit each set to six model specifications: POIS, NB, ZIP, ZINB, HPOIS, HNB,

Calculate mean %RBIAS for each parameter: λ, p, π and compute AICc.


SAMPLE = 100


SAMPLE = 100 Bias at LAMBDA = 0.3


SAMPLE = 100 Bias with ZIP & HPOIS


SAMPLE = 100 Bias with ZINB & HNB


POISNB

NBHNB

NBZINBHNB

ZIPZINB

HPOISHNB

ZIPZINB

HPOISHNB

NBZINBHNB

ZIPZINB

HPOISHNB

ZIPZINB

HPOISHNB

NBZIP

ZINBHPOISHNB

SAMPLE = 100 Lowest AICc





(1) Variance of estimated λ is the highest when LAMBDA = 0.3,

(2) Variance of estimated λ decreases with increasing LAMBDA but it increases with increasing RATIO and/or INFLA,

(3) Probability in Perfect State, p, from ZI models has largest (+ve and –ve) bias and variance at LAMBDA = 0.3,

(4) Overdispersion parameter, θ, requires ≥ 250 SAMPLE to achieve negligible bias.






Maximum Likelihood Theory

• Ingredient = a simulated set of count• Optimize the parameter estimates to match the

marginal count distribution.

• Large bias and variance of λ and p at LAMBDA = 0.3,

• When there are either too many zeros or ones, Binomial GLM seems to be unstable Min and Agresti (2005) unstable estimates of p unstable estimates of λ and θ.

• There might not be enough information when LAMBDA = 0.3.

• ZI models estimation are based on EM algorithm,

• H models separately maximize the likelihood functions of π and λ.





Count & Normal TheoryPerfect & Imperfect StatesPerfect & Imperfect StatesA Priori KnowledgeRarityConclusions


• Perfect State in Ecology Context:• It is a set of habitat conditions that do

not host the interested species,

• Imperfect State in Ecology Context:

• It is a set of habitat conditions that host the interested species but one may not find the species there,

• This does not directly differentiate sink & source, saturated & unsaturated habitat, fundamental & realized niche etc.

ZeroStructuralRandomAccidentalStochasticSamplingTrueFalse

A: “Did you smoke any cigarette last week?”B: “No” ; 0 A: “Are you a smoker?”B: “Yes”






• Zero-Inflated Models have 2 states:• Perfect and Imperfect States• Main Assumption: You do not know the

observation belong to which state.

• Hurdle models have 1 state:• Imperfect State

Zero-Inflated Models

Hurdle Models






A priori knowledge such as species habitat range, will likely influence the model choice

In ecology, scale matters …

Grains

Exte

nt

POISNBZIP

ZINBHPOISHNB






Modeling of rare species habitat association

Cunningham & Lindenmayer (2005)

and many others…




Count & Normal TheoryPerfect & Imperfect StatesPerfect & Imperfect StatesA Priori KnowledgeRarity, Extent & GrainsConclusions


What Do the Ecologists Need To Do?






The Great Escape (1963)

Capt. Hilts (The Cooler King)1961 British 650cc Triumphs






• Escape from defining the types of zeros:• There is no restriction on threshold for mixing

& hurdle,• Change current threshold from 0 1,• Change perfect to near-perfect state (ZI

models), changing ecological implication of the models.

• N-mixture models (Royle 2004)

• Escape from using ZI & H models :• If one is uncomfortable with two-states

processes,• Small Area Estimation Rao(2003),• Extreme Value Model Coles(2001).


AcknowledgementPossible Trap #2


Count & Normal TheoryPerfect & Imperfect StatesAcknowledgementA Priori KnowledgeRarity, Extent & GrainsConclusions


Hayes Family Foundation Funds for Silviculture Alternatives

Dilworth Awards, OSU

Doug Maguire


AcknowledgementPossible Trap #2


Count & Normal TheoryPerfect & Imperfect StatesAcknowledgementA Priori KnowledgeRarity, Extent & GrainsConclusions


Thank You for Listening!

Any *Err… Question?





• AICc, Information theory.• More flexible model parameterization will have better fit.

• Sample size is an issue for ZINB and HNB models for fitting.












A priori knowledge such as species habitat range, and extent and grains will likely influence the model choice.

Documents

An Ecological Trap for Ecologists: Zero-Modified Models Western Mensurationists’ Meeting 2009 Tzeng Yih Lam 06.23.2009 Tzeng Yih Lam, OSU Manuela Huso,