53
Forecasting using R 1 Forecasting using R Rob J Hyndman 2.1 State space models

Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Forecasting using R 1

Forecasting using R

Rob J Hyndman

2.1 State space models

Page 2: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Outline

1 Innovations state space models

2 ETS in R

3 Lab session 8

Forecasting using R Innovations state space models 2

Page 3: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Methods V Models

Exponential smoothing methodsAlgorithms that return point forecasts.

Innovations state space modelsGenerate same point forecasts but can also generateforecast intervals.A stochastic (or random) data generating process thatcan generate an entire forecast distribution.Allow for “proper” model selection.

Forecasting using R Innovations state space models 3

Page 4: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Methods V Models

Exponential smoothing methodsAlgorithms that return point forecasts.

Innovations state space modelsGenerate same point forecasts but can also generateforecast intervals.A stochastic (or random) data generating process thatcan generate an entire forecast distribution.Allow for “proper” model selection.

Forecasting using R Innovations state space models 3

Page 5: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS models

Each model has an observation equation andtransition equations, one for each state (level, trend,seasonal), i.e., state space models.Two models for each method: one with additive andone with multiplicative errors, i.e., in total 18 models.ETS(Error,Trend,Seasonal):

Error = {A,M}Trend = {N,A,Ad}Seasonal = {N,A,M}.

Forecasting using R Innovations state space models 4

Page 6: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Exponential smoothing methodsSeasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

Forecasting using R Innovations state space models 5

Page 7: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Exponential smoothing methodsSeasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

There are 18 separate models in the ETS frameworkForecasting using R Innovations state space models 6

Page 8: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Exponential smoothing methodsSeasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

There are 18 separate models in the ETS frameworkForecasting using R Innovations state space models 6

Page 9: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Exponential smoothing methodsSeasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors

There are 18 separate models in the ETS frameworkForecasting using R Innovations state space models 6

Page 10: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

A model for SESComponent form

Forecast equation yt+h|t = `tSmoothing equation `t = αyt + (1− α)`t−1

Forecast error: et = yt − yt|t−1 = yt − `t−1.

Error correction formyt = `t−1 + et`t = `t−1 + α(yt − `t−1)= `t−1 + αet

Specify probability distribution for et, we assumeet = εt ∼ NID(0, σ2).

Forecasting using R Innovations state space models 7

Page 11: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

A model for SESComponent form

Forecast equation yt+h|t = `tSmoothing equation `t = αyt + (1− α)`t−1

Forecast error: et = yt − yt|t−1 = yt − `t−1.

Error correction formyt = `t−1 + et`t = `t−1 + α(yt − `t−1)= `t−1 + αet

Specify probability distribution for et, we assumeet = εt ∼ NID(0, σ2).

Forecasting using R Innovations state space models 7

Page 12: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

A model for SESComponent form

Forecast equation yt+h|t = `tSmoothing equation `t = αyt + (1− α)`t−1

Forecast error: et = yt − yt|t−1 = yt − `t−1.

Error correction formyt = `t−1 + et`t = `t−1 + α(yt − `t−1)= `t−1 + αet

Specify probability distribution for et, we assumeet = εt ∼ NID(0, σ2).

Forecasting using R Innovations state space models 7

Page 13: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

A model for SESComponent form

Forecast equation yt+h|t = `tSmoothing equation `t = αyt + (1− α)`t−1

Forecast error: et = yt − yt|t−1 = yt − `t−1.

Error correction formyt = `t−1 + et`t = `t−1 + α(yt − `t−1)= `t−1 + αet

Specify probability distribution for et, we assumeet = εt ∼ NID(0, σ2).

Forecasting using R Innovations state space models 7

Page 14: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS(A,N,N)

Measurement equation yt = `t−1 + εtState equation `t = `t−1 + αεt

where εt ∼ NID(0, σ2).

“innovations” or “single source of error” becausesame error process, εt.Measurement equation: relationship betweenobservations and states.Transition equation(s): evolution of the state(s)through time.

Forecasting using R Innovations state space models 8

Page 15: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS(M,N,N)SES with multiplicative errors.

Specify relative errors εt =yt−yt|t−1yt|t−1

∼ NID(0, σ2)Substituting yt|t−1 = `t−1 gives:

yt = `t−1 + `t−1εtet = yt − yt|t−1 = `t−1εt

Measurement equation yt = `t−1(1 + εt)State equation `t = `t−1(1 + αεt)

Models with additive and multiplicative errors withthe same parameters generate the same pointforecasts but different prediction intervals.

Forecasting using R Innovations state space models 9

Page 16: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS(M,N,N)SES with multiplicative errors.

Specify relative errors εt =yt−yt|t−1yt|t−1

∼ NID(0, σ2)Substituting yt|t−1 = `t−1 gives:

yt = `t−1 + `t−1εtet = yt − yt|t−1 = `t−1εt

Measurement equation yt = `t−1(1 + εt)State equation `t = `t−1(1 + αεt)

Models with additive and multiplicative errors withthe same parameters generate the same pointforecasts but different prediction intervals.

Forecasting using R Innovations state space models 9

Page 17: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS(M,N,N)SES with multiplicative errors.

Specify relative errors εt =yt−yt|t−1yt|t−1

∼ NID(0, σ2)Substituting yt|t−1 = `t−1 gives:

yt = `t−1 + `t−1εtet = yt − yt|t−1 = `t−1εt

Measurement equation yt = `t−1(1 + εt)State equation `t = `t−1(1 + αεt)

Models with additive and multiplicative errors withthe same parameters generate the same pointforecasts but different prediction intervals.

Forecasting using R Innovations state space models 9

Page 18: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS(A,A,N)

Holt’s linear method with additive errors.

Assume εt = yt − `t−1 − bt−1 ∼ NID(0, σ2).Substituting into the error correction equations forHolt’s linear method

yt = `t−1 + bt−1 + εt`t = `t−1 + bt−1 + αεtbt = bt−1 + αβ∗εt

For simplicity, set β = αβ∗.

Forecasting using R Innovations state space models 10

Page 19: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS(M,A,N)

Holt’s linear method with multiplicative errors.

Assume εt = yt−(`t−1+bt−1)(`t−1+bt−1)

Following a similar approach as above, theinnovations state space model underlying Holt’s linearmethod with multiplicative errors is specified as

yt = (`t−1 + bt−1)(1 + εt)`t = (`t−1 + bt−1)(1 + αεt)bt = bt−1 + β(`t−1 + bt−1)εt

where again β = αβ∗ and εt ∼ NID(0, σ2).

Forecasting using R Innovations state space models 11

Page 20: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ETS(A,A,A)

Holt-Winters additive method with additive errors.

Forecast equation yt+h|t = `t + hbt + st−m+h+mObservation equation yt = `t−1 + bt−1 + st−m + εt

State equations `t = `t−1 + bt−1 + αεtbt = bt−1 + βεtst = st−m + γεt

Forecast errors: εt = yt − yt|t−1h+m = b(h− 1) mod mc + 1.

Forecasting using R Innovations state space models 12

Page 21: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Additive error models

Forecasting using R Innovations state space models 13

7/ exponential smoothing 149

ADDITIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1 + εt yt = `t−1 + st−m + εt yt = `t−1st−m + εt`t = `t−1 +αεt `t = `t−1 +αεt `t = `t−1 +αεt/st−m

st = st−m +γεt st = st−m +γεt/`t−1

yt = `t−1 + bt−1 + εt yt = `t−1 + bt−1 + st−m + εt yt = (`t−1 + bt−1)st−m + εtA `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt/st−m

bt = bt−1 + βεt bt = bt−1 + βεt bt = bt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 + bt−1)

yt = `t−1 +φbt−1 + εt yt = `t−1 +φbt−1 + st−m + εt yt = (`t−1 +φbt−1)st−m + εtAd `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt/st−m

bt = φbt−1 + βεt bt = φbt−1 + βεt bt = φbt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 +φbt−1)

yt = `t−1bt−1 + εt yt = `t−1bt−1 + st−m + εt yt = `t−1bt−1st−m + εtM `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt/st−m

bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/(st−m`t−1)st = st−m +γεt st = st−m +γεt/(`t−1bt−1)

yt = `t−1bφt−1 + εt yt = `t−1b

φt−1 + st−m + εt yt = `t−1b

φt−1st−m + εt

Md `t = `t−1bφt−1 +αεt `t = `t−1b

φt−1 +αεt `t = `t−1b

φt−1 +αεt/st−m

bt = bφt−1 + βεt/`t−1 bt = b

φt−1 + βεt/`t−1 bt = b

φt−1 + βεt/(st−m`t−1)

st = st−m +γεt st = st−m +γεt/(`t−1bφt−1)

MULTIPLICATIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1(1 + εt) yt = (`t−1 + st−m)(1 + εt) yt = `t−1st−m(1 + εt)`t = `t−1(1 +αεt) `t = `t−1 +α(`t−1 + st−m)εt `t = `t−1(1 +αεt)

st = st−m +γ(`t−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 + bt−1)(1 + εt) yt = (`t−1 + bt−1 + st−m)(1 + εt) yt = (`t−1 + bt−1)st−m(1 + εt)A `t = (`t−1 + bt−1)(1 +αεt) `t = `t−1 + bt−1 +α(`t−1 + bt−1 + st−m)εt `t = (`t−1 + bt−1)(1 +αεt)

bt = bt−1 + β(`t−1 + bt−1)εt bt = bt−1 + β(`t−1 + bt−1 + st−m)εt bt = bt−1 + β(`t−1 + bt−1)εtst = st−m +γ(`t−1 + bt−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 +φbt−1)(1 + εt) yt = (`t−1 +φbt−1 + st−m)(1 + εt) yt = (`t−1 +φbt−1)st−m(1 + εt)Ad `t = (`t−1 +φbt−1)(1 +αεt) `t = `t−1 +φbt−1 +α(`t−1 +φbt−1 + st−m)εt `t = (`t−1 +φbt−1)(1 +αεt)

bt = φbt−1 + β(`t−1 +φbt−1)εt bt = φbt−1 + β(`t−1 +φbt−1 + st−m)εt bt = φbt−1 + β(`t−1 +φbt−1)εtst = st−m +γ(`t−1 +φbt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bt−1(1 + εt) yt = (`t−1bt−1 + st−m)(1 + εt) yt = `t−1bt−1st−m(1 + εt)M `t = `t−1bt−1(1 +αεt) `t = `t−1bt−1 +α(`t−1bt−1 + st−m)εt `t = `t−1bt−1(1 +αεt)

bt = bt−1(1 + βεt) bt = bt−1 + β(`t−1bt−1 + st−m)εt/`t−1 bt = bt−1(1 + βεt)st = st−m +γ(`t−1bt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bφt−1(1 + εt) yt = (`t−1b

φt−1 + st−m)(1 + εt) yt = `t−1b

φt−1st−m(1 + εt)

Md `t = `t−1bφt−1(1 +αεt) `t = `t−1b

φt−1 +α(`t−1b

φt−1 + st−m)εt `t = `t−1b

φt−1(1 +αεt)

bt = bφt−1(1 + βεt) bt = b

φt−1 + β(`t−1b

φt−1 + st−m)εt/`t−1 bt = b

φt−1(1 + βεt)

st = st−m +γ(`t−1bφt−1 + st−m)εt st = st−m(1 +γεt)

Table 7.10: State space equationsfor each of the models in the ETSframework.

Page 22: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Multiplicative error models

Forecasting using R Innovations state space models 14

7/ exponential smoothing 149

ADDITIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1 + εt yt = `t−1 + st−m + εt yt = `t−1st−m + εt`t = `t−1 +αεt `t = `t−1 +αεt `t = `t−1 +αεt/st−m

st = st−m +γεt st = st−m +γεt/`t−1

yt = `t−1 + bt−1 + εt yt = `t−1 + bt−1 + st−m + εt yt = (`t−1 + bt−1)st−m + εtA `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt `t = `t−1 + bt−1 +αεt/st−m

bt = bt−1 + βεt bt = bt−1 + βεt bt = bt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 + bt−1)

yt = `t−1 +φbt−1 + εt yt = `t−1 +φbt−1 + st−m + εt yt = (`t−1 +φbt−1)st−m + εtAd `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt `t = `t−1 +φbt−1 +αεt/st−m

bt = φbt−1 + βεt bt = φbt−1 + βεt bt = φbt−1 + βεt/st−mst = st−m +γεt st = st−m +γεt/(`t−1 +φbt−1)

yt = `t−1bt−1 + εt yt = `t−1bt−1 + st−m + εt yt = `t−1bt−1st−m + εtM `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt `t = `t−1bt−1 +αεt/st−m

bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/`t−1 bt = bt−1 + βεt/(st−m`t−1)st = st−m +γεt st = st−m +γεt/(`t−1bt−1)

yt = `t−1bφt−1 + εt yt = `t−1b

φt−1 + st−m + εt yt = `t−1b

φt−1st−m + εt

Md `t = `t−1bφt−1 +αεt `t = `t−1b

φt−1 +αεt `t = `t−1b

φt−1 +αεt/st−m

bt = bφt−1 + βεt/`t−1 bt = b

φt−1 + βεt/`t−1 bt = b

φt−1 + βεt/(st−m`t−1)

st = st−m +γεt st = st−m +γεt/(`t−1bφt−1)

MULTIPLICATIVE ERROR MODELS

Trend SeasonalN A M

N yt = `t−1(1 + εt) yt = (`t−1 + st−m)(1 + εt) yt = `t−1st−m(1 + εt)`t = `t−1(1 +αεt) `t = `t−1 +α(`t−1 + st−m)εt `t = `t−1(1 +αεt)

st = st−m +γ(`t−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 + bt−1)(1 + εt) yt = (`t−1 + bt−1 + st−m)(1 + εt) yt = (`t−1 + bt−1)st−m(1 + εt)A `t = (`t−1 + bt−1)(1 +αεt) `t = `t−1 + bt−1 +α(`t−1 + bt−1 + st−m)εt `t = (`t−1 + bt−1)(1 +αεt)

bt = bt−1 + β(`t−1 + bt−1)εt bt = bt−1 + β(`t−1 + bt−1 + st−m)εt bt = bt−1 + β(`t−1 + bt−1)εtst = st−m +γ(`t−1 + bt−1 + st−m)εt st = st−m(1 +γεt)

yt = (`t−1 +φbt−1)(1 + εt) yt = (`t−1 +φbt−1 + st−m)(1 + εt) yt = (`t−1 +φbt−1)st−m(1 + εt)Ad `t = (`t−1 +φbt−1)(1 +αεt) `t = `t−1 +φbt−1 +α(`t−1 +φbt−1 + st−m)εt `t = (`t−1 +φbt−1)(1 +αεt)

bt = φbt−1 + β(`t−1 +φbt−1)εt bt = φbt−1 + β(`t−1 +φbt−1 + st−m)εt bt = φbt−1 + β(`t−1 +φbt−1)εtst = st−m +γ(`t−1 +φbt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bt−1(1 + εt) yt = (`t−1bt−1 + st−m)(1 + εt) yt = `t−1bt−1st−m(1 + εt)M `t = `t−1bt−1(1 +αεt) `t = `t−1bt−1 +α(`t−1bt−1 + st−m)εt `t = `t−1bt−1(1 +αεt)

bt = bt−1(1 + βεt) bt = bt−1 + β(`t−1bt−1 + st−m)εt/`t−1 bt = bt−1(1 + βεt)st = st−m +γ(`t−1bt−1 + st−m)εt st = st−m(1 +γεt)

yt = `t−1bφt−1(1 + εt) yt = (`t−1b

φt−1 + st−m)(1 + εt) yt = `t−1b

φt−1st−m(1 + εt)

Md `t = `t−1bφt−1(1 +αεt) `t = `t−1b

φt−1 +α(`t−1b

φt−1 + st−m)εt `t = `t−1b

φt−1(1 +αεt)

bt = bφt−1(1 + βεt) bt = b

φt−1 + β(`t−1b

φt−1 + st−m)εt/`t−1 bt = b

φt−1(1 + βεt)

st = st−m +γ(`t−1bφt−1 + st−m)εt st = st−m(1 +γεt)

Table 7.10: State space equationsfor each of the models in the ETSframework.

Page 23: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Estimating ETS models

Smoothing parameters α, β, γ and φ, and the initialstates `0, b0, s0, s−1, . . . , s−m+1 are estimated bymaximising the “likelihood” = the probability of thedata arising from the specified model.For models with additive errors equivalent tominimising SSE.For models with multiplicative errors, not equivalentto minimising SSE.We will estimate models with the ets() function inthe forecast package.

Forecasting using R Innovations state space models 15

Page 24: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Innovations state space models

Let xt = (`t, bt, st, st−1, . . . , st−m+1) and εtiid

∼ N(0, σ2).

yt = h(xt−1)︸ ︷︷ ︸ + k(xt−1)εt︸ ︷︷ ︸µt et

xt = f(xt−1) + g(xt−1)εt

Additive errorsk(x) = 1. yt = µt + εt.

Multiplicative errorsk(xt−1) = µt. yt = µt(1 + εt).εt = (yt − µt)/µt is relative error.

Forecasting using R Innovations state space models 16

Page 25: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Innovations state space models

Estimation

L∗(θ, x0) = n log( n∑

t=1ε2t /k

2(xt−1))+ 2

n∑t=1

log |k(xt−1)|

= −2 log(Likelihood) + constant

Estimate parameters θ = (α, β, γ, φ) and initial statesx0 = (`0, b0, s0, s−1, . . . , s−m+1) by minimizing L∗.

Forecasting using R Innovations state space models 17

Page 26: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Parameter restrictionsUsual region

Traditional restrictions in the methods 0 < α, β∗, γ∗, φ < 1(equations interpreted as weighted averages).In models we set β = αβ∗ and γ = (1− α)γ∗.Therefore 0 < α < 1, 0 < β < α and 0 < γ < 1− α.0.8 < φ < 0.98 — to prevent numerical difficulties.

Admissible region

To prevent observations in the distant past having acontinuing effect on current forecasts.Usually (but not always) less restrictive than the usualregion.For example for ETS(A,N,N):usual 0 < α < 1 — admissible is 0 < α < 2.

Forecasting using R Innovations state space models 18

Page 27: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Parameter restrictionsUsual region

Traditional restrictions in the methods 0 < α, β∗, γ∗, φ < 1(equations interpreted as weighted averages).In models we set β = αβ∗ and γ = (1− α)γ∗.Therefore 0 < α < 1, 0 < β < α and 0 < γ < 1− α.0.8 < φ < 0.98 — to prevent numerical difficulties.

Admissible region

To prevent observations in the distant past having acontinuing effect on current forecasts.Usually (but not always) less restrictive than the usualregion.For example for ETS(A,N,N):usual 0 < α < 1 — admissible is 0 < α < 2.

Forecasting using R Innovations state space models 18

Page 28: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Model selectionAkaike’s Information Criterion

AIC = −2 log(L) + 2k

where L is the likelihood and k is the number ofparameters initial states estimated in the model.Corrected AIC

AICc = AIC +2(k + 1)(k + 2)

T − k

which is the AIC corrected (for small sample bias).Bayesian Information Criterion

BIC = AIC + k(log(T)− 2).Forecasting using R Innovations state space models 19

Page 29: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Model selectionAkaike’s Information Criterion

AIC = −2 log(L) + 2k

where L is the likelihood and k is the number ofparameters initial states estimated in the model.Corrected AIC

AICc = AIC +2(k + 1)(k + 2)

T − k

which is the AIC corrected (for small sample bias).Bayesian Information Criterion

BIC = AIC + k(log(T)− 2).Forecasting using R Innovations state space models 19

Page 30: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Model selectionAkaike’s Information Criterion

AIC = −2 log(L) + 2k

where L is the likelihood and k is the number ofparameters initial states estimated in the model.Corrected AIC

AICc = AIC +2(k + 1)(k + 2)

T − k

which is the AIC corrected (for small sample bias).Bayesian Information Criterion

BIC = AIC + k(log(T)− 2).Forecasting using R Innovations state space models 19

Page 31: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Automatic forecasting

From Hyndman et al. (IJF, 2002):

Apply each model that is appropriate to the data.Optimize parameters and initial values using MLE (orsome other criterion).Select best method using AICc:Produce forecasts using best method.Obtain Forecast intervals using underlying state spacemodel.

Method performed very well in M3 competition.

Forecasting using R Innovations state space models 20

Page 32: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Some unstable models

Some of the combinations of (Error, Trend, Seasonal)can lead to numerical difficulties; see equations withdivision by a state.These are: ETS(A,N,M), ETS(A,A,M), ETS(A,Ad,M).Models with multiplicative errors are useful forstrictly positive data, but are not numerically stablewith data containing zeros or negative values. In thatcase only the six fully additive models will be applied.

Forecasting using R Innovations state space models 21

Page 33: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Exponential smoothing modelsAdditive Error Seasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) A,N,N A,N,A A,N,M

A (Additive) A,A,N A,A,A A,A,M

Ad (Additive damped) A,Ad,N A,Ad,A A,Ad,M

Multiplicative Error Seasonal ComponentTrend N A M

Component (None) (Additive) (Multiplicative)

N (None) M,N,N M,N,A M,N,M

A (Additive) M,A,N M,A,A M,A,M

Ad (Additive damped) M,Ad,N M,Ad,A M,Ad,MForecasting using R Innovations state space models 22

Page 34: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Forecasting with ETS models

Point forecasts: iterate the equations fort = T + 1, T + 2, . . . , T + h and set all εt = 0 for t > T.

Not the same as E(yt+h|xt) unless trend andseasonality are both additive.Point forecasts for ETS(A,x,y) are identical toETS(M,x,y) if the parameters are the same.

Forecasting using R Innovations state space models 23

Page 35: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Forecasting with ETS models

Point forecasts: iterate the equations fort = T + 1, T + 2, . . . , T + h and set all εt = 0 for t > T.

Not the same as E(yt+h|xt) unless trend andseasonality are both additive.Point forecasts for ETS(A,x,y) are identical toETS(M,x,y) if the parameters are the same.

Forecasting using R Innovations state space models 23

Page 36: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Forecasting with ETS models

Prediction intervals: cannot be generated using themethods.

The prediction intervals will differ between modelswith additive and multiplicative methods.Exact formulae for some models.More general to simulate future sample paths,conditional on the last estimate of the states, and toobtain prediction intervals from the percentiles ofthese simulated future paths.Options are available in R using the forecastfunction in the forecast package.

Forecasting using R Innovations state space models 24

Page 37: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Outline

1 Innovations state space models

2 ETS in R

3 Lab session 8

Forecasting using R ETS in R 25

Page 38: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Example: drug sales(fit <- ets(h02))

## ETS(M,Ad,M)#### Call:## ets(y = h02)#### Smoothing parameters:## alpha = 0.2173## beta = 2e-04## gamma = 1e-04## phi = 0.9756#### Initial states:## l = 0.3996## b = 0.0098## s=0.8675 0.8259 0.7591 0.7748 0.6945 1.2838## 1.3366 1.1753 1.1545 1.0968 1.0482 0.983#### sigma: 0.0647#### AIC AICc BIC## -123.21905 -119.52175 -63.49289

Forecasting using R ETS in R 26

Page 39: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Example: drug sales(fit2 <- ets(h02, model="AAA", damped=FALSE))

## ETS(A,A,A)#### Call:## ets(y = h02, model = "AAA", damped = FALSE)#### Smoothing parameters:## alpha = 0.1957## beta = 1e-04## gamma = 0.4211#### Initial states:## l = 0.4146## b = 0.0026## s=-0.1064 -0.1028 -0.1211 -0.1086 -0.161 0.2173## 0.2306 0.0671 0.0667 0.0299 -0.0156 0.0038#### sigma: 0.0538#### AIC AICc BIC## -73.33048 -70.04016 -16.92244

Forecasting using R ETS in R 27

Page 40: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The ets() function

Automatically chooses a model by default using theAIC, AICc or BIC.Can handle any combination of trend, seasonality anddampingProduces prediction intervals for every modelEnsures the parameters are admissible (equivalent toinvertible)Produces an object of class “ets”.

Forecasting using R ETS in R 28

Page 41: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

ets objects

Methods: coef(), autoplot(), plot(),summary(), residuals(), fitted(), simulate()and forecast()autoplot() and plot() functions show time plotsof the original time series along with the extractedcomponents (level, growth and seasonal).

Forecasting using R ETS in R 29

Page 42: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Example: drug sales

fit %>% autoplot

0.50

0.75

1.00

1.25

0.4

0.6

0.8

1.0

0.0000

0.0025

0.0050

0.0075

0.0100

0.8

1.0

1.2

obse

rved

leve

lsl

ope

seas

on

1995 2000 2005

Decomposition by ETS(M,Ad,M) method

Forecasting using R ETS in R 30

Page 43: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Example: drug sales

fit %>% forecast %>% autoplot

0.3

0.6

0.9

1.2

1.5

1995 2000 2005 2010Time

y

level

80

95

Forecasts from ETS(M,Ad,M)

Forecasting using R ETS in R 31

Page 44: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Example: drug sales

accuracy(fit)

## ME RMSE MAE MPE MAPE## Training set 0.003353397 0.05144236 0.03919035 0.0437546 5.054949## MASE ACF1## Training set 0.6465103 -0.007295816

accuracy(fit2)

## ME RMSE MAE MPE MAPE## Training set 0.000396129 0.05381921 0.04038841 -0.3530507 5.375484## MASE ACF1## Training set 0.6662743 -0.06641172

Forecasting using R ETS in R 32

Page 45: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The ets() functionets() function also allows refitting model to new data set.

train <- window(h02, end=c(2004,12))test <- window(h02, start=2005)fit1 <- ets(train)fit2 <- ets(test, model = fit1)accuracy(fit2)

## ME RMSE MAE MPE MAPE## Training set 0.002828719 0.0556371 0.04510416 -0.3111965 5.424523## MASE ACF1## Training set 0.7094596 -0.3807942

accuracy(forecast(fit1,10), test)

## ME RMSE MAE MPE MAPE## Training set 0.003385115 0.04466096 0.03278583 0.169324 4.332804## Test set -0.079220419 0.09420250 0.08237909 -10.353804 10.642337## MASE ACF1 Theil's U## Training set 0.5560485 -0.01039592 NA## Test set 1.3971516 0.01728834 0.6543488

Forecasting using R ETS in R 33

Page 46: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The ets() function in R

ets(y, model = "ZZZ", damped = NULL,additive.only = FALSE,lambda = NULL, biasadj = FALSE,lower = c(rep(1e-04, 3), 0.8),upper = c(rep(0.9999, 3), 0.98),opt.crit = c("lik", "amse", "mse", "sigma", "mae"),nmse = 3,bounds = c("both", "usual", "admissible"),ic = c("aicc", "aic", "bic"),restrict = TRUE,allow.multiplicative.trend = FALSE, ...)

Forecasting using R ETS in R 34

Page 47: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The ets() function in R

yThe time series to be forecast.modeluse the ETS classification and notation: “N” for none, “A”for additive, “M” for multiplicative, or “Z” for automaticselection. Default ZZZ all components are selected usingthe information criterion.dampedIf damped=TRUE, then a damped trend will be used (eitherAd or Md).damped=FALSE, then a non-damped trend will used.If damped=NULL (the default), then either a damped or anon-damped trend will be selected according to theinformation criterion chosen.

Forecasting using R ETS in R 35

Page 48: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The ets() function in R

additive.onlyOnly models with additive components will be consideredif additive.only=TRUE. Otherwise all models will beconsidered.lambdaBox-Cox transformation parameter. It will be ignored iflambda=NULL (the default value). Otherwise, the timeseries will be transformed before the model is estimated.When lambda is not NULL, additive.only is set to TRUE.biadadjUses bias-adjustment when undoing Box-Coxtransformation for fitted values.

Forecasting using R ETS in R 36

Page 49: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The ets() function in Rlower,upper bounds for the parameter estimates of α,β∗, γ∗ and φ.opt.crit=lik (default) optimisation criterion used forestimation.bounds Constraints on the parameters.

usual region – "bounds=usual";admissible region – "bounds=admissible";"bounds=both" (the default) requires the parameters tosatisfy both sets of constraints.

ic=aicc (the default) information criterion to be used inselecting models.restrict=TRUE (the default) models that cause numericaldifficulties are not considered in model selection.allow.multiplicative.trend allows models with amultiplicative trend.

Forecasting using R ETS in R 37

Page 50: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The forecast() function in R

forecast(object,h=ifelse(object$m>1, 2*object$m, 10),level=c(80,95), fan=FALSE,simulate=FALSE, bootstrap=FALSE,npaths=5000, PI=TRUE,lambda=object$lambda, biasadj=FALSE,...)

object: the object returned by the ets() function.h: the number of periods to be forecast.level: the confidence level for the prediction intervals.fan: if fan=TRUE, suitable for fan plots.simulate: If TRUE, prediction intervals generated viasimulation rather than analytic formulae. Even if FALSEsimulation will be used if no algebraic formulae exist.

Forecasting using R ETS in R 38

Page 51: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

The forecast() function in R

bootstrap: If bootstrap=TRUE and simulate=TRUE,then simulated prediction intervals use re-sampled errorsrather than normally distributed errors.npaths: The number of sample paths used in computingsimulated prediction intervals.PI: If PI=TRUE, then prediction intervals are produced;otherwise only point forecasts are calculated. If PI=FALSE,then level, fan, simulate, bootstrap and npaths areall ignored.lambda: The Box-Cox transformation parameter. Ignored iflambda=NULL. Otherwise, forecasts are back-transformedvia inverse Box-Cox transformation.biasadj: Apply bias adjustment after Box-Cox?

Forecasting using R ETS in R 39

Page 52: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Outline

1 Innovations state space models

2 ETS in R

3 Lab session 8

Forecasting using R Lab session 8 40

Page 53: Forecasting using R - Rob J. Hyndman · A,N,N: Simple exponential smoothing with additive errors A,A,N: Holt’s linear method with additive errors M,A,M: Multiplicative Holt-Winters’

Lab Session 8

Forecasting using R Lab session 8 41