STAT 497 LECTURE NOTES 7 FORECASTING 1. One of the most important objectives in time series analysis is to forecast its future values. It is the primary

STAT 497LECTURE NOTES 7

FORECASTING

1

FORECASTING• One of the most important objectives in time

series analysis is to forecast its future values. It is the primary objective of modeling.

• ESTIMATION (tahmin)the value of an estimator for a parameter.

• PREDICTION (kestirim)the value of a r.v. when we use the estimates of the parameter.

• FORECASTING (öngörü)the value of a future r.v. that is not observed by the sample.

2

FORECASTING

(Forecast) Y

n)(Predictio YY

) of (Estimate

aYY

1t

tt

ttt

ˆ

ˆˆ

ˆ

1

1

3

FORECASTING FROM AN ARMA MODELTHE MINIMUM MEAN SQUARED ERROR FORECASTS

Observed time series, Y1, Y2,…,Yn.

n: the forecast origin

4

Y1 Y2 Yn………………….. Yn+1? Yn+2?

Observed sample

n

n

nn

2nn

1nn

Y of forecast MSEminimum

Y of forecast ahead step

Y of value forecast theY



ˆ

2ˆ

1ˆ

FORECASTING FROM AN ARMA MODEL

5

sampleobserved the given

Y of nexpectatio lconditiona The

YYYYEY

n

nnnn

11 ,,,ˆ


• The stationary ARMA model for Yt is

or

• Assume that we have data Y1, Y2, . . . , Yn and we want to forecast Yn+l (i.e., l steps ahead from forecast origin n). Then the actual value is

tqtp aBYB 0

qtqttptptt aaaYYY 11110

qnqnnpnpnn aaaYYY 11110

6


• Considering the Random Shock Form of the series

7

nnnn

tp

qtn

aaaa

aB

BaBY

22110

00


• Taking the expectation of Yn+l , we have

where

8

11

11 ,,,ˆ

nn

nnnn

aa

YYYYEY

0,

0,0,, 1 ja

jYYaE

jnnjn

FORECASTING FROM AN ARMA MODEL• The forecast error:

9

1

0

1111

ˆ

iini

nnn

nnn

a

aaa

YYe

• The expectation of the forecast error: 0neE

• So, the forecast in unbiased.• The variance of the forecast error:

1

0

221

0

i

iai

inin aVareVar

FORECASTING FROM AN ARMA MODEL• One step-ahead (l=1)

10

2

11

1210

121101

1

1ˆ1

1ˆ

an

nnnn

nnn

nnnn

eVar

aYYe

aaY

aaaY


• Two step-ahead (l=2)

11

2

1

2

1122

20

211202

12

2ˆ2

2ˆ

an

nnnnn

nn

nnnn

eVar

aaYYe

aY

aaaY


• Note that,

• That’s why ARMA (or ARIMA) forecasting is useful only for short-term forecasting.

12

0lim

0ˆlim

n

n

eVar

Y

PREDICTION INTERVAL FOR Yn+l

• A 95% prediction interval for Yn+l (l steps ahead) is

13

1

0

296.1ˆ

96.1ˆ

iian

nn

Y

eVarY

For one step-ahead the simplifies to anY 96.11ˆ

For two step-ahead the simplifies to 2

1196.12ˆ anY • When computing prediction intervals from data, we substitute estimates for parameters, giving approximate prediction intervals

REASONS NEEDING A LONG REALIZATION

• Estimate correlation structure (i.e., the ACF and PACF) functions and get accurate standard errors.

• Estimate seasonal pattern (need at least 4 or 5 seasonal periods).

• Approximate prediction intervals assume that parameters are known (good approximation if realization is large).

• Fewer estimation problems (likelihood function better behaved).

• Possible to check forecasts by withholding recent data .• Can check model stability by dividing data and

analyzing both sides.

14

REASONS FOR USING A PARSIMONIOUS MODEL

• Fewer numerical problems in estimation.• Easier to understand the model.• With fewer parameters, forecasts less sensitive

to deviations between parameters and estimates.• Model may applied more generally to similar

processes.• Rapid real-time computations for control or other

action.• Having a parsimonious model is less important if

the realization is large.

15

EXAMPLES• AR(1)• MA(1)• ARMA(1,1)

16

UPDATING THE FORECASTS• Let’s say we have n observations at time t=n

and find a good model for this series and obtain the forecast for Yn+1, Yn+2 and so on. At t=n+1, we observe the value of Yn+1. Now, we want to update our forecasts using the original value of Yn+1 and the forecasted value of it.

17

UPDATING THE FORECASTSThe forecast error is

We can also write this as

18

1

0

ˆ

i

ininnn aYYe

n

e

iini

iini

iini

nnn

aa

aa

YYe

n

1

0

0011

1111 1ˆ1

UPDATING THE FORECASTS

19

1ˆ1ˆˆ

1ˆ1ˆˆ

1ˆˆ

ˆ1ˆ

11

11

1

1

nnnn

nnnn

nnn

nnnnn

YYYY

YYYY

aYY

aYYYY

n=100

1ˆ2ˆ1ˆ1001011100101 YYYY

FORECASTS OF THE TRANSFORMED SERIES

• If you use variance stabilizing transformation, after the forecasting, you have to convert the forecasts for the original series.

• If you use log-transformation, you have to consider the fact that

20

nnnn Yln,,YlnYlnEexpY,,YYE 11

FORECASTS OF THE TRANSFORMED SERIES

• If X has a normal distribution with mean and variance 2,

• Hence, the minimum mean square error forecast for the original series is given by

21

.expXexpE

2

2

nnnn Yln Z whereeVarZexp

2

1

nn Z,,ZZE 1 nn Z,,ZZVar 12

MEASURING THE FORECAST ACCURACY

22


23


24

MOVING AVERAGE AND EXPONENTIAL SMOOTHING

• This is a forecasting procedure based on a simple updating equations to calculate forecasts using the underlying pattern of the series. Not based on ARIMA approach.

• Recent observations are expected to have more power in forecasting values so a model can be constructed that places more weight on recent observations than older observations.

25

MOVING AVERAGE AND EXPONENTIAL SMOOTHING

• Smoothed curve (eliminate up-and-down movement)

• Trend• Seasonality

26

SIMPLE MOVING AVERAGES

• 3 periods moving averagesYt = (Yt-1 + Yt-2 + Yt-3)/3

• Also, 5 periods MA can be considered.

27

Period Actual 3 Quarter MA Forecast 5 Quarter MA forecast

Mar-83 239.3 Missing Missing

Jun-83 239.8 Missing Missing

Sep-83 236.1 Missing Missing

Dec-83 232 238.40 Missing

Mar-84 224.75 235.97 Missing

Jun-84 237.45 230.95 234.39

Sep-84 245.4 231.40 234.02

Dec-84 251.58 235.87 235.14

… So on..


• One can impose weights and use weighted moving averages (WMA).

Eg Y t = 0.6Yt-1 + 0.3Yt-2+ 0.1Yt-2

• How many periods to use is a question; more significant smoothing-out effect with longer lags.

• Peaks and troughs (bottoms) are not predicted.• Events are being averaged out.• Since any moving average is serially correlated,

any sequence of random numbers could appear to exhibit cyclical fluctuation.

28


• Exchange Rates: Forecasts using the SMA(3) model

29

Date RateThree-Quarter

Moving AverageThree-Quarter

Forecast

Mar-85 257.53 missing missingJun-85 250.81 missing missingSe-85 238.38 248.90 missing

Dec-85 207.18 232.12 248.90Mar-86 187.81 211.12 232.12

SIMPLE EXPONENTIAL SMOOTHING (SES)

• Suppressing short-run fluctuation by smoothing the series

• Weighted averages of all previous values with more weights on recent values

• No trend, No seasonality

30


• Observed time seriesY1, Y2, …, Yn

• The equation for the model is

where : the smoothing parameter, 0 1 Yt: the value of the observation at time t

St: the value of the smoothed obs. at time t.

31

11 1 ttt SYS


• The equation can also be written as

• Then, the forecast is

32

error forecast the

tttt SYSS 111

ttt

ttt

SYS

SYS

11


• Why Exponential?: For the observed time series Y1,Y2,…,Yn, Yn+1 can be expressed as a weighted sum of previous observations.

where ci’s are the weights.• Giving more weights to the recent

observations, we can use the geometric weights (decreasing by a constant ratio for every unit increase in lag).

33

221101ˆtttt YcYcYcY

.10,...;1,0;1 ic ii


• Then,

34

1ˆ11ˆ

1111ˆ

1

22

110

ttt

tttt

YYY

YYYY

St+1 St


• Remarks on (smoothing parameter).– Choose between 0 and 1.– If = 1, it becomes a naive model; if is close to

1, more weights are put on recent values. The model fully utilizes forecast errors.

– If is close to 0, distant values are given weights comparable to recent values. Choose close to 0 when there are big random variations in the data.

– is often selected as to minimize the MSE.

35


• Remarks on (smoothing parameter).– In empirical works, 0.05 0.3 commonly

used. Values close to 1 are used rarely.– Numerical Minimization Process:

• Take different values ranging between 0 and 1.• Calculate 1-step-ahead forecast errors for each .• Calculate MSE for each case.• Choose which has the min MSE.

36

n

ttttt e minSYe

1

2


• EXAMPLE:

37

Time Yt St+1 (=0.10) (YtSt)2

1 5 - -

2 7 (0.1)5+(0.9)5=5 4

3 6 (0.1)7+(0.9)5=5.2 0.64

4 3 (0.1)6+(0.9)5.2=5.08 5.1984

5 4 (0.1)3+(0.9)5.28=5.052 1.107

TOTAL 10.945

74.21

nSSE

MSE

• Calculate this for =0.2, 0.3,…,0.9, 1 and compare the MSEs. Choose with minimum MSE


• Some softwares automatically chooses the optimal using the search method or non-linear optimization techniques.

INITIAL VALUE PROBLEM1.Setting S1 to Y1 is one method of initialization.

2.Take the average of, say first 4 or 5 observations and use this as an initial value.

38

DOUBLE EXPONENTIAL SMOOTHING OR HOLT’S EXPONENTIAL SMOOTHING • Introduce a Trend factor to the simple

exponential smoothing method• Trend, but still no seasonality

SES + Trend = DES• Two equations are needed now to handle the

trend.

39

10,1

10,1

11

111

tttt

tttt

TSST

TSYS

Trend term is the expected increase or decrease per unit time period in the current level (mean level)

HOLT’S EXPONENTIAL SMOOTHING

• Two parameters : = smoothing parameter = trend coefficient

• h-step ahead forecast at time t is

• Trend prediction is added in the h-step ahead forecast.

40

ttt hTShY ˆ

Current level Current slope


• Now, we have two updated equations. The first smoothing equation adjusts St directly for the trend of the previous period Tt-1 by adding it to the last smoothed value St-1. This helps to bring St to the appropriate base of the current value. The second smoothing equation updates the trend which is expressed as the difference between last two values.

41


• Initial value problem: – S1 is set to Y1

– T1=Y2Y1 or (YnY1)/(n1)

and can be chosen as the value between 0.02< ,<0.2

or by minimizing the MSE as in SES.

42


• Example: (use = 0.6, =0.7; S1 = 4, T1= 1)

43

Holt Holt

time Yt St Tt

1 3 4 1

2 5 3.8 0.64

3 4 4.78 0.74

4 - 4.78+0.74

5 - 4.78+2*0.74

HOLT-WINTER’S EXPONENTIAL SMOOTHING

• Introduce both Trend and Seasonality factors• Seasonality can be added additively or

multiplicatively.• Model (multiplicative):

44

stt

tt

tttt

ttst

tt

ISY

I

TSST

TSIY

S

1

1

1

11

111


Here, (Yt /St) captures seasonal effects.

s = # of periods in the seasonal cycles(s = 4, for quarterly data)

Three parameters : = smoothing parameter = trend coefficient = seasonality coefficient

45


• h-step ahead forecast

• Seasonal factor is multiplied in the h-step ahead forecast , and can be chosen as

the value between 0.02< ,,<0.2 or by minimizing the MSE as in SES.

46

shtttt IhTShY ˆ


• To initialize Holt-Winter, we need at least one complete season’s data to determine the initial estimates of It-s.

• Initial value:

47

ssYsYT or

sYY

sYY

sYY

sT

sYS

s

stt

s

tt0

sssss

s

tt

///

1.2

/.1

2

11

22110

10


• For the seasonal index, say we have 6 years and 4 quarter (s=4).

STEPS TO FOLLOWSTEP 1: Compute the averages of each of 6

years.

48

averagesyearly The ,,,n,/YAi

in

62144

1


• STEP 2: Divide the observations by the appropriate yearly mean.

49

Year 1 2 3 4 5 6

Q1 Y1/A1 Y5/A2 Y9/A3 Y13/A4 Y17/A5 Y21/A6

Q2 Y2/A1 Y6/A2 Y10/A3 Y14/A4 Y18/A5 Y22/A6

Q3 Y3/A1 Y7/A2 Y11/A3 Y15/A4 Y19/A5 Y23/A6

Q4 Y4/A1 Y8/A2 Y12/A3 Y16/A4 Y20/A5 Y24/A6


• STEP 3: The seasonal indices are formed by computing the average of each row such that

50

6/

6/

6/

6/

6

24

5

20

4

16

3

12

2

8

1

44

6

23

5

19

4

15

3

11

2

7

1

33

6

22

5

18

4

14

3

10

2

6

1

22

6

21

5

17

4

13

3

9

2

5

1

11

AY

AY

AY

AY

AY

AY

I

AY

AY

AY

AY

AY

AY

I

AY

AY

AY

AY

AY

AY

I

AY

AY

AY

AY

AY

AY

I


• Note that, if a computer program selects 0 for and , this does not mean that there is no trend or seasonality.

• For Simple Exponential Smoothing, a level weight near zero implies that simple differencing of the time series may be appropriate.

• For Holt Exponential Smoothing, a level weight near zero implies that the smoothed trend is constant and that an ARIMA model with deterministic trend may be a more appropriate model.

• For Winters Method and Seasonal Exponential Smoothing, a seasonal weight near one implies that a nonseasonal model may be more appropriate and a seasonal weight near zero implies that deterministic seasonal factors may be present.

51

EXAMPLE> HoltWinters(beer)

Holt-Winters exponential smoothing with trend and additive seasonal component.

Call:

HoltWinters(x = beer)

Smoothing parameters:

alpha: 0.1884622

beta : 0.3068298

gamma: 0.4820179

Coefficients:

[,1]

a 50.4105781

b 0.1134935

s1 -2.2048105

s2 4.3814869

s3 2.1977679

s4 -6.5090499

s5 -1.2416780

s6 4.5036243

s7 2.3271515

s8 -5.6818213

s9 -2.8012536

s10 5.2038114

s11 3.3874876

s12 -5.6261644

52

EXAMPLE (Contd.)> beer.hw<-HoltWinters(beer)

> predict(beer.hw,n.ahead=12)

Jan Feb Mar Apr May Jun Jul Aug

1963

1964 49.73637 55.59516 53.53218 45.63670 48.63077 56.74932 55.04649 46.14634

Sep Oct Nov Dec

1963 48.31926 55.01905 52.94883 44.35550

53

ADDITIVE VS MULTIPLICATIVE SEASONALITY

• Seasonal components can be additive in nature or multiplicative. For example, during the month of December the sales for a particular toy may increase by 1 million dollars every year. Thus, we could add to our forecasts for every December the amount of 1 million dollars (over the respective annual average) to account for this seasonal fluctuation. In this case, the seasonality is additive.

• Alternatively, during the month of December the sales for a particular toy may increase by 40%, that is, increase by a factor of 1.4. Thus, when the sales for the toy are generally weak, than the absolute (dollar) increase in sales during December will be relatively weak (but the percentage will be constant); if the sales of the toy are strong, than the absolute (dollar) increase in sales will be proportionately greater. Again, in this case the sales increase by a certain factor, and the seasonal component is thus multiplicative in nature (i.e., the multiplicative seasonal component in this case would be 1.4).

54


• In plots of the series, the distinguishing characteristic between these two types of seasonal components is that in the additive case, the series shows steady seasonal fluctuations, regardless of the overall level of the series; in the multiplicative case, the size of the seasonal fluctuations vary, depending on the overall level of the series.

• Additive model:Forecastt = St + It-s

• Multiplicative model:Forecastt = St*It-s

55


56

Exponential Smoothing Models

1. No trend and additive seasonal variability (1,0)

2. Additive seasonal variability with an additive trend (1,1)

3. Multiplicative seasonal variability with an additive trend (2,1)

4. Multiplicative seasonal variability with a multiplicative trend (2,2)

Exponential Smoothing Models

• Select the type of model to fit based on the presence of – Trend – additive or multiplicative, dampened or not– Seasonal variability – additive or multiplicative

5. Dampened trend with additive seasonal variability (1,1)

6. Multiplicative seasonal variability and dampened trend (2,2)

OTHER METHODS

(i) Adaptive-response smoothing .. Choose from the data using the smoothed and

absolute forecast errors (ii) Additive Winter’s Models.. The seasonality equation is modified. (iii) Gompertz Curve.. Progression of new products(iv) Logistics Curve.. Progression of new products (also with a limit, L)(v) Bass Model

59

EXPONENTIAL SMOOTING IN R

General notation: ETS(Error,Trend,Seasonal)ExponenTial Smoothing

ETS(A,N,N): Simple exponential smoothing with additive errorsETS(A,A,N): Holt's linear method with additive errorsETS(A,A,A): Additive Holt-Winters' method with additive errors

60

EXPONENTIAL SMOOTING IN RFrom Hyndman et al. (2008):• Apply each of 30 methods that are appropriate to

the data. Optimize parameters and initial values using MLE (or some other method).

• Select best method using AIC:AIC = -2 log(Likelihood) + 2p

where p = # parameters.• Produce forecasts using the best method.• Obtain prediction intervals using underlying state

space model (this part is done by R automatically).61***http://robjhyndman.com/research/Rtimeseries_handout.pdf


ets() function• Automatically chooses a model by default using the

AIC• Can handle any combination of trend, seasonality

and damping• Produces prediction intervals for every model• Ensures the parameters are admissible (equivalent to

invertible)• Produces an object of class ets.

62***http://robjhyndman.com/research/Rtimeseries_handout.pdf

EXPONENTIAL SMOOTING IN R> library(tseries)> library(forecast)> library(expsmooth)> fit=ets(beer) > fit2 <- ets(beer,model="MNM",damped=FALSE)> fcast1 <- forecast(fit, h=24)> fcast2 <- forecast(fit2, h=24)

sigma: 1.2714

AIC AICc BIC 478.1877 480.3828 500.8838

63

R automatically finds the best model.

We are defining the model as MNM

EXPONENTIAL SMOOTING IN R> fitETS(A,Ad,A)

Call: ets(y = beer)

Smoothing parameters: alpha = 0.0739 beta = 0.0739 gamma = 0.213 phi = 0.9053

Initial states: l = 38.2918 b = 0.6085

s=-5.9572 3.6056 5.1923 -2.8407 sigma: 1.2714

AIC AICc BIC 478.1877 480.3828 500.8838

64


65

EXPONENTIAL SMOOTING IN R> fit2ETS(M,N,M)

Call: ets(y = beer, model = "MNM", damped = FALSE)

Smoothing parameters: alpha = 0.3689 gamma = 0.3087

Initial states: l = 39.7259 s=0.8789 1.0928 1.108 0.9203

sigma: 0.0296

AIC AICc BIC 490.9042 491.8924 506.0349

66


67


• GOODNESS-OF-FIT> accuracy(fit) ME RMSE MAE MPE MAPE MASE 0.1007482 1.2714088 1.0495752 0.1916268 2.2306151 0.1845166 > accuracy(fit2) ME RMSE MAE MPE MAPE MASE 0.2596092 1.3810629 1.1146970 0.5444713 2.3416001 0.1959651

68

The smaller is the better.


> plot(forecast(fit,level=c(50,80,95)))

69


> plot(forecast(fit2,level=c(50,80,95)))

70

Documents

STAT 497 LECTURE NOTES 7 FORECASTING 1. One of the most important objectives in time series analysis is to forecast its future values. It is the primary