Lecture 9-b ARIMA – Diagnostic Testing & Seasonalities › rsusmel › 4397 › fec-9-b.pdfRS – EC2 - Lecture 14 4 Example (continuation): We check the stationarity and invertibility

RS – EC2 - Lecture 14

1

1

Lecture 9-bARIMA – Diagnostic Testing &

Seasonalities

• Once the model is estimated, we run diagnostic tests. Usually, we check for extra-AR structure in the mean. We check visual plots of residuals, ACFs, and the distribution of residuals. More formally, we compute the LB test on the residuals. If we find extra-AR structure, we increase p and/or q.

If we use arima() or auto.arima() functions, we can use the function checkresiduals() to do the plots and testing for us.

Example: We check the MA(1) model for U.S. long returns> fit_arima_lr_p checkresiduals(fit_arima_lr_p)

Ljung-Box test

data: Residuals from ARIMA(0,0,1) with non-zero meanQ* = 18.579, df = 8, p-value = 0.01728 There seems to be more AR structure

Model df: 2. Total lags used: 10

ARIMA Process – Diagnostic Tests


2

Example (continuation):


Example (continuation): We check stationarity/invertibility too -i.e., if the roots are inside the unit circle. In this case, an MA model, stationarity is not an issue (MA are stationary), but invertibility is.


Note: All roots are inside the unit circle and are real: invertible MA(1).


3

Example: We change the model for U.S. stock returns. We estimate an ARIMA(1,0,5).> fit_arima_lr_p15 fit_arima_lr_p15

Coefficients:ar1 ma1 ma2 ma3 ma4 ma5 intercept

0.7077 -0.4071 -0.1965 -0.0671 0.0338 0.0807 0.0035s.e. 0.1039 0.1058 0.0392 0.0263 0.0256 0.0250 0.0014

sigma^2 estimated as 0.001502: log likelihood = 3278.2, aic = -6540.4

> checkresiduals(fit_arima_lr_p15)

Ljung-Box test

data: Residuals from ARIMA(1,0,5) with non-zero meanQ* = 1.7047, df = 3, p-value = 0.6359 The joint 10 lag autocorrelation not significant.





Note: We still see some small autocorrelations different from 0.


4

Example (continuation): We check the stationarity and invertibilityof ARIMA(1,0,5) model> autoplot(fit_arima_lr_p15)


Note: All roots inside the unit circle: stationary and invertible. Notice that we have some roots on the MA part that are imaginary.

Example: We check the fit of the ARIMA model for U.S. long interest rates> fit_arima_i checkresiduals(fit_arima_i)

Ljung-Box test

data: Residuals from ARIMA(0,1,2)Q* = 34.029, df = 8, p-value = 4.014e-05 Again, more AR or MA structure needed




5



Note: We still see some large autocorrelations. change model (usually, increase p and/or q). But, we may in the presence of a series with regime change. We may need to focus on 2nd regime (post 1950s).

Example (continuation): We check the invertibility of ARIMA(0,1,2) model> autoplot(fit_arima_i)


Note: All roots are inside the unit circle. MA process is invertible. Notice that all roots are real.


6

Example: We check the fit of the ARIMA(4,0,0) model selected by auto.arima for changes in Oil Prices:fit_arima_oil fit_arima_oilSeries: lr_oilARIMA(4,0,0) with zero mean

Coefficients:ar1 ar2 ar3 ar4

0.295 -0.102 -0.057 -0.098s.e. 0.052 0.054 0.055 0.054

sigma^2 estimated as 0.00891: log likelihood=344.52AIC=-679.04 AICc=-678.87 BIC=-659.55

> checkresiduals(fit_arima_oil)

Ljung-Box test

data: Residuals from ARIMA(4,0,0) with zero meanQ* = 2.72, df = 9, p-value = 0.84 No significant joint AR structure





Note: Nothing significant. Happy with fit. Ready to forecast.


7

Example (continuation): We check the stationarity of AR(4) model> autoplot(fit_arima_oil)


Note: All roots inside the unit circle –we have imaginary roots.

• Stationarity in mean does not imply stationarity in variance. However, non-stationarity in mean implies non-stationarity in variance.

• If the mean function is time dependent:1. The variance, Var(yt) is time dependent.2. Var[yt] is unbounded as t.3. Autocovariance functions and ACFs are also time dependent.4. If t is large with respect to the initial value y0, then k 1.

• It is common to use variance stabilizing transformations: Find a function G(.) so that the transformed series G(yt) has a constant variance. Very popular transformation:

Non-Stationarity in Variance

log


8

Example: We log transform the monthly variable Total U.S. Vehicle Sales data (1976: Jan – 2020: Sep):ts_car 0, usually between 0 and 2 (it can be estimated too). When

=1, we have a linear ; when →0, a log transformation for .

Example: We do a Box-Cox transformation of the monthly variable Total U.S. Vehicle Sales data (1976: Jan – 2020: Sep), setting = 0.75:lambda


9

• Variance stabilizing transformation is only done for positive series, usually for nominal series (say, in USD total retail sales or units, like Total U.S. vehicle sales). If a series has negative values, then we need to add each value with a positive number so that all the values in the series are positive.

• Then, we can search for any need for transformation.

• It should be performed before any other analysis, such as differencing.

• Not only stabilize the variance, but we tend to find that it also improves the approximation of the distribution by Normal distribution.

Variance Stabilizing Transformation - Remarks

• In time series, seasonal patterns (“seasonalities”) can show up in two forms: additive and multiplicative.- Additive: The seasonal variation is independent of the level.- Multiplicative: The seasonal variation is a function of the level.

Note: In the multiplicative case, the amplitude of the seasonal pattern is changing over time, while in the additive the amplitude is constant.

Seasonal Time Series


10

Examples: We simulate the two seasonal patterns, additive and multiplicative, with trend and no trend.A. With trend

A. With trend


• In the presence of seasonal patterns, we proceed to do seasonal adjustments to remove these predictable influences, which can blurboth the true underlying movement in the series, as well as certain non-seasonal characteristics which may be of interest to analysts.

The type of adjustment depends on how we view the seasonal pattern: Deterministic or Stochastic.

• Similar to the situation where the series had a trend, once we determine the nature of the seasonal pattern, we filter the series –i.e., we remove the seasonal patter- to conduct further ARIMA modeling.

• When we work with a nominal series (not changes, say, USD total retail sales or total units sold), it is common to first apply a variance stabilizing transformation to the data, usually using logs.



11

• Two types of seasonal behavior:- Deterministic – Usual treatment: Build a deterministic function,

, 0, 1, 2,⋯We can include seasonal (means) dummies, for example, monthly or quarterly dummies. (This is the approach in Brooks’ Chapter 10).

Instead of dummies, trigonometric functions (sum of cosine curves) can be used. A linear time trend is often included in both cases.

-Stochastic – Usual treatment: SARIMA model. For example:Φ Θ

or1 Φ 1 Θ

where s the seasonal periodicity –associated with the frequency– of . For quarterly data, s = 4; monthly, s= 12; daily, s = 7, etc.

Seasonal Time Series: Types

• The raw series along with the ACF and PACF can be used to discover seasonal patterns.

Seasonal Time Series – Visual Patterns

Signs: Periodic repetitive wave pattern in ACF, repetition of significant ACFs, PACFs after s periods.


12

Seasonal Time Series – Visual Patterns

Sign: Significant spikes in ACF/PACF at frequency s, in this case s= 12.

• We use seasonal dummy variables, say monthly, in a linear model to capture the seasonalities. Depending on the seasonality pattern, we have different specifications to remove the pattern.

• Suppose has monthly frequency and we suspect that in every December increases. – For the additive model, we can regress against a constant and a December dummy, Dt: μFor the multiplicative model, we can regress against a constant and a December dummy, Dt:, interacting with a trend:μ ∗The residuals of this regressions, , –i.e., = filtered , free of “monthly seasonal effects”– are used for further ARMA modeling.

Seasonal Time Series – Deterministic


13

Example: We simulate an AR(1) series, with a multiplicative December seasonal behavior.

μ ∗Seas_12


14

Example (continuation): We detrend (“filter” the simulated series.trend


15

Example (continuation): The December seasonal pattern is gone from the detrended series. We run an ARIMA(1,0,0):> fit_y_seas_ar1


16

Example: We model log changes in real estate prices in the LA market, . First, we run a regression to remove (filter) the monthly effects from . Then, we model as an ARMA(p, q) process.RE_da


17

Example (continuation): We define monthly dummiesFeb1


18

Example (continuation): Now, we model et, the filtered LA seriesx_la_filt checkresiduals(fit_ar_la_filt)

Ljung-Box test

data: Residuals from ARIMA(2,0,1) with zero meanQ* = 13.5, df = 7, p-value = 0.06083 Reject H0 at 5% lever. But, judgement call is OK.



Example (continuation): We check residual plots.


Note: ACF shows some small, but significant autocorrelations, but the seasonal (wave) pattern is no longer there.


19

Example (continuation): Finally, we check the stationarity & the invertibility of the ARIMA(2,0,1) process.


Note: All roots inside the unit circle (& real): stationarity and invertibility.

• For stochastic seasonality, we use the Seasonal ARIMA model. In general, we have the SARIMA(P, D, Q)s:

Φ 1 Θwhere 0 is constant andΦ 1 Φ Φ ⋯ Φ

Θ 1 Θ Θ ⋯ ΘExample 1: SARIMA(0,0,1)12= SMA(1)12Θ- Invertibility Condition: |Θ |< 1.

E[ ] = 0. 1 Θ

: , 120, otherwise

ACF non-zero at seasonal lags 12, 24,...

Seasonal Time Series – SARIMA


20

Example 2: SARIMA(1,0,0)12 = SAR(1)121 Φ

The process is Φ

- This is a simple seasonal AR model.- Stationarity Condition: |Φ | < 1.

: Φ , 0, 1, 2,⋯

• When Φ = 1, the series is non-stationary.

Seasonal Time Series - SARIMA

• Now, we put together the seasonal behavior and the ARMA behavior. That is, we have the multiplicative SARIMA model (p,d,q) x (P,D,Q)s

Example 1: ARIMA(0,0,1) x (0,0,1)12 (usually, with monthly data):1 1 Θ

Then, the process isΘ

Example 2: Suppose p = Q = 1 and P = q = 0, with s=4, then, we have an ARIMA(1,0,0) x (0,0,1)4 (usually, with quarterly data):

1 1 ΘThen, the process is

Θ



21

• In general, we the multiplicative SARIMA model (p,d,q) x (P,D,Q)s is written as:

Φ Θ

where is the AR lag polynomial, is the MA lag polynomial, Φ is the seasonal AR lag polynomial, and Θ is the seasonal MA lag polynomial.


Example: We model with a SARIMA model for U.S. vehicle sales. First, we look at the raw data:Car_da


22

Example (continuation): Then, we log transform the data:ts_car


23

Example (continuation): Should we try deterministic seasonalities? No clear trend in data. We regress l_car against monthly dummies:zz


24

Example (continuation): Now, we use auto.arima to check for best SARIMA model:> fit_lcar fit_lcarSeries: l_carARIMA(1,0,1)(0,1,2)[12]

Coefficients:ar1 ma1 sma1 sma2

0.9539 -0.5113 -0.5921 -0.2099s.e. 0.0163 0.0509 0.0464 0.0442

sigma^2 estimated as 0.006296: log likelihood=581.76AIC=-1153.52 AICc=-1153.41 BIC=-1132.21

> checkresiduals(fit_lcar)

Ljung-Box test

data: Residuals from ARIMA(1,0,1)(0,1,2)[12]Q* = 44.006, df = 20, p-value = 0.001502




25

Example (continuation): Now, we check residuals, ACF and distribution.


Note: ACF shows small and significant autocorrelation, but the seasonal pattern is gone. More lags?

Documents

Lecture 9-b ARIMA – Diagnostic Testing & Seasonalities › rsusmel › 4397 › fec-9-b.pdfRS – EC2 - Lecture 14 4 Example (continuation): We check the stationarity and invertibility