Upload
lethu
View
221
Download
0
Embed Size (px)
Citation preview
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Interval Forecasting
Based on Chapter 7 of the Time Series Forecasting byChatfield
Econometric Forecasting, January 2008
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Outline
1 Introduction
2 Prediction Mean Square Error
3 Prediction Intervals
4 Empirically Based P.I.s
5 Summary
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
IntroductionTerminologyInterval ForecastsDensity ForecastFan Chart
IntroductionIntroduction
Most forecasters realize the importance of providinginterval forecasts and point forecasts in order to:
asses future uncertainty,enable different strategies to be planned for the range ofpossible outcomes indicated by the interval forecast,compare forecasts from different methods more thoroughly.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
IntroductionTerminologyInterval ForecastsDensity ForecastFan Chart
IntroductionTerminology
An interval forecast usually consists of an upper and lower limit.The limits are called
forecast limits,prediction bounds.
The interval is called aconfidence interval,forecast region,prediction interval.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
IntroductionTerminologyInterval ForecastsDensity ForecastFan Chart
IntroductionPredictions
Predictions:are not often a prediction intervals,most often are given as a single value.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
IntroductionTerminologyInterval ForecastsDensity ForecastFan Chart
IntroductionInterval Forecasts
Reasons for not using interval forecasts:rather neglected in statistical literature,no generally accepted method for calculating predictionintervals (with some exception),theoretical prediction intervals are difficult or impossible toevaluate for many econometric models containing manyequations or which depend on non-linear relationship.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
IntroductionTerminologyInterval ForecastsDensity ForecastFan Chart
IntroductionDensity Forecast
By density forecast we mean finding the entire probabilitydistribution of a future value of interest.Linear models with normally distributed innovations:
the density forecast is typically normal with mean equal tothe point forecast and variance equal to that used incomputing a predicion interval.
Linear model without normally distributed innovations:seems to be more prevalent and used e.g. in forecastingvolatility,different percentiles or quantiles of the conditionalprobability distribution of future values are estimated.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
IntroductionTerminologyInterval ForecastsDensity ForecastFan Chart
IntroductionFan Chart
Fan charts
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square Error
Preditcion Mean Square Error (PMSE):
E [en(h)2].
Forecast is unbiased that is,when x̂N(h) is the mean of the predictive distriburion,then
E [eN(h)] = 0 and E [eN(h)2] = Var [eN(h)].
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorPotential Pitfall
Assesing forecast uncertainty, remember:
the var of the forecast 6= the var of the forecast error.
Given data up to time N and a particular method or model:the forecast x̂N(h) is not a random variable, it has varianceof zero,XN+h and eN(h) are random variables, condtioned by theobserved data.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square Error
How to evaluate:
E [eN(h)2] or Var [eN(h)]
and what assumptions should be made?
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorExample
Consider the zero-mean AR(1):
Xt = αXt−1 + εt , {εt} ∼ N(0, σ2ε ).
Assume: complete knowledge of the model (α and σ2ε ).
The point forecasts x̂N(h) will be
α̂hxN rather than αhxN .
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorBias of PMSE
Even assuming that the true model is known a priori, there willstill be biases in the usual estimate obtained by substitutingsample estimates of the model parameters and the residualvariance into the true-model PMSE formula.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Known vs. Unknown Parameters
Consider the case of h = 1 and conditioning on Xn = xn,
eN(1) = XN+1 − x̂N(1).
If the model parameters were known, then
x̂N(1) = αxN ⇒ eN(1) = εN+1
If parameters are not known:
eN(1) = XN+1 − x̂N(1) = αxN + εN+1 − α̂xN = (α− α̂)xN + εN+1
Assume: parameters estimates are obtained by a procedurethat is asymptotically equivalent to maximum likelihood.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorConditional and Unconditional Errors
Looking once more at equation
eN(1) = (α− α̂)xN + εN+1.
Consider:conditional on xN forecast error: xN is fixed α̂ - biasedestimator of α, then the expected value of eN(1) need notbe zero.unconditional forecast error: If, however, we average overall possible values of xN , as well as over εN , then it can beshown that te expectation will indeed be zero giving anunbiased forecast.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorComputing PMSE
Important when computing PMSE.To have unconditional PMSE:
average over the distribution of future innovations(e.g. eN+1),average over the distribution of current observed values(e.g. xN ).
eN(1) = (α− α̂)xN + eN+1
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Prediction Mean Square ErrorUnconditional PMSE
Unconditional PMSE can be usefull to assess the ’success’ of aforecasting method on average. This apporach if used tocompute P.I.s, it effectively assumes that te observatoins usedto estimate the model parameters are independent of thoseused to construct the forecasts. This assumption can bejustified asymptotically. Box assesed that the correction termswould generally be of order 1
N (effect of parameter uncertainty).
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Parameters UncertaintyCorrection
Assume: K-variable vector AR(p) process.True model PMSE at lead time one has to be multiplied by thecorrection factor
[1 + K 1N ] + o( 1
N )
to give the corresponding unconditional PMSE allowing forparameters uncertainty.What does it mean?
the more parameters (K ),the shorter the series (N),
⇒ the greater will be the correction term.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Parameters UncertaintyExample
N = 50, K = 1 and p = 2
the correction for the square root of PMSE is only 2%. for
N = 30, K = 3 and p = 2
the correction for the square root of PMSE rises to 6%.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
FormulaPotential PitfallExampleKnown vs. Unknown ParametersConditioning Forecast ErrorExample
Parameters UncertaintyExample
The effect on probabilities.For normal distribution
95% lie in the interval +/− 1.96.
Suppose the s.d. is 6% larger (no correction factor used).Now,
96.2% lie in the interval +/− (1.96× 1.06) = 2.0776.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
Calculating P.I.
In general P.I.s are of the form:
100(1− α)%.
P.I. for XN+h is given by:
x̂N(h) + /− zα/2√
Var [eN(h)].
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
Formula for P.I.
Properties and assumptions of the formula for P.I.:symmetric about x̂N(h),assumes the forecast is unbiased with PMSEE [eN(h)2] = Var [eN(h)],forecast errors are assumed to be normally distributed.
Note: some authors state that the zα/2 should be replaced bythe precentage point of a t-distribution, with appropriate numberof degrees of freedom (worth making for less than 20 obs).
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
Formula for P.I.
The formula
x̂N(h) + /− zα/2√
Var [eN(h)].
is generally used for P.I.s. preferably after checking theassumptions (e.g. forecast errors are approximately normallydistributed) are at least reasonably satisfied. For any givenforecasting method the main problem will then lie withevaluating Var [eN(h)].
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
P.I.s derived from a fitted probability modelFormulas for PMSE
PMSE can be derived for:ARMA models (also seasonal and integrated),structural state-space,various regression models (typically allow for parameteruncertainty and are conditional in the sense that theydepend on the particular values of the explanatoryvarialbes from where a prediction is being made).
Cannot be derived:some complicated simultaneous equation,non-linear.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
P.I.s without model identification
What to do when a forecasting method is selected without anyformal model identification procedure ?
assume that the method is optimal (in some sense)apply some empirical procedure
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
P.I.s when assumed that method is optimalExample
Exponential smoothing:no obvious trend or seasonality,no attempt to identify the underlying model(i.e. ARIMA(0,1,1)).
PMSE formula:
Var [eN(h)] = [1 + (h − 1)α2]σ2ε .
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
P.I.s when assumed that method is optimalWhen it is reasonable ?
When it is reasonable?Observed one-step-ahead forecast errors show no obviousautocorrelation.No other obvious features of the data (e.g. trend) whichneed to be modeled.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
Forecasting methods not based on a probability model
Assume that the method is optimal in the sense that theone-step ahead errors are uncorrelated.Easy to check by looking at the correlogram of theone-step-ahead errors:
if there is correlation we have more structure in the datawhich should improve the forecast.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
Forecasting methods not based on a probability modelExample
Holt-Winters method with
additive and multiplicative seasonality
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
The additive case
Properties:results equivalent to SARIMA model for which additiveHolt-Winters is optimal,so complicated that it would never be identified in practice.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
Calculating P.I.sProbability ModelNon Formal ModelExample
The multiplicative case
No ARIMA model for which the method is optimal.
Assume: one-step-ahead forecast errors are uncorrelated.Results:
Var [eN(h)] not monotonic increase with h,P.I.s are wider near a seasonal peak as would intuitively beexpected.
Remark: Wider P.I.s near a seasonal peak - not captured bymost alternative approaches (except variance-stabilizingtransformation).
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Empirically based P.I.s
When to use empirically based P.I.s?theoretical formulae not available,doubts about validity of the true model.
Remark: computationally intensive based on:observed distribution of errors,simulation or resample methods.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Empirically based P.I.s1st method
Apply forecasting method to all the past data.Find within-sample ’forecasts’ at 1, 2, 3, . . . steps ahead(from all available time origins).Find the variance of these errors (at each lead time overthe periods of fit).Assume normality.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Empirically based P.I.s1st method
Result:approximate empirical 100(1− α)% P.I. for XN+h is given by
x̂N(h) + /− zα/2√
Var [eN(h)].
Problems:if N small - assume t-distribution,long series is needed to get reliable values for se,h,smooth values to make them increase monotonically with hvalues of
√Var [eN(h)] based on in-sample residuals not
on out-of-sample forecast errors,results comparable to theoretical formulae (if available).
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Empirically based P.I.s2nd method
Split data into two parts.Fit on 1st part.Forecast on 2nd part.Get prediction errors.Refit model - move the fitting window.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Simulation and resampling methods
More computationally intensive approach.Increasingly used for the construction of P.I.s.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Simulation and resampling methodsSimulation (Monte Carlo approach)
Assumption:Probability time-series model is known (and identifiedcorrectly)Generate random innovationsGenerate possible past and future valuesRepeat many timesFind the interval within which the required percentage offuture values lie
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Simulation and resampling methodsResampling (bootstrapping)
Sample from the empirical not theoretical distribution
⇒ distribution-free approach
The idea (the same as for simulation):use the knowledge about the primary structure of themodelgenerate a sequence of possible future valuesfind a P.I. containing the appropriate percentage of futurevalues
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
BootstrappingBrief description
N independent observations,take random sample of size N with replacement.
Result:some values occur twice (or more) some not occur at allIn time-series:makes no sense - correlation over time
Bootstrap by resampling the fitted errors - depend on model.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
BootstrappingProperties
Properties of bootstrap:Bootstrap P.I.s are useful non-parametric alternative to theusual Box-Jenkins intervals.It is difficult to resample correlated data.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
When to use?1st method2nd methodSimulation and Resampling
Uncertainty in Forecasts
Sources of uncertainty in forecasts from econometric models:the model innovations,having estimates of model parameters rather than truevalues,having forecasts of exogenous variables rather than truevalues,misspecification of the model.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
SummaryFindings and Recomendations
Summarized main findings and recommendations:Formulate a model, that provides a reasonable apporx forthe process generating a given series, derive PMSE, anduse the formula.Distinction between a forecasting method and a forecastingmodel should be borne in mind. The former may, or maynot, depend (explicitly or implicitly) on the latter.Use not model but method based approach (e.g. theHolt-Winters method).
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
SummaryFindings and Recomendations
No theoretical formulae, or there are doubts about modelassumptions, use the empirically based approach.The reason why out-of-sample forecasting ability is worsethan within-sample fit is that the wrong model may havebeen identified or may change through time.The formula x̂N(h) + /− zα/2
√Var [eN(h)] assumes:
model has been correctly identified,innovations are normally distributed,the future will be like the past.
Rather than compute P.I.s based on a single ’best’ model,use Bayesian model averaging, or not model-basedapproach.
Pekalski, Swierczyna, Zalewski interval forecasting
IntroductionPrediction Mean Square Error
Prediction IntervalsEmpirically Based P.I.s
Summary
The End
Thank you for yourattention.
Pekalski, Swierczyna, Zalewski interval forecasting