View
6
Download
0
Category
Preview:
Citation preview
Forecasting Value at Risk and Expected Shortfall with Mixed Data
Sampling
Trung H. Lea,b,∗
aBanking Faculty, Banking Academy of Vietnam, VietnambNorwich Business School, University of East Anglia, United Kingdom
Abstract
I propose applying the Mixed Data Sampling (MIDAS) framework to forecast Value at Risk(VaR) and Expected shortfall (ES). The new methods exploit the serial dependence in short-horizon returns to directly forecast the tail dynamics at the desired horizon. I perform acomprehensive comparison of out-of-sample VaR and ES forecasts with established modelsfor a wide range of financial assets and backtests. The MIDAS-based models significantlyoutperform traditional GARCH-based forecasts and alternative conditional quantile specifi-cations, especially at multi-day forecast horizons. My analysis advocates models featuringasymmetric conditional quantile and the use of Asymmetric Laplace density to jointly esti-mate VaR and ES.
Keywords: Mixed Data Sampling (MIDAS), Value at Risk, Expected Shortfall, Backtests,Model Confidence Set
?The author would like to gratefully acknowledge Carol Alexander, Alberto Plazzi, Raphael Markellos,Apostolos Kourtis, Lazaros Symeonidis, Maurizio Murgia, Yener Altunbas, Jia Liu and seminar participantsin the Young Scholar Conference 2018 at University of Sussex, the BAFA Corporate Finance and AssetPricing Symposium 2018 at Manchester, the International Symposium in Finance 2018 at Greece, Financialgroup Research Seminar at University of East Anglia and Research Seminar series at Faculty of Economicsand Management, The Free University of Bozen-Bolzano for their helpful comments.
∗Correspondence to: Trung H. Le, Banking Faculty, Banking Academy of Vietnam, 12 Chua Boc Street,Dong Da, Hanoi, Vietnam
Email address: trunglh@hvnh.edu.vn (Trung H. Le)
Preprint submitted to Elsevier December 25, 2019
1. Introduction
The recent 2007-2009 financial crisis has triggered the debate on the accuracy of riskmeasurement models, especially those focusing on tail risk. Yet, two important groundsremain largely unexplored. First, a voluminous literature study tail risk based on Value atRisk (VaR) estimates, although this measure fails to meet the requirements of a coherentrisk metric as defined by Artzner et al. (1999).1 Among the alternatives, expected shortfall(ES) has recently gained more attention.2 Despite its importance, there is little empiricalworks focusing on ES. This is mainly due to the difficulty in estimation and backtestingprocedures (Gneiting, 2011). Second, the large extant literature focuses on 1-day aheadrisk forecasts, which is clearly insufficient to warn investors and financial institutions andliquidate their positions. As emphasised by Engle (2011), p. 438, the financial crisis waspredicable one day ahead, and as such, the key failure in risk modelling in financial crisis lieson their deteriorations in multi-day ahead risk forecasts (Brownlees et al., 2011).
This study addresses these gaps by extending the novel quantile regression based onMixed Data Sampling (MIDAS) of Ghysels et al. (2016) to forecast VaR and ES. The newmethods allow for direct forecasting VaR and ES at the desired horizon, while the use ofsemiparametric specifications avoids making restrictive assumption about conditional returndistribution. To the best of my knowledge, this is the first study in the literature thatapplies MIDAS to obtain ES forecasts. I perform a comprehensive analysis of the forecastingaccuracy of the proposed method. The main analysis involves: 43 international indices; threeforecast horizons (i.e., 1-day, 5-day and 10-day, respectively); twelve forecasting models; sixstatistical backtests on both VaR and ES; and an out-of-sample forecast comparison withtwo loss functions.
My proposal draws on two streams of the literature. First, it is well-established thatfinancial return distribution is not normal and this fact is more pronounced at the multi-dayhorizon.3 Consequently, a good forecasting model at short horizon, such as 1-day ahead,does not necessary yield accurate forecasts at multi-day horizon. Moreover, each quantilein a nonnormal distribution may evolve in different dynamics and depends on different setsof information.4 These observations suggest that a tail risk model may benefit from a di-rect estimation of tail area rather than the traditional approach using conditional returndistribution models, such as the GARCH family.
Second, several studies document that the dynamic of return volatility is characterisedby multiple components capturing information at different time horizons.5 Given the strong
1Previous papers mainly examine the predictive power of risk models in producing VaR forecasts, eitherexplicitly (see, e.g, Berkowitz et al., 2011; Boucher et al., 2014) or implicitly via volatility forecasting (see,e.g, Brownlees et al., 2011; Bams et al., 2017)
2The “Minimum capital requirements for market risk” of Basel Committee on Banking Supervision (2016)has moved toward using ES, as a complement of VaR, to calculate the regulatory capital requirement. Thisregulatory agreement is expected to be fully implemented on January 1, 2022.
3Engle (2011) and Neuberger (2012) find that the asymmetry in return distribution increases with horizonup to one-year and converges very slowly to normality. Recently, Fama and French (2018) apply bootstrap-ping simulations and document significant return skewness at even 20- and 30-years returns.
4Cenesizoglu and Timmermann (2008) and Lima and Meng (2017) document asymmetric effects of eco-nomic variables on different parts of the return distribution and time-variation in their explanatory powers.
5Some notable examples are Chernov et al. (2003), Corsi (2009) and Engle et al. (2013).
2
correlation between volatility and return quantiles, it is natural to calibrate a model thatcould capture different components of information in modelling the tail dynamic. Moreover,Engle (2011) and Neuberger (2012) highlight that long-horizon return distribution dependscrucially on the dynamics in short-horizon return process. Therefore, one needs to take intoaccount the serial dependence in short-horizon return when forecasting VaR and ES at themulti-horizon-ahead.
Altogether, I propose to extend the novel MIDAS quantile regression of Ghysels et al.(2016) to directly forecast VaR and ES at the desired horizon. The MIDAS frameworkintroduced by Ghysels et al. (2004) provides an efficient method to link variables sampled atdifferent frequencies. The use of flexible and parsimonious lag polynomials allows MIDASto directly forecast lower frequency variables by exploiting the data-rich environment athigher frequencies. Thus, MIDAS also provides a suitable framework to capture differentcomponents in the tail dynamics by data-driven weighting scheme with flexible shapes. Moreimportantly, this approach offers a direct projection from short-horizon return to multi-horizon return distribution. Andreou et al. (2011) locate the MIDAS approach in the middleof the ‘direct ’ and ‘iterate’ methods in the forecasting literature.6
To forecast ES, however, one needs to address its central problem of “non-elicitability”,which is the lack of a scoring function to facilitate the estimation (Gneiting, 2011). Toovercome this issue, I follow two semiparametric approaches proposed in the literature, whichdirectly model VaR and ES and allow their dynamics to vary for each quantile levels. Istart from the premise that it is important to account for the serial dependence of high-frequency return process (i.e. daily, in this study) in modelling the conditional density atthe desired horizon (Neuberger, 2012). For this purpose, I develop the proposed models onthe MIDAS-based quantile regression of Ghysels et al. (2016). In particular, the conditionalquantile is based on a mixture of lagged higher frequency returns, which is driven by thedata environment and flexibly differs for each quantile level and forecast horizon. Moreover,I also develop an asymmetric specification, which provides better out-of-sample forecastperformance than its symmetric counterparts of Ghysels et al. (2016) in most cases.
In the first approach, I adopt the semiparametric model of Taylor (2019) based on theAsymmetric Laplace (AL) density. The author explores the fact that although ES is notindividually elicitable, it is jointly elicitable with VaR under a set of suitable scoring functions(Fissler and Ziegel, 2016). Since the AL log-likelihood is a member of this set, VaR and EScan be jointly estimated via maximum likelihood of an AL density. In the second approach,I follow Manganelli and Engle (2004) to combine quantile regression and extreme valuetheory (EVT). The conditional VaR and ES are estimated by fitting a Generalised ParetoDistribution (GPD) to the extreme observations that exceeded a threshold level.
In the empirical analysis, I employ several well-established models as the benchmarksto investigate the forecasting ability of the proposed method. First, I adopt the historicalsimulation approach to simply compute conditional VaR and ES as the empirical quantileand the average of the realized violations of the estimation period. This benchmark has
6A number of recent studies document the advantage of applying MIDAS in financial forecasts, includingAndreou et al. (2013); Foroni et al. (2018) for macroeconomic predictions; Pettenuzzo et al. (2016) for returndensity; Ghysels et al. (2006) for volatility.
3
been the most popular method used in the financial institutions (Berkowitz et al., 2011).Second, I consider the filtered historical simulation approach introduced by Barone-Adesiet al. (1999) and Giannopoulos and Tunaru (2005). I use two GARCH models to prefilterthe data. VaR and ES forecasts are then obtained from the empirical distribution approxi-mated from simulated paths of returns at the desired horizon using bootstrapping methods.Finally, I replace MIDAS-based quantile specifications by the conditional autoregressive VaR(CAViaR) specifications of Engle and Manganelli (2004). The CAViaR-based dynamics haveattractive autoregressive structure, yet one needs to form a single-horizon return series thatmatches the forecast horizon in the model estimation (see, for example, Meng and Taylor,2018; Taylor, 2019, for applications in VaR and ES forecasts).
I use a battery of statistical tests to compare the out-of-sample VaR and ES forecastsbetween competing models. In the first stage, I analyse their absolute performance basedon the desired properties of VaR and ES as risk metrics, such as correct tail coverage andinterdependent exceedances. The backtests include the unconditional coverage test of Kupiec(1995), the dynamic quantile test of Engle and Manganelli (2004), the unconditional ES teston violation residuals of McNeil and Frey (2000), the unconditional and conditional ES testusing probability-integral-transform (PIT) of Du and Escanciano (2017) and the multinomialVaR test of Kratz et al. (2018). In the second stage, I investigate the relative performanceof competing models in term of minimizing two loss functions. To this end, I form a set ofsuperior models using the Model Confidence Set (MCS) technique of Hansen et al. (2011).
In summary, I obtain strong evidence in favor of the new models across quantile levelsand forecasting horizons. Although my focus is to improve multi-day horizon VaR and ES,the MIDAS-based models provide competitive performance to the benchmarks at the 1-dayhorizon as well. The benefit of MIDAS framework is more pronounced at multi-day forecasthorizons. The MIDAS-based models lead to the lowest number of test rejections for bothVaR and ES forecasts at the 5- and 10-day horizons. The asymmetric MIDAS-based modelsgenerate the lowest forecast errors and are often included in the set of superior models. Myempirical results reveal that the naive aggregation to single-horizon return series leads tosubstantial loss in forecasting information as the historical simulation and the CAViaR-basedmodels are usually inferior to all other models in multi-day forecasting horizons. Finally,I find evidence supporting the joint model of Taylor (2019) that use the AL likelihood inforecasting VaR and ES. The main results are robust when I repeat the analysis for individualstocks, alternative asset classes, different market regimes and separately for developed versusemerging stock markets.
The remainder of the paper is structured as follows. Section 2 introduces my proposedmethods. Section 3 reviews the benchmark methods, while Section 4 presents the backtest-ing procedures for VaR and ES forecasts. Section 5 presents the empirical results on theout-of-sample forecast comparison. Section 6 presents several robustness checks on modelperformance spanning different market regimes, alternative assets and alternative length ofestimation windows. Section 7 concludes the study.
4
2. Proposed new methods for VaR and ES forecasts
2.1. The MIDAS-based Conditional Quantile
Let rt = ln(Pt/Pt−1) be the daily continuously compounded return series where Pt isthe closing price of trading day t. The h-day horizon return is defined as rt,h =
∑hi=1 rt+i.
The h-day VaR of an asset or portfolio returns at the (1 − α)% confidence level is simplythe conditional quantile at α, Qα(rt,h) estimated at time t − 1.7 The main ingredient ofthe proposed models is the MIDAS-based conditional quantile specification introduced byGhysels et al. (2016). The conditional quantile of returns at any horizon is specified as alinear function of conditioning variables, which can be sampled at different frequencies:
Qα(rt,h) = β0α,h + β1
α,h
D∑d=1
ϕd(κα,h)|rt−d,1| (1)
where the absolute daily return |rt−d,1| is the conditioning variable with a lag length of Ddays. ϕd(.) is the polynomial function that linearly filters the conditioning variable andprojects to the conditional quantile. κα,h is a low-dimensional parameter vector that parsi-moniously defines the shape of the filtering function. The vector of estimated parametersθα,h = (β0
α,h, β1α,h, κα,h) is quantile-specific at considered horizon.
A natural extension of (1) is to capture the well-documented asymmetric effects of pos-itive and negative returns (see, e.g. Engle and Manganelli, 2004; Taylor, 2019). The asym-metric conditional quantile can be specified as follow:
(2)Qα(rt,h) = β0α,h + β1−
α,h
D∑d=1
ϕd(κα,h)I(rt−d,1<0)|rt−d,1|+β1+α,h
D∑d=1
ϕd(κα,h)I(rt−d≥0)|rt−d,1|
where I(.) is the indicator function. To retain the parsimonious advantage of the MIDASframework, I apply one polynomial ϕd(κα,h) but allowing for different slope coefficients fornegative and positive lagged returns. I follow Ghysels et al. (2016) to specify ϕd(κα,h)by the “Beta” polynomial with two parameters, ϕd(κ1, κ2), given that it provides highlyflexible shapes (see, Ghysels et al., 2007, for technical discussions and alternative polynomialfunctions). The Beta polynomial is expressed as follows:
ϕ(κ1, κ2) =f( d
D, κ1, κ2)∑D
d=1 f( dD, κ1, κ2)
(3)
where:
f(x, a, b) =xa−1(1− x)b−1Γ(a+ b)
Γ(a)Γ(b)
Γ(a) =
∫ ∞0
e−xxa−1dx.
7Throughout the paper, I use the terms ”VaR” and ”conditional quantile” at the α quantile level inter-changeably to imply the conditional VaR at the (1− α) confidence level.
5
I restrict κ1 = 1 and κ2 > 1 in order to have decaying weights on the conditioning variable.8
The lag length is set at D = 100 days for all forecasting horizons. This choice is basedon the observation of Ghysels et al. (2006) that using lags longer than 50 days has littleeffect on volatility forecasts up to 20-days horizon. The conditional quantile is estimated byminimizing the following tick loss function:
θα,h = argminθα,h
T−1
T∑t=1
[rt,h −Qα(rt,h)][α− I(rt,h≤Qα(rt,h)
](4)
The conditional ES is the expected loss given a VaR violation occurred and can be expressedas:
ESα(rt,h) = E [rt,h|rt,h ≤ Qα(rt,h)] (5)
where E[.] denotes the expectation at time t − 1. In the next subsections, I present twoalternative approaches to estimate ES based on the above MIDAS-based conditional VaRspecifications.
2.2. Forecast VaR and ES with Asymmetric Laplace Distribution
In the first approach, I jointly estimate VaR and ES using Asymmetric Laplace (AL)density as proposed by Taylor (2019). This model is motivated by the work of Koenker andMachado (1999), who link the minimisation of the ‘tick loss ’ function in (4) to the maximumlikelihood of an AL density specified as follows:
f(rt) =α(1− α)
σexp
(−(rt −Qα(rt))(α− I(rt ≤ Qα(rt))
σ
)(6)
where, for this density, Qα(rt) is the time-varying location, while σ > 0 and 0 < α < 1 arethe scale and skew parameters, respectively.9 Note that the return process is not assumed tofollow AL distribution since the skew parameter α is chosen corresponding to the quantilelevel of interest. Taylor (2019) argues that if the scale parameter σ varies over time, itsmaximum likelihood estimation can be interpreted as the time-varying expectation of the‘tick loss ’ function:
σt = E [(rt −Qα(rt)) (α− I (rt ≤ Qα(rt)))] (7)
Since Bassett et al. (2004) link the conditional ES to quantile regression by:
ESα(rt) = E(rt)−1
αE [(rt −Qα(rt)) (α− I (rt ≤ Qα(rt)))]
8Similar Ghysels et al. (2016), I find that optimising both two parameters can improve the goodness-of-fitin quantile estimate marginally. However, the optimisation comes at significant computational cost and alower convergence rate.
9To simplify the notation, I drop the horizon subscript h in this subsection, keeping in mind that theseries refers to the non overlapping h-day horizon from day t to t+ h.
6
then (7) can be rewritten in term of conditional ES and conditional mean µt = E(rt) as:
σt = α(µt − ESα(rt))
Thus, for given specifications on the dynamics of the conditional mean, conditional VaR andES, the AL density in (6) can be rewritten in conditional terms as:
f(rt) =1− α
µt − ESα(rt)exp
(−(rt −Qα(rt))(α− I(rt ≤ Qα(rt))
α(µt − ESα(rt))
)(8)
Without the loss of generality, I specify the conditional mean return as an AR(1) formulationwhere µt = a0 + a1rt−1 to account for possible autocorrelation in the return process. I followTaylor (2019) to specify conditional ES as an exponential function of the conditional quantile:
ESα(rt) = [1 + exp(γ)]Qα(rt) (9)
where γ controls the joint dynamics of VaR and ES. The use of exponentiation function is toprevent possible crossovers between conditional VaR and ES. The Qα(rt) can follow eitherthe MIDAS-based specifications in (1) and (2).
I follow the optimisation procedure of Taylor (2019) to jointly estimate VaR and ES. Toassist the optimisation, I separately estimate the coefficients in the conditional mean usingmaximum likelihood and the conditional quantile using the MIDAS quantile regression inthe first step.10 Next, I randomly sample 104 potential values for the γ coefficient in the ESformulation of (9) using an uniform random number generator between -3 and 0, based onthe initial experimentation and estimated results in Taylor (2019). These random numbersare combined with the optimised parameters of the conditional mean and quantile functionsto form the vectors of candidate parameters. For each of these vectors, I compute thenegative of the log-likelihood function in (8) and select ten vectors that produce the lowestvalues as starting parameters. Similar to Engle and Manganelli (2004), I employ both theNelder-Mead simplex algorithm and a quasi-Newton method as the optimisation routines.For each starting vector, I first run the simplex method and then feed the optimised valuesto the quasi-Newton algorithm. The optimal parameters are then fed again to the simplexalgorithm as new starting parameters. This procedure is repeated until the convergencecriterion are satisfied. I term the models which define Qα(rt) in (1) and (2) as ‘Midas-AL’,and ‘MidasAs-AL’, respectively.
2.3. Forecast of VaR and ES with Extreme Value Theory
In the second approach, I adopt the two-step estimation procedure suggested by Man-ganelli and Engle (2004). First, the MIDAS quantile regression of Ghysels et al. (2016)is estimated at a threshold level which is not as extreme as the quantile level of interest.Similar to Manganelli and Engle (2004), I choose the threshold level at αu = 7.5%. The
10The estimation is based on an R code created by the author following the Matlab toolbox provided byEric Ghysels.
7
standardised quantile residuals, Zαu,t, are then obtained as follows:
Zαu,t =rt
Qαu(rt)− 1 (10)
where Qαu(rt) is the conditional quantile at threshold level αu. Second, I fit the GeneralisedPareto Distribution (GPD) to the standardised quantile residuals of threshold violations, i.e
Zexceedαu,t = Zαu,t|Zαu,t > 0 ∼ GPD(ξ, ς), where ξ < 1 is the shape parameter and ς is the scale
parameter. Conditional VaR and ES at any quantile level α < αu then can be computedusing the results of McNeil and Frey (2000):
qα(Zαu,t) =ς
ξ
[(αT
Tu
)−ξ− 1
]
esα(Zαu,t) = qα(Zαu,t)
(1
1− ξ+
ς
(1− ξ)qα(Zαu,t)
)Qα(rt) = Qαu(rt) [1 +Qα(Zαu,t)]
ESα(rt) = Qαu(rt) [1 + ESα(Zαu,t)]
where qα(Zαu,t) is the EVT-based estimate of the α-quantile of the marginal distributionof Zαu,t and esα(Zαu,t) = E [Zαu,t|Zαu,t < qα(Zαu,t)], whereas Tu denotes the number of ex-ceedances beyond the conditional threshold. In this approach, I denote the model that usesthe (1) specification in quantile regression as ‘Midas-Evt ’, whereas I use the term ‘MidasAs-Evt ’ when specification in (2) is used.
3. Benchmark Models
In this section, I present a set of benchmark models to examine the predictive power ofnew methods on out-of-sample VaR and ES forecasts. The details of forecasting models arepresented in Table A.1 in Appendix.
3.1. Historical Simulation
My first benchmark is the historical simulation method, which estimates VaR and ES asthe empirical quantiles and the average of realized violations in each estimation period.11
This benchmark has the advantage of making no assumption about the return distribution,yet simple to implement. As a result, this method has been the most popular procedure usedin the financial institutions, especially in the commercial banks (Berkowitz et al., 2011). Theestimates of conditional VaR and ES are based on the empirical quantile at a certain window
11I thank the Referee for suggesting this benchmark
8
length as
Qα(rt,h) = rα|ri,ht−1t−2500 (11)
ESα(rt,h) =1
Sα
S∑s=1
ri,hI(ri,h<Qα(rt,h)) (12)
where ri,h is the non-overlapping h-day returns and rα is the (100 × α)th order statistic ofthe estimation window containing S observations. The forecasts obtained using historicalsimulation method is sensitive to the estimation window and the chosen window length(Nieto and Ruiz, 2016). In the main analysis, I consider the moving windows of size of 2500observations at the daily frequency to be consistent with the alternative forecasting methods.I refer the forecast from this method as the HistSim model.
3.2. Filtered Historical Simulation
The second benchmark method is the Filtered Historical Simulation (Fhs) introduced byBarone-Adesi et al. (1999) for VaR and extended to ES by Giannopoulos and Tunaru (2005).Kuester et al. (2006) find that this approach outperforms the simple historical simulation aswell as the analytical approximation in VaR forecasts.
I consider two GARCH models to prefilter the daily return, namely the GARCH(1,1)model of Bollerslev (1987) and its asymmetric version, i.e. GJR-GARCH(1,1) model ofGlosten et al. (1993). Brownlees et al. (2011) document better volatility forecasting perfor-mance for the latter relative to alternative GARCH-type models. To be consistent with theMIDAS-based models, I model conditional mean as an AR(1) process:
rt = a0 + a1rt−1 + σtzt (13)
while the conditional variance process is defined as:
GARCH: σ2t = β0 + β1ε
2t−1 + β2σ
2t−1 (14)
GJR-GARCH: σ2t = β0 + β1ε
2t−1 + β2I(εt−1<0)ε
2t−1 + β3σ
2t−1 (15)
where εt = σtzt is the residuals from the mean equation and zt is the series of standardisedresdiuals, which follows the standardised Skewed Generalised Error (SGE) distribution ofTheodossiou (2015), i.e. zt ∼ SGE(0, 1, λ, η). This distribution allows for tail-fatness andasymmetry in the return process, where the shape parameters −1 < λ < 1 and η > 0 controlasymmetry and tail thickness, respectively. The distributional density is symmetric whenλ = 0 and skews to the left (right) when λ < 0 (λ > 0). When λ = 0 and η = 2, it gives thestandardised normal distribution (see, e.g., Feunou et al., 2016; Anatolyev and Petukhov,2016, for application of SGE distribution to financial data).
To estimate conditional VaR and ES at h-day horizon, I perform the following algo-rithm. First, I randomly sample z∗t+1, z
∗t+2, .., z
∗t+h with replacement on day t from the
set of standardised residuals. Then, the sampled residuals are plugged into the conditionalmean and variance equations (i.e., (13) - (15)) to generate a simulated path of returnsr∗t+1, r
∗t+2, ..., r
∗t+h. The bootstrapped h-day return is then constructed as r∗t,h =
∑hi=1 r
∗t+i.
These above steps are repeated B = 10, 000 times to form an empirical return distribution
9
at the h-day horizon, rbt,h = r1t,h, r
2t,h, ..., r
Bt,h. Finally, the conditional VaR is obtained as
the αth percentile of the simulated return distribution:
QBα (rt,h) = rbt,hBα (16)
and the corresponding conditional ES is:
ESBα (rt,h) =1
Bα
B∑b=1
rbt,hI(rbt,h<QBα (rt,h)) (17)
where I(rbt,h<QBα (rt,h)) is an indicator function. I term the VaR and ES forecasts from this
method as ”GARCH-Evt” when the conditional variance in (14) is used and ”GJR-Fhs”when (15) is used.
3.3. EVT-based Filter Historical Simulation
An alternative simulation approach is to combine Fhs with EVT as proposed by McNeiland Frey (2000). Novales and Garcia-Jorcano (2018) find that the EVT-based models providebetter VaR and ES forecasts than those of non EVT-based models. To this end, I fit a GPDto the in-sample standardised residuals that exceeded the threshold, which correspond tothe 7.5% percentile of the standardised residuals. Next, I follow McNeil and Frey (2000)to simulate the conditional return distribution at the h-day horizon using the followingalgorithm. Similar to the Fhs, I randomly sample z∗t+1, z
∗t+2, .., z
∗t+h with replacement from
the set of standardised residuals. If the bootstrapped z∗ is lower than the threshold level, Ireplace it with a simulated value from a GPD (ξ,η), where ξ and η are the estimated GPDparameters from in-sample standardised residuals. Otherwise, the sampled standardisedresiduals is used. Then, I obtain the empirical return distribution at the h-day horizon usingB = 10, 000 trails similar to the Fhs algorithm, rbt,h = r1
t,h, r2t,h, ..., r
Bt,h. Finally, the
conditional VaR and ES are also obtained by (16) and (17) as above. I term the VaR andES forecasts from this approach “GARCH-Evt” and “GJR-Evt”, depending on whether thefiltering model is GARCH(1,1) and GJR-GARCH(1,1), respectively.
3.4. CAViaR-based Models
In the next benchmark method, I replace the MIDAS-based specifications with two ana-logues drawing from the CAViaR model of Engle and Manganelli (2004):Symmetric Absolute Value:
Qα(rt) = β0 + β1Qα(rt−1) + β2|rt−1| (18)
Asymmetric Slope:
Qα(rt) = β0 + β1Qα(rt−1) + β−2 I(rt−1<0)|rt−1|+β+2 I(rt−1≥0)|rt−1| (19)
The conditional VaR and ES are estimated using either AL density or EVT as describedearlier. A contrasting difference between this benchmark method and MIDAS-based modelsis the treatment of higher frequency observations. The CAViaR model works exclusively on
10
single-horizon setting. This means that one needs to aggregate higher frequency returns tomatch the target forecasting horizon to perform model estimation. I term the forecastingmodels ‘Sav-AL’ and ‘Sav-Evt ’ when symmetric absolute value specification is utilised. Alter-natively, I refer the models as ‘As-AL’ and ‘As-Evt ’ when the asymmetric slope specificationis employed.
4. Evaluation Methods for VaR and ES Forecasts
I employ two alternative ways to evaluate the accuracy of out-of-sample VaR and ESforecasts. First, I assess the absolute performance of VaR and ES forecasts correspondingto their usages as risk measures. Second, I evaluate the relative performance of competingmodels using two loss functions.
4.1. Absolute Performance Evaluation
4.1.1. VaR backtests
I employ two popular tests to investigate the accuracy of VaR forecasts, including theunconditional coverage (UC ) test of Kupiec (1995) and the dynamic quantile (DQ) test ofEngle and Manganelli (2004). Under the null hypothesis of the UC test, the number ofVaR violations is not statistically different from the chosen quantile level. The test can beperformed using the log-likelihood ratio (LR) statistic:
LR = 2[Tuln(Tu/(αT )) + (T − Tu)ln((T − Tu)/(T − αT ))]
where T is the number of observations, α is the probability level and Tu is the number of VaRexceedances. The LR test statistic follows a χ2(1) distribution. Apart from unconditionalcoverage, DQ further examines the dependence between VaR violations. The test statisticinvolves a transformation of VaR series to a hit sequence, Hitt = I(rt<Qα(rt)) − α. Underthe null hypothesis of correct VaR forecasts, Hitt should have a zero unconditional andconditional expectation given the information set available at time t − 1. The test can beperformed using a linear regression Hitt = Xβ+εt, where X is a set of potential explanatoryvariables, including a constant, the current level of VaR and five lags of Hitt. The teststatistic is specified as:
DQ =b′X ′Xb′
α(1− α)
where b are the estimated coefficients of the linear regression and the DQ test statistic followsχ2(7) distribution, where 7 is the column dimension of X.
4.1.2. ES backtests
I consider three backtesting procedures for ES forecasts. First, I employ the discrep-ancy test of McNeil and Frey (2000). After standardizing by corresponding VaR estimates,the standardised discrepancies between VaR violations and ES forecasts should have uncon-ditional mean of zero under correct risk model. This null hypothesis can be tested usingbootstrap method with 10,000 trials as documented in McNeil and Frey (2000).
11
Second, I adopt the unconditional and conditional ES tests of Du and Escanciano (2017)due to their analogy with the VaR backtests. Instead of explicitly employing ES estimates,they implicitly examine the accuracy of the risk model in tail coverage. These tests are basedon the observation that VaR violations should form a class of martingale difference sequence(MDS), indexed by the considered quantile level.12 Du and Escanciano (2017) argue thatthe cumulative violations also form MDS and provide meaningful information about theconditional tail when a violation occurs to backtest ES. The cumulative violation process isdefined as the integral of VaR violations:
Ht(α) =1
α
∫ α
0
ht(u)du
where ht(u) = I(rt<Qu(rt)) is the hit indicator at quantile level, u, at time t. If the riskmodel is correctly specified, ht(u) has mean u. Similar to Du and Escanciano (2017), I
define ut = F (rt|θα,Ωt−1) for computational purposes, where F (.|Ωt−1) is the conditional
cumulative return distribution given the estimated parameters of the risk model, θα. Then,ht(u) = I(rt<Qu(rt)) = I(ut<u) and the cumulative violations process can be written as:
Ht(α, θα) =1
α
∫ α
0
I(ut<u)du =1
α(α− ut)I(ut<α)
The unconditional ES test can be conducted by testing the null hypothesisH0 : E[Ht(α, θα)
]=
α/2 using a standard t-test:
UES =
√T(H(α)− α/2
)var(Ht(α))
∼ N(0, 1) (20)
where T is the number of forecasts and var(Ht(α)) =√α(1/3− α/4), and H(α) is the
sample mean of H(α)Tt=1. The conditional ES test can be obtained by checking the null
hypothesis being H0 : E[Ht(α, θα)− α/2|Ωt−1
]= 0. For this purpose, the lag-j autovari-
ance, γT,j, and autocorrelation, ρT,j, of Ht(α)Tt=1 for j ≥ 0 are defined as:
γT,j =1
T − j
T∑t=j+1
[Ht(α)− α/2] [Ht−j(α)− α/2] and ρT,j =γT,jγT,0
To be consistent with the DQ test, I chose a lag order m = 5. The test can then be conducted
12Du and Escanciano (2017) also propose the modified tests that are robust to the estimation risk due tothe size distortion of the test statistics, which is recently extended by Banulescu et al. (2019) to the bivariatecase. Since the in-sample size in my analysis is relatively large to the out-of-sample forecast, I opt for theirbasis tests given the computation cost. I thank the Associate Editor for pointing this out.
12
using a simple Box-Pierce test statistic.
CES(m) = NT∑j=1
ρ2T,j ∼ χ2
m (21)
Finally, I employ the multinomial VaR (MultiVaR) test of Kratz et al. (2018) to evaluate theaccuracy in tail coverage by simultaneously testing VaR estimates at multiple quantile levels.From a practical viewpoint, this test has the advantage of not having to store predictivedistribution F (.|Ωt−1) from the risk model at each forecast. For a given starting quantilelevel of interest, α, I consider a series of VaR forecasts at levels α1, .., αN given by:
αj = α +j − 1
N(1− α), j = 1, .., N (22)
I consider the starting quantile level α = 0.025 , which is equivalent to backtesting ESforecasts at the 2.5% quantile level. This choice is motivated by the requirement of BaselCommittee on Banking Supervision (2016) for ES forecasts.13 The sequence Xt =
∑Nj=1 It,j
with It,j = I(rt<Qαj ) counts the number of VaR estimates being violated at each time t.
Similar to the individual VaR estimate, the sequence (Xt) should satisfy the unconditionalcoverage, i.e. P (Xt ≤ j) = αj+1, j = 0, ..., N for all t and the conditional coverage, i.e.Xt is independent of Xs for all s 6= t. Kratz et al. (2018) show that the two above condi-tions can be tested using multinomial distribution MN(T, (p0, ..., PN)) where T is number oftrials. At each trial, I observe N + 1 outcomes (0, 1, ..., N) depending on how many VaRlevels are breached with corresponding probabilities p0, ..., pN . The observed cell counts aredefined as Oj =
∑Tt=1 IXt=j. Under the null hypothesis of correct model, the random vector
(O0, O1, ..., OT ) should follow a multinomial distribution. Kratz et al. (2018) propose severaltest statistics to examine this hypothesis. In my application, I choose the Nass test (Nass,1959) with N = 4, that exhibits to be a good compromise between size and power of test(for technical details, refer to Kratz et al., 2018).
4.2. Relative Performance Evaluation
To evaluate the relative accuracy and facilitate decision making between different fore-casting methods, it is necessary to employ loss functions. Models generate low expected lossarguably preferred over those with higher loss values. To simplify the notation in this sub-section, let Qt = Qα(rt) be the conditional VaR and ESt = ESα(rt) be the conditional ES.Since VaR is elicitable using (4), Giacomini and Komunjer (2005) argues that this functionis a natural choice to compare VaR forecasts:
LQ(Qt) = (rt − Qt)[α− I(rt≤Qt)
](23)
13The use of quantile regressions cannot guard against the possibility of the well-known quantile crossing.On any day when the issue is observed, I apply the recently developed method of monotonically rearrangementof Chernozhukov et al. (2010) to correct the problem.
13
The recent study of Fissler and Ziegel (2016) suggests a family of strictly consistent lossfunctions in which VaR and ES forecasts are jointly elicitable. I adapt a member of thisfamily defined in Fissler et al. (2015) to jointly compare the forecast errors of VaR and ESas:
LFZG(Qt, ESt) = (I(rt<Qt)− α)Qt − I(rt<Qt)
rt
+exp(ESt)
1 + exp(ESt))
(ESt − Qt +
1
αI(rt<Qt)
(Qt − rt))
(24)
+ ln
(2
1 + exp(ESt)
)
Using these loss functions, I apply the model confidence set (MCS) method of Hansen et al.(2011) to form a set of superior models. The MCS procedure starts with the initial set offorecasting models, M0 to deliver the superior set of models M∗
1−α∗ , which contains smallernumber of models, m∗ < M0 , for a given significant level α∗.14 In the main analysis, I useα∗ = 5% to construct the 5% MCS.15 The test applies an elimination rule where at eachstep, a significance test is conducted to eliminate the worst performing model based on anequivalence test, δM, and an elimination rule eM, as follows:
H0,M : E(∆Li,j,t) = 0, for all i, j ∈MHA,M : E(∆Li,j,t) 6= 0, for some i, j ∈M
where M ⊂ M0 is the set of remaining models at each step and ∆Li,j,t is the loss differencebetween model i and j at time t. If the null hypothesis H0,M is not rejected by the equivalencetest δM, the MCS is defined as M∗
1−α∗ = M. Otherwise, the worst performing model iseliminated using the elimination rule eM. I employ the equivalence test based on the rangestatistic in Hansen et al. (2011):16
TM = maxi∈M|ti,j| (25)
where
ti,j =∆Li,j√
V ar(∆Li,j); ∆Li,j = T−1
T∑t=1
∆Li,j,t
where ∆Li,j is the average sample loss difference between models i and j, V ar(∆Li,j) isestimate of the asymptotic variance of ∆Li,j, computed using a block-bootstrap with 10, 000
14Note that I use α∗ to differentiate the significant level of MCS analysis to the quantile level, α, in VaRand ES forecasts.
15The 10% MCS is presented in Table A.11 and provides similar result.16I also employ the alternative test statistic in Hansen et al. (2011), which is the semi-quadratic statistic.
The results are presented in Table A.12 and yields similar to the main analysis
14
trials and a block size set at l = 4 observations.17
The elimination rule is then specified as:
eM = arg maxi∈M
supj∈M
ti,j (26)
where the model with the highest value of ti,j is eliminated if the null hypothesis is rejected.The test is sequentially repeated until the MCS is reached at a given confidence level.
5. Empirical Results
5.1. Data
I employ daily U.S. dollar-dominated returns for 42 international indices and the MSCIWorld index. I obtain total return indices of 24 developed markets (DM) from FTSE, and forthe 18 emerging markets (EM) indices from the S&P/IFCI database. The series correspondto highly liquid and investable indices, which track real returns for a foreign investor investingin the country’s equity market. The sample period is from January 2, 1996 to December31, 2017 for most of the markets with a total of 5740 days.18 The full list of countries isprovided in Table A.1 in Appendix.
Table 1 reports the descriptive statistics for the index return series. Panel A displaysinformation about the 1-day return horizon, while Panels B and C present the results for5- and 10-day horizons, respectively. The columns provide the mean and quantiles for thecross-sectional distribution of the statistics presented in rows, including the annualised mean,annualised standard deviation, skewness, kurtosis and the Jarque-Bera statistic. With theonly exception of Portugal, all markets have positive mean returns over the sample in allthe three horizons. The return series have, on average, negatively skewed and leptokurticempirical distributions. Notably, average skewness increases in absolute value with horizonswhich is in line with the findings of Neuberger (2012) and Ghysels et al. (2016). Indeed,the Jarque-Bera statistics strongly reject the null hypothesis of normality for all indices andhorizons.
5.2. Estimates of MIDAS-based models
In this paper, I am interested in the VaR and ES forecasts using two commonly usedquantiles in the literature, at α = (0.01, 0.05) probability levels, respectively. I considerthree forecast horizons: 1-day, 5-day and 10-day. The choice of 1-day horizon allows fordirect comparison of my results to the established methods in the literature, which mainlyfocus on 1-day ahead forecasts. The choice of 10-day horizon is motivated by the baselinehorizon used for the capital requirements under the Basel III regulatory agreement.19
17The MCS results with alternative block sizes (2 and 6) or the use stationary bootstrapping in A.13 givesimilar results
18The only two exceptions are Portugal ,which starts on May 04, 1998 and Russia, which starts on April02, 1997.
19Although it is possible to extend the forecasting period to longer horizons, it requires longer historicaldata to produce sufficient number of out-of-sample VaR and ES observations for reliable backtests. Thereason is that the estimation period would have been expanded significantly to ensure the convergence ofthe optimisation procedure.
15
The main focus of this study is to improve the out-of-sample performance of VaR and ESforecasts using MIDAS-based models. However, the in-sample estimation of the proposedmodels provide some noteworthy observations. For this purpose, I present the estimatedparameters of the MIDAS-based models using the first estimation window of 2500 dailyreturns. I start by the estimation results for the MSCI world index at α = 0.05.20 Next, Ifurther examine the cross-sectional variations in parameter estimates across countries. I alsocompare the weights of lagged absolute returns in the MIDAS-based specifications and thecorresponding CAViaR-based models to highlight the advantage of the proposed methods.
Table 2 presents results for the AL-based models described in Section 2.1. Columns (1)are the results for the Midas-AL model, while columns (2) are the results for the MidasAs-AL model. The row “Log-L” provides the maximised log-likelihood value of AL densitypresented in (8), while “Hit” is the empirical violation rate in the estimation sample.
I observe strong time-variation in the conditional VaR since the slope coefficients β1α,h
(β1−α,h,β
1+α,h) are always statistically significant at conventional levels. The γ coefficient govern-
ing the dynamics of conditional ES is also statistically significant across models and horizons.Not surprisingly, the negative and positive returns have different impact on the quantile dy-namics, although the asymmetry is less pronounced at longer horizons. For example, the β1+
α,h
estimate at the 1-day horizon is -0.354 and not statistically significant, whereas its value atthe 10-day horizon is 9.548 and highly significant with the magnitude almost equal to that ofβ1−α,h (-10.842). The MidasAs-AL model provides better goodness-of-fit than the symmetric
counterpart as shown by the “Log-L” values. Finally, the percentages of VaR exceedancesare always close to 5%, signalling good tail coverages for both models over the estimationperiod.
Table 3 reports the estimated parameters for the EVT-based models. Columns (1) cor-respond to the Midas-Evt model, while columns (2) refer to the MidasAs-Evt model. I alsoreport the likelihood value of (8) using estimated VaR and ES for comparison purposes,although the estimation of EVT-based models does not involve AL density maximisation.The estimation results are generally in line with those reported in Table 2. The asymmetriceffects of lagged returns become less pronounced at longer horizon. Both models have Hitpercentages close to 5% and the likelihood values are only slightly lower than their counter-parts in Table 2, which directly maximise the AL likelihood.
Tables 4 and 5 provide a summary of the cross-sectional parameter estimates for thenewly proposed models. Some observations are worth noting. First, the coefficients of neg-ative lagged returns (β1−
α,h) have greater magnitude on average than those of lagged positive
returns (β1+α,h). This finding provides evidence of asymmetric effects of lagged returns across
countries and forecast horizons. Second, the cross-sectional standard deviation of parameterκ2 is relatively more pronounced than those of other parameters, particularly at multi-dayshorizons. Although κ2 does not have a direct economic interpretation, this observation in-dicates significant variation in the shapes of the weighting function applied to the laggedconditioning variable. Since I apply the same lag length in all estimations, this finding high-lights the flexibility of the MIDAS framework in capturing significant heterogeneity in taildynamics across market indices and forecasting horizons (see, e.g., Gu and Ibragimov, 2018,
20The estimation results for α = 0.01 provide largely similar conclusions and available upon request.
16
for similar evidence of heterogeneity in the tail of international index return using the “Cubiclaw”).
One difference between the MIDAS-based model and the alternative semiparametricbenchmark, the CAViaR-based models, is the weighting scheme.21 While the MIDAS speci-fication provides a weighted average of the lagged absolute returns via the Beta polynomialsspecified in (3), the weight on past returns is capture by the autoregressive coefficient in (18)and (19). Figures 1-3 show the profiles of the estimated weighting functions for four semi-parametric models on D = 100 lagged absolute returns for 1-, 5- and 10-day forecasting hori-zons, respectively.22 The weights are normalized and sum up to one to ensure comparability.The decay functions are not significantly different at the 1-day horizon since both specifi-cations employ same frequency as the targeting forecast horizon. The weighting schemes ofthe MIDAS-based models have slightly faster decaying tails than the CAViaR-based coun-terparts, thus the formers attribute more weights to the more recent lagged observations.The differences are remarkable at the multi-day horizons. The MIDAS-based models pro-duce smoothly declining weighting schemes, whereas the CAViaR-based methods attributeweights to past observations via stepwise functions. The out-of-sample forecasting results,which will be discussed in detail later, show that the smoothing approximation by the Betapolynomial provides more accurate VaR and ES forecasts than that of the CAViaR-basedalternatives.
5.3. Out-of-Sample Forecast Evaluation
I now focus on the out-of-sample (OOS) VaR and ES forecasts from the MIDAS-basedmodels and the benchmark models presented in Section 2.2. To this end, I employ a rollingwindow approach with a fixed length of 2500 daily observations. In particular, I estimate theparameters for each model using the estimation window of 2500 daily observations and obtainthe non-overlapping VaR and ES forecasts for the next 10-day of forecasting period at allquantile levels and returns horizons. It means that for each forecasting period, I iterativelyobtain ten 1-day ahead VaR and ES forecasts for the 1-day horizon; two 5-day ahead VaRand ES forecasts for the 5-day horizon and one 10-day ahead VaR and ES forecasts for the10-day horizon, using the estimated parameters from the estimation window. Then, I movethe estimation window 10 days forward to re-estimate the model parameters and iterate thisprocedure until I reach the end of the sample. Thus, this procedure yields a total of 324OOS forecasting periods, spanning the period from August 2, 2005 to December 30, 2017.
5.3.1. Absolute Forecasting Performance
The results for VaR forecasts of competing models at the 1% and 5% quantile levels arepresented in Table 6. Panel A shows the results for the 1-day horizon, whereas Panels Band C display the results for 5- and 10-day forecast horizons, respectively. The first twocolumns present the empirical hit percentage over the OOS period. For each test, I countthe number of model rejections across the countries. Column ‘Total ’ is the sum of rejectionsacross quantile levels for each test. For example, the value of 3 for the GARCH-Fhs model
21I thank the Referee for suggesting this analysis.22To save space, I display the models that use the AL likelihood in forecasting VaR and ES. The interpre-
tation for the alternative EVT-based approaches is similar and available from the author upon request.
17
at the 1% quantile in the UC column of Panel A indicates that the 1% VaR forecasts of thismodel at the 1-day horizon are rejected by UC test in 3 out of 43 indices. Thus, for eachforecasting horizon, the best model has the lowest value in each column.
The MIDAS-based models provide competitive results to the benchmark models at the1-day horizon, but superior results at the 5- and 10-day horizons. All models, except thehistorical simulation method, perform reasonably well in the UC test at 1-day horizons andthe levels of hit percentage are close to the quantile level. At longer forecast horizons,however, most of the benchmark models significantly underestimate the risk, whereas theMIDAS-based models produce the violation rates close to the quantile levels. At 10-dayhorizon, the two MIDAS-based models with AL density provide the best performance sincethey are not rejected in any market at both quantile levels.
The results from the DQ test offer several additional insights. First, HistSim is clearlythe worst performing model in both quantile levels in all three forecasting horizons. Thisresult is similar to Nieto and Ruiz (2016) and Taylor (2019) since the HistSim method haslow variability and are highly sensitive to the extreme returns in the estimation windows.Second, the asymmetric models often provide smaller number of test rejections than thesymmetric alternatives, especially at the 1-day horizon. However, this effect is considerablyweaker at the 10-day horizon, which is in line with the in-sample estimates of the previoussubsection. Third, the performance of CAViaR-based models deteriorate significantly at the5- and 10-day forecast horizons. For instance, the 5% VaR forecast of the As-AL model isrejected in only 3 out of 43 indices at the 1-day horizon. This number rises remarkably atthe 10-day horizon, indicating that the As-AL model is rejected in 33 out of 43 markets.Finally, the MIDAS-based models consistently provide competitive performance in all threeforecasting horizons. In fact, the Midas-Evt model has the lowest number of rejections in boththe 5- and 10-day forecast horizons. The contrasting performance between MIDAS-basedand CAViaR-based models at the multi-day horizon highlights the deficiency of temporalaggregation to match target horizon in VaR forecasts and consistent with the simulationstudy in Ghysels et al. (2016).
Next, I focus on the result for ES forecasts in Table 7. In the columns, I present evaluationresults for the four ES backtests described earlier in Section 2.3. These tests include thediscrepancy test of McNeil and Frey (2000) (denoted UES1), the unconditional (UES2) andconditional (CES) tests of Du and Escanciano (2017) and the multi-VaR test of Kratz et al.(2018). Again, for each test, I report the number of model rejections across countries, whilecolumn ‘Total ’ is the sum of this number across quantile levels. Lower number in eachcolumn indicates superiority.
The results are generally in line with those in Table 6. First, all models provide acceptableresults in two unconditional ES tests with no clear superiority of one model over another,except perhaps the HistSim model that continues to underperform the alternatives. Second,similar to VaR forecasts, the models with asymmetric specification in conditional quantileyield smaller numbers of test rejection. This observation, however, is less pronounced at the5- and 10-day forecast horizons. Finally, the CAViaR-based models are clearly the worstperforming models, whereas the MIDAS-based models are superior at multi-day forecastinghorizons. Particularly in the multi-VaR test, all benchmark models are inferior to the new
18
models at 5-day and 10-day horizons23. This finding further highlights the benefit of MIDASframework in exploiting the richness of daily returns to forecast the tail dynamics at multi-day return horizons.
5.3.2. Relative Forecasting Performance
While the absolute performance evaluation is useful to validate the competing models,it provides little insight about their relative performance. Next, I investigate the relativeperformance of forecasting models based on the two loss functions presented in the previoussection. Table 8 reports the average OOS forecast losses for all models under consideration.Panel A shows results for the 1-day horizon, while Panels B and C report results for the 5- and10-day forecast horizons, respectively. In each panel, I compute the cross-sectional averageof the mean forecast losses across the 43 indices using the LQ and LFZG loss functions, andthen report them separately for the 1% and 5% quantile levels. For each column, I highlightcell with the best method.
The most accurate methods often appear in the final two rows, which correspond to theasymmetric MIDAS-based models, whereas the HistSim model provides the highest forecasterrors. The MidasAs-AL model yields the most accurate forecasts at the 1% quantile, whilethe MidasAs-Evt is the best model at 5% quantile. The only exception is the 1-day horizon,for which the GJR-Fhs model achieves the best performance. The CAViaR-based modelsalso perform well at the 1-day horizon, but their average losses rise significantly at multi-dayforecast horizons.
Table 9 presents the MCS results for the LQ and LFZG loss functions separately for eachquantile level and forecast horizons. The entry in each column counts the number of times(out of 43 indices), that the model in row is excluded from the 5% MCS. For example, theentry for LQ function of the GARCH-Fhs model at the 1% quantile level and 1-day horizonis 7. This number indicates that this model is excluded from the MCS in 7 out 43 cases.Therefore, a smaller number indicates superior performance cross-sectionally.
The main findings from the MCS results are following. First, the HistSim model isoften excluded from the MCS, especially at the 1-day horizon as this method is excluded inall the considered indices. Second, in line with the absolute performance evaluation, thereis significant benefit of using asymmetric models at 1-day horizon, but the impact is lesspronounced as the forecast horizon gets longer. Third, the MidasAs-AL model provides thebest overall performance and often be included in the set of superior models in most cases.For example, this model is never excluded from the MCS in all indices at both quantilelevels at 10-day forecasting horizon. The GARCH-based models also perform well but areoften inferior to the asymmetric MIDAS-based models. Finally, the CAViaR-based modelsperform worst at the multi-day forecast horizon and are often excluded from MCS, especiallyat the 1% quantile level.
Overall, I obtain promising results for the MIDAS-based models for VaR and ES forecasts.The proposed models consistently belong to the best performing models with low number ofrejections across backtests in all quantile levels and forecasting horizons. The new methodsalso yield the lowest forecast errors and are often included in the set of superior models,
23The only exception is the MidasAs-AL model at 5-day horizon
19
especially at forecasting horizons longer than 1-day ahead. In contrast, the alternativemodels that rely on a single-horizon returns are always inferior to all other models at multi-day forecast horizons. This finding suggests significant benefits of accounting for serialdependence in short-horizon return process to predict the tail dynamics of long-horizonreturn distribution. Finally, I also find evidence supporting the asymmetric specification inconditional quantile. In terms of ES forecasting method, the jointly model using AL densitygenerally provide better forecasts than the EVT-based alternative.
6. Robustness Checks
6.1. Model Performance and Market Regimes
The accuracy of risk measures is particularly important during periods of financial dis-tress. Thus, I evaluate model performance across different market regimes. Especially, Iseparate the OOS forecasts into three subsamples: (i) the pre-crisis period from August 2,2000 to July 31, 2007; (ii) the crisis period from August 1, 2007 to December 31, 2009; (iii)the post-crisis period from January 1, 2010 to December 31, 2017.
Tables A.3 and A.4 in Appendix report the average OOS forecast losses and MCS resultsfor the competing models for each forecasting horizon, quantile level and sub-period. Notsurprisingly, the forecast losses increase significantly during the crisis period for all models,quantile levels and forecasting horizons. This finding is in line with the recent result ofKourtis et al. (2016) in volatility forecasting. The MIDAS-based models generate similarforecast losses than GARCH-based models during crisis at 1-day and 5-day horizon, but out-perform the latter at 10-day horizon. During the pre-crisis and post-crisis sub-samples, theMIDAS-based models yield the best performance compared to all other competing models.Consistent with results of the full-sample results, the CAViaR-based forecasts often belongto the worst performing models in all sub-samples at the multi-days horizons. Finally, theMidasAs-AL model is often included in the superior set across three sub-samples, where thesuperiority is more pronounced at multi-day forecasting horizons.
6.2. Alternative Assets
My main results focus on the international equity indices. To provide further evidence, Iinvestigate model performance using alternative assets. To this end, I source stock prices of20 largest companies globally from the ”Global Top 100 companies by market capitalisation”report by PricewaterhouseCoopers (PwC) on March 3, 2018. The companies are: Apple, Mi-crosoft, Amazon.com, Tencent, Berkshire Hathaway, JPMorgan Chase, Johnson & Johnson,Exxon Mobile, Bank of America, Royal Dutch Shell, Walmart, Wells Fargo, Intel, Anheuser-Busch InBev, Taiwan Semiconductor, AT&T, Chevron, PetroChina, Novartis. The data iscollected from DataStream with the maximum available sample period from January 3, 1997to December 31, 2017.24 I also consider two alternative asset classes, including: the BarclaysU.S. Aggregate Bond Index from September 29, 2003 to December 31, 2017 as a proxy for thebond class. I also consider the S&P Goldman Sachs Commodity Total Return Index (GSCI)from January 1, 2003 to December 31, 2017 as a proxy for the commodity class. These two
24Some stocks have shorter historical length but the first observation is no later than January 1, 2005
20
indices are investable and track the return of an investor from a fully collateralised portfolioof bonds and commodities. For these two indices, I collect data from the CapitalIQ database.
Table A.5 reports the average OOS forecast losses across the considered assets. In linewith the main analysis, the MIDAS-based models provide clearly the best VaR and ESforecasts. The asymmetric models yield slightly lower forecast losses than the symmetriccounterparts. This observation is generally in line with the model confidence set resultsin Table A.6. An interesting observation is that the performance of CAViaR-based modelswith AL density are not considerably inferior to the GARCH-based models compared to theanalysis involving only stock indices.
6.3. Model Performance Between Developed and Emerging Markets
The return distributions in developed and emerging markets are typically characterisedby distinct features. Therefore, it is of interest to compare the model performance betweentwo the country groups.
Table A.7 provides the average OOS forecast losses separately for each country group.The forecast losses are substantially higher for the emerging countries in all cases. Thisobservation may be the outcome of more noisy data for the emerging stock markets. Nev-ertheless, the relative performance between competing models is consistent with the mainresults. The lowest forecast losses are often recorded in the final two rows, which correspondto the asymmetric MIDAS-based models. The MCS results in Table A.8 indicate that theasymmetric MIDAS-based model with AL density provides the best overall performance inboth country groups. Therefore, I conclude that the performance of the new models is robustto different characteristics in the return process.
6.4. Alternative Window Length
The OOS forecasts in the main analysis is conducted using rolling window of 2500 obser-vations. This choice is largely driven by the convergence rates of the CAViaR-based models.The single-horizon setting leads to substantial loss of observation for the model estimation.For example, the CAViaR-based models are optimized using only 250 non-overlapping re-turn observations at the 10-day forecast horizon. However, one may concern that using longestimation windows may give unfair advantage to the MIDAS-based methods, for example,compared to the GARCH-based models. To explore this issue, I repeat our analysis usingrolling window of 1500 and 2000 observations, respectively. In the former case, I excludethe CAViaR-based models due to their low rates of convergence. Tables A.9 and A.10 inAppendix show that my main conclusions are robust to the length of rolling windows. No-tably, the performance of EVT-based models deteriorates remarkably in shorter estimationwindows. This observation is not surprising since the numbers of extreme exceptions in thesecases are lower, which thereby increases estimation errors and reduces the goodness-of-fit inthe GPD estimation.
7. Conclusion
Using the MIDAS framework, I propose new models to directly forecast VaR and ES atthe desired horizon and quantile level. The semiparametric approach allows flexible dynam-ics in different quantile levels and avoid making distributional assumptions. In addition, the
21
MIDAS framework utilises the data-rich environment of higher frequency return process toimprove the forecast of the tail dynamics in longer horizon. Using a large cross-section ofinternational stock indices, I examine the predictive performance of the proposed modelsrelative to several popular forecasting models at various quantile levels and forecast hori-zons. Using a battery of backtesting procedures, I obtain strong evidence in favor of theproposed models, which consistently belong to the best performing methods. The MIDASframework significantly outperforms the GARCH-based models and the alternative semi-parametric models which rely on single-period quantile regression. Finally, models thatincorporate asymmetry in the quantile dynamics, and use of the AL density to jointly es-timate VaR and ES, generally provide the best forecasts across quantile levels and returnhorizons. This result is robust to different market regimes, alternative assets and forecastspecifications.
My main analysis focuses on VaR and ES forecast, given their practical importance tofinancial institutions and regulators. Given the superiority of MIDAS-based models on quan-tile forecasts, an interesting question for future research is whether the MIDAS frameworkcan also improve return density forecast or equity risk premium using the combination ofquantile forecasts. Moreover, several studies document significant explanatory powers of eco-nomic variables on conditional return distribution features such as volatility (Engle et al.,2013) or different parts of return density (Cenesizoglu and Timmermann, 2008). Thus, ad-ditional information from macroeconomic variables can further improve the forecasts of thetail dynamics. The MIDAS framework provides a suitable setting for incorporating suchvariables, which typically sampled at different frequencies. I leave such extensions to thefuture research.
22
Table 1 Descriptive Statistics of International Indices
This table reports the descriptive statistics for the cross section of log index returns. The columns show themean and quantiles from the distribution of cross-sectional statistics presented in the rows. Panel A reportsthe statistics for the 1-day horizon, while Panels B and C show the corresponding statistics for the 5- and10-day horizon, respectively. The last row in each panel reports the Jaque-Bera test statistics under the nullhypothesis of normally distributed in the return series.
Mean 5% 25% Median 75% 95%
Panel A: 1-day horizonMean 0.070 0.024 0.053 0.076 0.086 0.114Std dev 0.262 0.186 0.222 0.248 0.292 0.410Skewness -0.206 -0.736 -0.355 -0.192 -0.069 0.300Kurtosis 12.305 7.291 9.238 10.848 13.230 22.826Jarque-Bera 30544.96 4426.63 9074.24 14904.09 25029.66 106849.23
Panel B: 5-day horizonMean 0.350 0.119 0.265 0.380 0.431 0.569Std dev 0.631 0.421 0.515 0.598 0.710 1.065Skewness -0.472 -0.945 -0.695 -0.566 -0.281 0.141Kurtosis 9.162 5.383 6.228 7.917 10.363 18.562Jarque-Bera 3083.68 306.03 526.61 1163.91 2698.50 11688.43
Panel C: 10-day horizonMean 0.700 0.238 0.530 0.759 0.863 1.137Std dev 0.855 0.562 0.691 0.810 0.971 1.435Skewness -0.521 -1.181 -0.718 -0.517 -0.277 0.078Kurtosis 7.749 4.433 5.342 6.557 8.915 16.746Jarque-Bera 909.35 57.40 147.94 316.02 930.40 4735.75
23
Table 2 Estimation of AL-based Models at 5% quantile for the MSCI World Index
This table provides estimated parameters of two AL-based models under the MIDAS framework for the 5%quantile level for the MSCI World index. The results are presented for 1-, 5- and 10-day return horizons. Theparameters are estimated using the first moving window with 2500 observations. Columns (1) are the resultsfor the Midas-AL model, while Columns (2) are the results for the MidasAs-AL model, which specify theconditional quantile in (1) and (2), respectively. The numbers in parentheses below the estimated parametersare p-values, based on bootstrapped standard errors. For parameter κ2, the null hypothesis is κ2 = 1. Therow Log-L reports the maximised log-likelihood value of AL distribution described in (8), while the row Hit(%) denotes the percentage of times the VaR is exceeded.
1-day horizon 5-day horizon 10-day horizon
Model (1) (2) (1) (2) (1) (2)
-0.003 -0.004 -0.012 -0.016 -0.036 -0.045β0α,h (0.000) (0.000) (0.000) (0.000) (0.000) (0.000)
-1.743 -2.706 -4.265 -7.321 -1.865 -10.842(0.000) (0.000) (0.000) (0.000) (0.007) (0.000)
-0.354 0.966 9.548β1α,h
β1−α,h
β1+α,h
(0.049) (0.088) (0.000)
8.523 7.147 4.968 3.060 20.039 2.613κ2 (0.000) (0.000) (0.000) (0.011) (0.034) (0.000)
-1.064 -1.162 -1.228 -0.959 -0.878 -1.081γ
(0.000) (0.000) (0.000) (0.000) (0.013) (0.000)Log-L 7092.82 7179.12 931.39 945.11 388.72 406.47
Hit (%) 4.833 4.750 5.000 5.000 5.000 4.583
24
Table 3 Estimation of EVT-based Models at 5% quantile for the MSCI World Index
This table provides estimated parameters of two EVT-based models under the MIDAS framework for the 5%quantile level for the MSCI World index. The results are presented for 1-, 5- and 10-day return horizons. Theparameters are estimated using the first moving window with 2500 observations. Columns (1) are the resultsfor the Midas-Evt model, while Columns (2) are the results for the MidasAs-Evt model, which specify theconditional quantile in (1) and (2), respectively. The numbers in parentheses below the estimated parametersare p-values, based on bootstrapped standard errors. For parameter κ2, the null hypothesis is κ2 = 1. Therow Log-L reports the maximised log-likelihood value of AL distribution described in (8), while the row Hit(%) denotes the percentage of times the VaR is exceeded.
1-day horizon 5-day horizon 10-day horizon
Model (1) (2) (1) (2) (1) (2)
-0.002 -0.004 -0.011 -0.016 -0.031 -0.033β0α,h (0.001) (0.000) (0.003) (0.000) (0.003) (0.000)
-1.625 -2.726 -3.201 -7.124 -1.375 -9.564(0.000) (0.000) (0.000) (0.000) (0.077) (0.000)
0.035 2.116 6.545β1α,h
β1−α,h
β1+α,h
(0.160) (0.031) (0.000)
8.608 6.073 5.230 2.777 18.960 2.557κ2 (0.000) (0.000) (0.000) (0.002) (0.000) (0.027)ξ 0.085 0.185 -0.156 -0.227 0.064 0.053β 0.349 0.294 0.467 0.520 0.585 0.380
Log-L 7000.13 7166.88 931.06 943.69 386.45 404.36Hit (%) 5.125 5.250 5.000 5.000 5.417 4.583
25
Table 4 Cross-sectional Estimates of AL-based Models at the 5% quantile
This table provides the average of estimated parameters across countries of the AL-based models at the5% quantile level. Results are reported at 1-day, 5-day and 10-day return horizons, respectively. Theparameters are estimated using the first moving window of 2500 observations. Columns (1) are the resultsfor the Midas-AL model, while Columns (2) are the results for the MidasAs-AL model, which specify theconditional quantile in (1) and (2), respectively. The numbers in parentheses display cross-sectional standarddeviation of the above parameters.
1-day horizon 5-day horizon 10-day horizon
Model (1) (2) (1) (2) (1) (2)
-0.006 -0.008 -0.018 -0.021 -0.031 -0.034β0α,h (0.003) (0.004) (0.018) (0.019) (0.038) (0.032)
-1.674 -2.296 -3.908 -5.705 -5.089 -9.298(0.278) (0.380) (1.555) (2.541) (3.232) (5.675)
-0.660 -1.678 -0.325β1α,h
β1−α,h
β1+α,h
(0.424) (2.144) (6.731)
12.606 14.234 21.294 14.901 32.993 12.892κ2 (7.252) (7.097) (52.672) (24.515) (65.669) (24.757)
-0.914 -0.984 -0.959 -0.961 -1.094 -1.196γ
(0.185) (0.182) (0.228) (0.218) (0.356) (0.377)
26
Table 5 Cross-sectional Estimates of EVT-based Models at the 5% quantile
This table provides the average of estimated parameters across countries of the Evt-based models at the5% quantile level. Results are reported at 1-day, 5-day and 10-day return horizons, respectively. Theparameters are estimated using the first moving window of 2500 observations. Columns (1) are the resultsfor the Midas-Evt model, while Columns (2) are the results for the MidasAs-Evt model, which specify theconditional quantile in (1) and (2), respectively. The numbers in parentheses display cross-sectional standarddeviation of the above parameters.
1-day horizon 5-day horizon 10-day horizon
Model (1) (2) (1) (2) (1) (2)
-0.005 -0.006 -0.014 -0.018 -0.021 -0.035β0α,h (0.004) (0.004) (0.017) (0.016) (0.032) (0.032)
-1.436 -2.157 -3.432 -4.708 -4.445 -6.723(0.288) (0.422) (1.367) (2.114) (2.526) (4.324)
-0.450 -1.415 -0.327β1α,h
β1−α,h
β1+α,h
(0.397) (1.887) (3.800)
11.826 12.823 17.905 19.390 17.246 22.111κ2 (7.527) (7.474) (53.094) (40.973) (49.149) (56.400)
0.067 0.074 -0.007 0.011 -0.043 0.066ξ
(0.123) (0.109) (0.256) (0.208) (0.362) (0.397)0.434 0.407 0.497 0.476 0.512 0.423
ς(0.057) (0.048) (0.172) (0.138) (0.197) (0.208)
27
Table 6 Results of Out-of-Sample VaR Absolute Forecasting Performance
This table summarises the performance of out-of-sample VaR forecasts across 43 international equity indices. Forecasts are
based on rolling window of 2500 observations. Panel A provides the results for the 1-day horizon, while Panels B and C
reports the results for the 5- and 10-day forecast horizons, respectively. The columns labelled Hit(%) report the percentage
of times the VaR estimates are exceeded. The next six columns display the absolute performance of VaR forecasts, based on
the unconditional coverage test (UC) of Kupiec (1995) and the dynamic quantile test (DQ) of Engle and Manganelli (2004).
For each test in column, I report the number of test rejections out of 43 indices at 5% significant level. Lower number implies
superior performance.
Hit(%) UC DQ
Models 1% 5% 1% 5% Total 1% 5% Total
Panel A: 1-day horizonHistSim 1.031 4.618 14 26 40 43 43 86GARCH-Fhs 1.065 5.021 3 1 4 15 20 35GARCH-Evt 0.997 5.064 0 0 0 14 19 33GJR-Fhs 1.063 4.966 3 5 8 11 7 18GJR-Evt 0.996 5.054 2 4 6 4 5 9Sav-AL 1.074 4.994 3 1 4 21 21 42Sav-Evt 1.042 5.188 1 1 2 21 24 45As-AL 1.069 4.908 5 4 9 12 3 15As-Evt 1.020 5.024 0 3 3 14 5 19Midas-AL 1.030 4.950 2 1 3 14 21 35Midas-Evt 1.021 5.119 1 0 1 19 24 43MidasAs-AL 1.042 4.897 4 4 8 10 4 14MidasAs-Evt 0.998 4.975 2 4 6 11 5 16
Panel B: 5-day horizonHistSim 1.000 4.375 3 12 15 38 39 77GARCH-Fhs 1.529 5.951 7 6 13 15 11 26GARCH-Evt 1.513 5.920 7 10 17 13 10 23GJR-Fhs 1.389 5.628 4 4 8 9 7 16GJR-Evt 1.342 5.520 4 3 7 8 4 12Sav-AL 1.195 5.030 0 0 0 22 24 46Sav-Evt 1.267 5.104 0 0 0 16 17 33As-AL 1.237 4.946 2 4 6 27 15 42As-Evt 1.269 5.075 1 0 1 19 12 31Midas-AL 1.023 4.706 1 1 2 11 5 16Midas-Evt 1.012 4.808 0 1 1 7 4 11MidasAs-AL 0.924 4.675 1 2 3 12 4 16MidasAs-Evt 1.015 4.782 0 1 1 8 5 13
Panel C: 10-day horizonHistSim 1.079 4.409 3 5 8 24 28 52GARCH-Fhs 1.514 5.692 3 2 5 11 5 16GARCH-Evt 1.514 5.641 2 2 4 13 3 16GJR-Fhs 1.181 5.307 0 3 3 5 5 10GJR-Evt 1.188 5.276 1 3 4 6 5 11Sav-AL 1.261 6.265 3 3 6 21 34 55Sav-Evt 1.557 5.579 5 1 6 23 17 40As-AL 1.329 6.530 3 10 13 15 33 48As-Evt 1.659 5.548 8 1 9 20 14 34Midas-AL 1.061 4.811 0 0 0 8 1 9Midas-Evt 1.079 4.427 0 1 1 7 1 8MidasAs-AL 0.918 4.913 0 0 0 8 2 10MidasAs-Evt 1.188 4.676 1 1 2 13 2 15
28
Table 7 Results of Out-of-Sample ES Absolute Forecasting Performance
This table summarises the performance of out-of-sample ES forecasts across 43 international equity indices. Forecasts are based
on rolling window of 2500 observations. Panel A provides results for the 1-day horizon, while Panels B and C reports results for
the 5- and 10-day forecast horizons, respectively. The next six columns display the absolute performance of ES forecasts, based
on the unconditional ES test of zero discrepancy (UES1) of McNeil and Frey (2000), the unconditional (UES2) and conditional
ES (CES) tests of Du and Escanciano (2017), the multi-VaR test of Kratz et al. (2018). For each test in column, I report the
number of test rejections out of 43 indices at 5% significant level. Lower number implies superior performance.
UES1 UES2 CES
Models 1% 5% Total 1% 5% Total 1% 5% Total MultiVaR
Panel A: 1-day horizonHistSim 1 8 9 15 23 38 41 43 84 17GARCH-Fhs 1 1 2 2 1 3 16 35 51 1GARCH-Evt 0 0 0 1 1 2 16 33 49 2GJR-Fhs 1 0 1 1 3 4 7 12 19 4GJR-Evt 2 1 3 3 2 5 8 12 20 4Sav-AL 0 1 1 4 0 4 22 42 64 3Sav-Evt 1 1 2 4 1 5 23 41 64 5As-AL 1 2 3 8 3 11 11 15 26 4As-Evt 1 2 3 6 4 10 11 8 19 4Midas-AL 0 1 1 5 0 5 25 40 65 3Midas-Evt 1 0 1 4 0 4 20 41 61 3MidasAs-AL 0 2 2 4 2 6 11 10 21 1MidasAs-Evt 1 2 3 3 2 5 10 10 20 3
Panel B: 5-day horizon horizonHistSim 0 0 0 4 7 11 35 43 78 6GARCH-Fhs 1 1 2 3 4 7 1 4 5 8GARCH-Evt 1 0 1 4 5 9 0 4 4 9GJR-Fhs 3 2 5 3 3 6 1 4 5 7GJR-Evt 2 2 4 0 2 2 1 2 3 5Sav-AL 4 1 5 2 0 2 22 34 56 2Sav-Evt 0 0 0 0 0 0 7 28 35 2As-AL 3 3 6 13 0 13 15 13 28 3As-Evt 2 0 2 3 0 3 11 14 25 4Midas-AL 0 0 0 1 0 1 5 8 13 1Midas-Evt 0 0 0 0 0 0 4 4 8 1MidasAs-AL 1 0 1 2 1 3 4 5 9 3MidasAs-Evt 1 2 3 2 1 3 3 4 7 1
Panel C: 10-day horizon horizonHistSim 0 2 2 4 6 10 17 35 52 4GARCH-Fhs 1 0 1 0 3 3 2 2 4 6GARCH-Evt 1 0 1 0 3 3 3 2 5 4GJR-Fhs 1 0 1 1 1 2 2 4 6 3GJR-Evt 0 0 0 2 1 3 2 3 5 3Sav-AL 3 2 5 9 0 9 16 16 32 7Sav-Evt 0 1 1 3 2 5 12 13 25 6As-AL 4 6 10 17 2 19 11 13 24 8As-Evt 0 1 1 3 2 5 9 10 19 8Midas-AL 0 1 1 1 0 1 4 4 8 1Midas-Evt 0 0 0 4 1 5 2 2 4 1MidasAs-AL 0 2 2 0 1 1 3 2 5 1MidasAs-Evt 0 3 3 6 2 8 3 3 6 1
29
Tab
le8
Su
mm
ary
of
Ou
t-of-
Sam
ple
Fore
cast
Loss
es
Th
ista
ble
pro
vid
esth
eav
erag
eou
t-of
-sam
ple
fore
cast
loss
esat
the
1%
an
d5%
qu
anti
lele
vel
sfo
r1-,
5-
an
d10-d
ayhori
zon
s,re
spec
tive
ly.LQ
den
ote
sth
equ
anti
lelo
ssfu
nct
ion
of(2
3)an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
eLQ
an
dLFZG
valu
esare
mu
ltip
lied
by
104
and
103,
resp
ecti
vely
,to
faci
lita
tepre
senta
tion
.L
ower
valu
esco
rres
pon
dto
sup
erio
rp
erfo
rman
ce.
Bold
nu
mb
ers
ind
icate
bes
tm
eth
od
sin
each
colu
mn
.
Pan
elA
:1-
day
hori
zon
Pan
elB
:5-d
ayh
ori
zon
Pan
elC
:10-d
ayh
ori
zon
1%5%
1%
5%
1%
5%
Mod
els
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
His
tSim
0.69
33.
465
2.028
2.1
96
1.7
03
8.2
80
4.8
60
5.1
63
2.2
94
10.9
45
6.4
57
6.7
66
GA
RC
H-F
hs
0.48
92.
447
1.684
1.8
21
1.2
85
6.2
78
4.1
38
4.3
87
1.7
01
8.1
47
5.7
35
5.9
88
GA
RC
H-E
vt
0.49
02.
448
1.684
1.8
21
1.2
74
6.2
23
4.1
26
4.3
73
1.6
98
8.1
32
5.7
35
5.9
88
GJR
-Fh
s0.4
83
2.4
15
1.668
1.8
04
1.2
68
6.1
92
4.1
44
4.3
91
1.7
17
8.2
13
5.7
78
6.0
31
GJR
-Evt
0.48
42.
416
1.667
1.8
03
1.2
52
6.1
15
4.1
21
4.3
67
1.7
15
8.2
01
5.7
82
6.0
36
Sav
-AL
0.50
02.
498
1.702
1.8
40
1.4
69
7.1
46
4.4
73
4.7
43
2.2
40
10.7
08
6.4
20
6.7
33
Sav
-Evt
0.50
02.
499
1.703
1.8
42
1.4
37
6.9
99
4.4
18
4.6
84
2.1
46
10.8
62
6.2
03
6.6
61
As-
AL
0.48
92.
445
1.670
1.8
06
1.3
94
6.7
90
4.4
24
4.6
92
2.1
84
10.4
24
6.4
92
6.8
13
As-
Evt
0.48
42.
420
1.668
1.8
04
1.3
69
6.6
82
4.3
36
4.5
97
2.0
71
10.6
62
6.1
18
6.5
70
Mid
as-A
L0.
497
2.48
71.
701
1.8
39
1.2
75
6.2
15
4.1
87
4.4
38
1.7
36
8.3
11
5.8
84
6.1
44
Mid
as-E
vt
0.49
82.
489
1.700
1.8
39
1.2
84
6.2
82
4.1
41
4.3
90
1.7
63
8.6
35
5.6
91
5.9
85
Mid
asA
s-A
L0.
485
2.42
61.
670
1.8
06
1.2
30
5.9
92
4.1
43
4.3
91
1.6
21
7.7
59
5.7
08
5.9
63
Mid
asA
s-E
vt
0.48
52.
424
1.6
67
1.8
03
1.2
63
6.2
15
4.0
87
4.3
36
1.7
57
8.6
65
5.6
77
5.9
79
30
Tab
le9
Mod
el
Con
fid
en
ce
Set
Resu
lts
Th
ista
ble
rep
orts
the
resu
lts
ofth
e5%
Mod
elC
on
fid
ence
Set
(MC
S)
at
the
1%
an
d5%
qu
anti
lele
vels
for
1-,
5-
an
d10-d
ayfo
reca
sth
ori
zon
s,re
spec
tive
ly.
Th
een
try
inea
chco
lum
np
rese
nts
the
nu
mb
erof
tim
esou
tof
43
ind
ices
,th
at
the
mod
elin
row
isex
clu
ded
from
the
5%
MC
S.LQ
den
otes
the
qu
anti
lelo
ssfu
nct
ion
of(2
3)an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
era
nge
stati
stic
in(2
5)
isu
sed
toth
eeq
uiv
alen
cete
stof
the
MC
S.
Low
erva
lues
corr
esp
on
ds
tosu
per
ior
per
form
an
ce.
Bold
nu
mb
ers
ind
icate
bes
tm
eth
od
sin
each
colu
mn
.
1-d
ayhor
izon
5-d
ayh
ori
zon
10-d
ayh
ori
zon
1%5%
1%
5%
1%
5%
Mod
els
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
His
tSim
4343
43
43
11
12
15
15
21
17
13
14
Gar
ch-F
hs
77
13
14
33
10
34
22
Gar
ch-E
vt
67
13
14
33
10
33
11
GJR
-Fh
s2
22
22
21
06
62
2G
JR
-Evt
22
22
22
10
76
22
Sav
-AL
98
18
19
98
77
20
19
77
Sav
-Evt
1010
20
21
88
65
13
19
37
As-
AL
55
01
76
54
20
18
99
As-
Evt
44
11
66
22
13
19
25
Mid
as-A
L7
717
19
44
43
77
55
Mid
as-E
vt
109
17
18
55
11
48
00
Mid
asA
s-A
L0
02
31
11
10
00
0M
idas
As-
Evt
44
33
22
00
36
00
31
Figure 1 The weights of the lagged absolute returns in the MIDAS-based specification and the correspondingCAViaR-based specification for the World portfolio returns at 1-day horizon at 5% quantile
0.00
0.02
0.04
0.06
0.08
0 25 50 75 100Lags
Wei
ghts
Models
Sav−ALAs−ALMidas−ALMidasAs−AL
32
Figure 2 The weights of the lagged absolute returns in the MIDAS-based specification and the correspondingCAViaR-based specification for the World portfolio returns at 5-day horizon at 5% quantile
0.00
0.01
0.02
0.03
0.04
0.05
0 25 50 75 100Lags
Wei
ghts
Models
Sav−ALAs−ALMidas−ALMidasAs−AL
33
Figure 3 The weights of the lagged absolute returns in the MIDAS-based specification and the correspondingCAViaR-based specification for the World portfolio returns at 10-day horizon at 5% quantile
0.00
0.05
0.10
0.15
0.20
0 25 50 75 100Lags
Wei
ghts
Models
Sav−ALAs−ALMidas−ALMidasAs−AL
34
References
Anatolyev, S. and Petukhov, A. (2016). Uncovering the skewness news impact curve. Journalof Financial Econometrics, 14(4):746–771.
Andreou, E., Ghysels, E., and Kourtellos, A. (2011). Forecasting with Mixed-FrequencyData. In Clements, M. P. and Hendry, D. F., editors, The Oxford Handbook of EconomicForecasting, pages 1–24.
Andreou, E., Ghysels, E., and Kourtellos, A. (2013). Should Macroeconomic Forecasters UseDaily Financial Data and How? Journal of Business and Economic Statistics, 31(2):240–251.
Artzner, P., Delbaen, F., Eber, J.-M., and Heath, D. (1999). Coherent Measures of risk.Mathematical Finance, 9(3):203–228.
Bams, D., Blanchard, G., and Lehnert, T. (2017). Volatility measures and Value-at-Risk.International Journal of Forecasting, 33(4):848–863.
Banulescu, D., Hurlin, C., Leymarie, J., and Scaillet, O. (2019). Backtesting MarginalExpected Shortfall and Related Systemic Risk Measures. Working Papers.
Barone-Adesi, G., Giannopoulos, K., and Vosper, L. (1999). VaR without correlations forportfolios of derivative securities. Journal of Futures Markets, 19(5):583–602.
Basel Committee on Banking Supervision (2016). Minimum capital requirements for marketrisk. Technical report.
Bassett, G. W., Koenker, R., and Kordas, G. (2004). Pessimistic Portfolio Allocation andChoquet Expected Utility. Journal of Financial Econometrics, 2(4):477–492.
Berkowitz, J., Christoffersen, P., and Pelletier, D. (2011). Evaluating Value-at-Risk Modelswith Desk-Level Data. Management Science, 57(12):2213–2227.
Bollerslev, T. (1987). A Conditionally Hetroskedastic Time Series Model for SpeculativePrices and Rates of Return. Review of Economics and Statistics, 69(3):542–547.
Boucher, C. M., Danıelsson, J., Kouontchou, P. S., and Maillet, B. B. (2014). Risk Models-at-risk. Journal of Banking and Finance, 44:72–92.
Brownlees, C. T., Engle, R. F., and Kelly, B. T. (2011). A Practical Guide to VolatilityForecasting through Calm and Storm. The Journal of Risk, 14(2):3–22.
Cenesizoglu, T. and Timmermann, A. G. (2008). Is the Distribution of Stock ReturnsPredictable? Working Papers.
Chernov, M., Gallant, A. R., Ghysels, E., and Tauchen, G. (2003). Alternative models forstock price dynamics. Journal of Econometrics, 116(1-2):225–257.
35
Chernozhukov, V., Fernandez-Val, I., and Galichon, A. (2010). Quantile and ProbabilityCurves Without Crossing. Econometrica, 78(3):1093–1125.
Corsi, F. (2009). A simple approximate long-memory model of realized volatility. Journalof Financial Econometrics, 7(5):174–196.
Du, Z. and Escanciano, J. C. (2017). Backtesting Expected Shortfall : Accounting for TailRisk. Management Science, 63(04):940–958.
Engle, R. F. (2011). Long-term skewness and systemic risk. Journal of Financial Economet-rics, 9(3):437–468.
Engle, R. F., Ghysels, E., and Sohn, B. (2013). Stock Market Volatility and MacroeconomicFundamentals. Review of Economics and Statistics, 95(3):776–797.
Engle, R. F. and Manganelli, S. (2004). CAViaR: Conditional autoregressive value at riskby regression quantiles. Journal of Business and Economic Statistics, 22(4):367–381.
Fama, E. F. and French, K. R. (2018). Long-Horizon Returns. Review of Asset PricingStudies, 8(2):232–252.
Feunou, B., Jahan-Parvar, M. R., and Tedongap, R. (2016). Which parametric model forconditional skewness? European Journal of Finance, 22(13):1237–1271.
Fissler, T. and Ziegel, J. F. (2016). Higher order elicitability and osband’s principle. Annalsof Statistics, 44(4):1680–1707.
Fissler, T., Ziegel, J. F., and Gneiting, T. (2015). Expected Shortfall is jointly elicitablewith Value at Risk - Implications for backtesting. Risk, pages 1–7.
Foroni, C., Guerin, P., and Marcellino, M. (2018). Using low frequency information forpredicting high frequency variables. International Journal of Forecasting, 34(4):774–787.
Ghysels, E., Plazzi, A., and Valkanov, R. (2016). Why Invest in Emerging Markets? TheRole of Conditional Return Asymmetry. Journal of Finance, 71(5):2145–2192.
Ghysels, E., Santa-Clara, P., and Valkanov, R. (2004). The MIDAS touch: mixed datasampling regressions.
Ghysels, E., Santa-Clara, P., and Valkanov, R. (2006). Predicting volatility: Getting themost out of return data sampled at different frequencies. Journal of Econometrics, 131(1-2):59–95.
Ghysels, E., Sinko, A., and Valkanov, R. (2007). MIDAS regressions: Further results andnew directions. Econometric Reviews, 26(1):53–90.
Giacomini, R. and Komunjer, I. (2005). Evaluation and Combination of Conditional QuantileForecasts. Journal of Business & Economic Statistics, 23(4):416–431.
36
Giannopoulos, K. and Tunaru, R. (2005). Coherent risk measures under filtered historicalsimulation. Journal of Banking and Finance, 29(4):979–996.
Glosten, L. R., Jagannathan, R., and Runkle, D. E. (1993). On the Relation between theExpected Value and the Volatility of the Nominal Excess Return on Stocks. Journal ofFinance, 48(5):1779–1801.
Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Sta-tistical Association, 106(494):746–762.
Gu, Z. and Ibragimov, R. (2018). The “Cubic Law of the Stock Returns” in emergingmarkets. Journal of Empirical Finance, 46(September 2017):182–190.
Hansen, P. R., Lunde, A., and Nason, J. M. (2011). The Model Confidence Set. Economet-rica, 79(2):453–497.
Koenker, R. and Machado, J. A. F. (1999). Goodness of Fit and Related Inference Processesfor Quantile Regression. Journal of the American Statistical Association, 94(448):1296–1310.
Kourtis, A., Markellos, R. N., and Symeonidis, L. (2016). An International Comparisonof Implied, Realized, and GARCH Volatility Forecasts. Journal of Futures Markets,36(12):1164–1193.
Kratz, M., Lok, Y. H., and McNeil, A. J. (2018). Multinomial VaR backtests: A simpleimplicit approach to backtesting expected shortfall. Journal of Banking and Finance,88:393–407.
Kuester, K., Mittnik, S., and Paolella, M. S. (2006). Value-at-risk prediction: A comparisonof alternative strategies. Journal of Financial Econometrics, 4(1):53–89.
Kupiec, P. H. (1995). Techniques for Verifying the Accuracy of Risk Measurement Models.Journal of Derivatives, 3(2):73–84.
Lima, L. R. and Meng, F. (2017). Out-of-Sample Return Predictability: A Quantile Combi-nation Approach. Journal of Applied Econometrics, 32(4):877–895.
Manganelli, S. and Engle, R. F. (2004). A Comparison of Value-at-Risk Models in Finance.In Szego, G., editor, Risk Measures for the 21st Century, pages 123–144. Wiley.
McNeil, A. J. and Frey, R. (2000). Estimation of tail-related risk measures for heteroscedasticfinancial time series: an extreme value approach. Journal of Empirical Finance, 7(3-4):271–300.
Meng, X. and Taylor, J. W. (2018). An approximate long-memory range-based approach forvalue at risk estimation. International Journal of Forecasting, 34(3):377–388.
Nass, C. (1959). The χ2 Test for Small Expectations in Contingency Tables , with SpecialReference to Accidents and Absenteeism. Biometrika, 46(3):365–385.
37
Neuberger, A. (2012). Realized skewness. Review of Financial Studies, 25(11):3424–3455.
Nieto, M. R. and Ruiz, E. (2016). Frontiers in VaR Forecasting and Backtesting. Interna-tional Journal of Forecasting, 32(2):475–501.
Novales, A. and Garcia-Jorcano, L. (2018). Backtesting extreme value theory models ofexpected shortfall. Quantitative Finance, 7688:1–27.
Pettenuzzo, D., Timmermann, A. G., and Valkanov, R. (2016). A MIDAS approach tomodeling first and second moment dynamics. Journal of Econometrics, 193(2):315–334.
Taylor, J. W. (2019). Forecasting Value at Risk and Expected Shortfall Using a Semipara-metric Approach Based on the Asymmetric Laplace Distribution. Journal of Business andEconomic Statistics, 37(1):1–13.
Theodossiou, P. (2015). Skewed Generalized Error Distribution of Financial Assets andOption Pricing. Multinational Finance Journal, 19(4):223–266.
38
Appendix A. ONLINE APPENDIX
A.1. List of Index and Forecasting Models
Table A.1 List of VaR and ES Forecasting Models
This table summarizes the competing forecasting models for VaR and ES under consideration.
Abbreviation Description
Benchmark Models
HistSim VaR and ES are estimated as the empirical quantiles and the average of realizedviolations over the moving windows of 2500 observations.
GARCH-Fhs VaR and ES are extracted from the GARCH model of Bollerslev (1987), assuminga SGE distribution (Theodossiou, 2015) for daily returns. Empirical distributionis approximated using filter historical simulation with 10,000 trials.
GARCH-Evt VaR and ES are extracted from the GARCH model of Bollerslev (1987), assuminga SGE distribution (Theodossiou, 2015) for daily returns. Empirical distributionis approximated by combining filter historical simulation and EVT with 10,000trials.
GJR-Fhs VaR and ES are extracted from the GJR-GARCH model of Glosten et al. (1993),assuming a SGE distribution (Theodossiou, 2015) for daily returns. Empiricaldistribution is approximated using filter historical simulation with 10,000 trials.
GJR-Evt VaR and ES are extracted from the GJR-GARCH model of Glosten et al. (1993),assuming a SGE distribution (Theodossiou, 2015) for daily returns. Empiricaldistribution is approximated by combining filter historical simulation and EVTwith 10,000 trials.
Sav-AL VaR and ES are jointly estimated using maximum likelihood of AL density. VaRfollows symmetric absolute value specification in (18), while ES dynamic followsspecification in (9).
Sav-Evt Conditional quantile at threshold level of 7.5% is estimated using CAViaR modelwith symmetric absolute value specification in (18). VaR and ES are jointlycomputed using the results of McNeil and Frey (2000).
As-AL VaR and ES are jointly estimated using maximum likelihood of AL density. VaRfollows asymmetric slope specification in (19), while ES dynamic follows specifi-cation in (9).
As-Evt Conditional quantile at threshold level of 7.5% is estimated using CAViaR modelwith asymmetric slope specification in (19). VaR and ES are jointly computedusing the results of McNeil and Frey (2000).
New Models
Midas-AL VaR and ES are jointly estimated using maximum likelihood of AL density. VaRfollows MIDAS-based symmetric absolute value specification in (1), while ES dy-namic follows specification in (9).
Midas-Evt Conditional quantile at threshold level of 7.5% is estimated using MIDAS quantileregression with symmetric absolute value specification in (1). VaR and ES arejointly computed using the results of McNeil and Frey (2000).
MidasAs-AL VaR and ES are jointly estimated using maximum likelihood of AL density. VaRfollows MIDAS-based asymmetric slope specification in (2), while ES dynamicfollows specification in (9).
MidasAs-Evt Conditional quantile at threshold level of 7.5% is estimated using MIDAS quantileregression with asymmetric slope specification in (2). VaR and ES are jointlycomputed using the results of McNeil and Frey (2000).
39
Table A.2 List of Country Index and Sources
Country Source
World World Portfolio MSCIDeveloped Markets Australia, Austria, Belgium, Canada,
Denmark, Finland, France, Germany,The Netherlands, Hongkong, Ireland,Israel, Italia, Japan, South Korea, NewZealand, Norway, Portugal, Singapore,Spain, Sweden, Switzerland, UnitedKingdom, United States
FTSE
Emerging Markets Brazil, Chile, China, Czech Republic,Hungary, India, Indonesia, Malaysia,Mexico, Pakistan, Peru, Philippines,Poland, Russia, South Africa, Taiwan,Thailand, Turkey
S&P/IFCI
40
A.2
.R
obu
stn
ess
Che
cks:
Mar
ket
Reg
imes
Tab
leA
.3O
ut-
of-
Sam
ple
Fore
cast
Loss
es
-D
iffere
nt
Mark
et
Regim
es
Th
ista
ble
pro
vid
esth
eav
erag
eou
t-of
-sam
ple
fore
cast
loss
esfo
rd
iffer
ent
mark
etre
gim
esat
the
1%
an
d5%
quanti
lele
vels
for
1-,
5-
an
d10-d
ayh
oriz
ons,
resp
ecti
vely
.LQ
den
otes
the
qu
anti
lelo
ssfu
nct
ion
of
(23)
an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
eLQ
andLFZG
valu
esar
em
ult
ipli
edby
104
and
103
,re
spec
tive
ly,
tofa
cili
tate
pre
senta
tion
.L
ower
valu
esco
rres
pon
dto
sup
erio
rp
erfo
rman
ce.
Bold
nu
mb
ers
ind
icat
eb
est
met
hod
sin
each
colu
mn
.
1-d
ayh
oriz
on
5-d
ayh
ori
zon
10-d
ayh
ori
zon
1%5%
1%
5%
1%
5%
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
Panel
A:02/08/2005-31/07/2007(P
re-crisisSubsample)
His
tSim
0.52
12.
572
1.637
1.7
31
1.3
39
6.3
47
4.3
41
4.4
13
1.9
20
8.7
70
6.0
38
5.9
36
GA
RC
H-F
hs
0.42
62.
095
1.431
1.5
07
1.2
60
5.9
93
4.0
49
4.1
02
1.7
60
8.1
01
5.404
5.274
GA
RC
H-E
vt
0.42
52.
092
1.430
1.5
05
1.2
37
5.8
82
3.9
85
4.0
33
1.7
65
8.1
26
5.4
10
5.2
81
GJR
-Fh
s0.4
16
2.0
47
1.418
1.4
92
1.2
48
5.9
32
4.0
69
4.1
23
1.7
93
8.2
55
5.4
91
5.3
66
GJR
-Evt
0.41
72.
051
1.4
17
1.4
91
1.2
21
5.8
01
3.9
88
4.0
36
1.7
94
8.2
66
5.4
89
5.3
65
Sav
-AL
0.43
62.
147
1.449
1.5
27
1.2
52
5.9
50
4.1
19
4.1
78
1.8
24
8.3
85
5.7
14
5.6
05
Sav
-Evt
0.43
72.
152
1.452
1.5
29
1.2
73
6.0
68
4.0
71
4.1
26
1.8
40
9.0
19
5.5
64
5.5
24
As-
AL
0.42
22.
074
1.425
1.5
00
1.2
54
5.9
62
4.1
40
4.2
02
2.0
27
9.4
01
6.0
54
5.9
79
As-
Evt
0.42
12.
071
1.423
1.4
98
1.2
73
6.0
65
4.0
95
4.1
52
2.1
70
10.8
23
5.9
04
6.0
22
Mid
as-A
L0.
433
2.13
11.
448
1.5
25
1.1
37
5.3
84
3.9
89
4.0
36
1.6
02
7.3
53
5.6
08
5.4
85
Mid
as-E
vt
0.43
22.
127
1.447
1.5
24
1.2
24
5.8
14
3.9
76
4.0
23
1.6
66
7.8
18
5.3
94
5.3
19
Mid
asA
s-A
L0.
417
2.05
11.
425
1.5
00
1.1
20
5.3
02
4.0
01
4.0
49
1.5
59
7.1
37
5.6
38
5.5
28
Mid
asA
s-E
vt
0.42
12.
070
1.422
1.4
97
1.1
94
5.6
71
3.9
38
3.9
82
1.7
89
8.3
77
5.4
52
5.3
49
Panel
B:01/08/2007-31/12/2009(C
risisSubsample)
His
tSim
1.26
76.
328
3.435
3.7
45
3.1
19
15.1
38
8.1
50
8.7
55
4.0
18
19.2
05
10.5
97
11.2
92
GA
RC
H-F
hs
0.72
13.
612
2.553
2.7
84
1.8
73
9.1
95
6.0
79
6.5
39
2.3
17
11.1
28
8.3
10
8.8
40
GA
RC
H-E
vt
0.72
03.
609
2.555
2.7
86
1.8
55
9.1
05
6.0
53
6.5
11
2.3
00
11.0
50
8.3
07
8.8
36
GJR
-Fh
s0.
711
3.56
32.5
30
2.7
59
1.8
39
9.1
19
6.1
31
6.5
90
2.4
21
11.5
80
8.5
41
9.0
79
GJR
-Evt
0.71
03.
558
2.532
2.7
61
1.8
06
8.8
69
6.0
81
6.5
37
2.4
13
11.5
36
8.5
35
9.0
71
Sav
-AL
0.74
93.
755
2.591
2.8
25
2.5
85
12.5
25
7.4
41
7.9
84
4.0
16
19.2
46
11.0
42
11.8
01
Sav
-Evt
0.75
13.
761
2.597
2.8
32
2.5
12
12.1
87
7.2
44
7.7
71
3.9
69
19.4
72
10.3
67
11.1
06
As-
AL
0.72
93.
653
2.545
2.7
76
2.2
83
11.1
10
7.1
78
7.7
13
3.6
79
17.5
88
10.9
80
11.7
65
As-
Evt
0.7
09
3.5
55
2.538
2.7
68
2.2
35
10.8
84
6.8
55
7.3
56
3.4
78
17.1
41
9.7
06
10.4
20
41
Tab
leA
.3(continued)
1-d
ayh
oriz
on
5-d
ayh
ori
zon
10-d
ayh
ori
zon
1%5%
1%
5%
1%
5%
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
Mid
as-A
L0.
744
3.72
72.
588
2.8
22
1.8
97
9.4
09
6.3
16
6.7
86
2.5
63
12.2
62
8.8
08
9.3
69
Mid
as-E
vt
0.74
53.
733
2.590
2.8
25
1.9
23
9.4
06
6.2
00
6.6
63
2.6
79
13.1
19
8.5
06
9.0
90
Mid
asA
s-A
L0.
724
3.62
72.
542
2.7
73
1.8
30
8.9
83
6.0
93
6.6
03
2.4
12
11.5
06
8.2
88
8.7
94
Mid
asA
s-E
vt
0.71
23.
570
2.537
2.7
66
1.8
97
9.3
12
6.0
69
6.5
29
2.6
52
12.9
38
8.5
24
9.0
99
Panel
C:01/01/2010-31/12/2017(P
ost-crisisSubsample)
His
tSim
0.56
02.
807
1.692
1.8
36
1.3
57
6.6
44
3.9
80
4.2
46
1.8
56
8.9
40
5.2
93
5.5
82
GA
RC
H-F
hs
0.43
52.
179
1.481
1.6
05
1.1
13
5.4
65
3.5
75
3.8
06
1.4
96
7.2
36
5.0
27
5.2
89
GA
RC
H-E
vt
0.43
52.
183
1.481
1.6
05
1.1
07
5.4
34
3.5
79
3.8
11
1.4
95
7.2
31
5.0
28
5.2
90
GJR
-Fh
s0.
430
2.15
61.
467
1.5
89
1.1
04
5.4
13
3.5
63
3.7
94
1.4
84
7.1
72
5.0
03
5.2
63
GJR
-Evt
0.43
02.
157
1.465
1.5
87
1.0
93
5.3
61
3.5
63
3.7
93
1.4
82
7.1
65
5.0
12
5.2
72
Sav
-AL
0.43
92.
203
1.493
1.6
17
1.1
85
5.8
10
3.6
59
3.8
97
1.8
04
8.6
94
5.1
89
5.4
69
Sav
-Evt
0.43
92.
201
1.493
1.6
17
1.1
51
5.6
54
3.6
47
3.8
85
1.6
66
8.6
98
5.0
92
5.5
86
As-
AL
0.43
32.
170
1.464
1.5
86
1.1
60
5.6
89
3.6
60
3.8
97
1.7
71
8.5
14
5.2
35
5.5
12
As-
Evt
0.43
12.
162
1.463
1.5
85
1.1
31
5.5
65
3.6
35
3.8
71
1.6
19
8.6
59
5.0
75
5.5
29
Mid
as-A
L0.
438
2.19
81.
492
1.6
17
1.1
11
5.4
52
3.5
91
3.8
24
1.5
17
7.3
40
5.0
57
5.3
20
Mid
as-E
vt
0.43
92.
200
1.491
1.6
16
1.1
06
5.4
57
3.5
61
3.7
94
1.5
08
7.4
69
4.9
04
5.1
99
Mid
asA
s-A
L0.4
30
2.1
54
1.465
1.5
87
1.0
64
5.2
18
3.5
73
3.8
04
1.3
96
6.7
54
4.9
07
5.1
59
Mid
asA
s-E
vt
0.43
12.
163
1.4
62
1.5
84
1.0
89
5.4
14
3.5
24
3.7
58
1.4
76
7.4
32
4.8
59
5.1
77
42
Table A.4 Model Confidence Set - Different Market Regimes
This table reports the results of the 5% Model Confidence Set (MCS) for different market regimes atthe 1% and 5% quantile levels for 1-, 5- and 10-day forecast horizons, respectively. The entry in eachcolumn presents the number of times out of 43 indices, that the model in row is excluded from the 5%MCS. LQ denotes the quantile loss function of (23) and LFZG is the FZG loss function of Fissler et al.(2015) given in (24). The range statistic in (25) is used to the equivalence test of the MCS. Lowervalues corresponds to superior performance.
1-day horizon 5-day horizon 10-day horizon
1% 5% 1% 5% 1% 5%
LQ LQS LQ LQS LQ LQS LQ LQS LQ LQS LQ LQS
Panel A: 02/08/2000 - 31/07/2008 (Pre-crisis Subsample)HistSim 25 25 31 31 22 22 12 12 33 32 19 20Garch-Fhs 7 8 7 7 10 11 3 3 25 23 3 4Garch-Evt 7 8 4 4 7 7 2 2 26 24 4 5GJR-Fhs 2 3 1 1 12 11 5 5 35 36 13 14GJR-Evt 2 3 1 1 12 11 4 4 36 35 13 14Sav-AL 14 15 10 10 19 19 5 5 33 33 14 14Sav-Evt 15 15 9 9 16 17 6 6 24 24 10 10As-AL 5 6 4 4 16 15 7 7 35 35 16 17As-Evt 3 3 3 3 16 16 6 6 34 34 13 15Midas-AL 13 14 9 9 11 12 7 6 19 19 9 10Midas-Evt 14 13 6 7 15 16 2 2 27 28 5 6MidasAs-AL 6 6 5 5 7 7 3 3 13 13 5 6MidasAs-Evt 3 4 2 1 10 10 2 2 28 27 3 4
Panel B: 01/08/2008 - 31/12/2009 (Crisis Subsample)HistSim 14 14 28 28 16 15 5 5 9 9 2 2Garch-Fhs 2 2 1 1 4 3 0 0 9 10 1 1Garch-Evt 3 3 2 3 4 4 0 0 8 8 1 1GJR-Fhs 1 1 0 0 4 4 1 0 11 12 0 1GJR-Evt 2 2 0 0 4 4 1 1 10 11 1 1Sav-AL 4 4 1 1 19 17 5 5 16 17 3 2Sav-Evt 4 4 3 3 18 16 7 7 9 11 3 3As-AL 2 2 1 1 10 9 5 5 14 15 4 3As-Evt 4 4 0 0 6 6 5 5 9 11 1 1Midas-AL 2 2 2 2 9 9 1 0 10 10 3 2Midas-Evt 3 3 2 3 13 12 2 2 9 11 1 1MidasAs-AL 1 1 0 0 2 2 1 1 2 2 0 0MidasAs-Evt 4 4 2 2 6 6 0 0 7 8 1 1
Panel C: 01/01/2010 - 31/12/2017 (Crisis Subsample)HistSim 42 42 43 43 35 35 26 26 36 35 15 14Garch-Fhs 3 3 11 11 6 7 2 2 6 8 3 1Garch-Evt 3 2 13 12 7 7 2 2 6 7 3 1GJR-Fhs 0 0 2 3 5 5 3 3 15 15 4 1GJR-Evt 4 3 4 4 5 5 3 3 15 15 4 2Sav-AL 8 7 19 19 18 18 6 6 34 33 6 5Sav-Evt 5 4 17 17 10 11 4 4 22 28 6 5As-AL 5 4 2 3 17 16 7 7 34 34 9 7As-Evt 3 2 5 5 11 12 3 3 24 32 3 6Midas-AL 6 5 18 17 10 10 5 5 15 16 8 6Midas-Evt 6 5 16 16 11 12 1 1 15 14 0 0MidasAs-AL 3 2 2 3 1 2 2 2 2 2 2 0MidasAs-Evt 2 1 5 5 8 10 1 1 11 17 1 0
43
A.3
.R
obu
stn
ess
Che
cks:
Alt
ern
ativ
eA
sset
s
Tab
leA
.5O
ut-
of-
Sam
ple
Fore
cast
Loss
es
-A
ltern
ati
ve
Ass
ets
Th
ista
ble
pro
vid
esth
eav
erag
eou
t-of
-sam
ple
fore
cast
loss
esfo
ralt
ern
ati
veass
ets
at
the
1%
an
d5%
qu
anti
lele
vels
for
1-,
5-
an
d10-d
ayh
ori
zon
s,re
spec
tive
ly.
Th
eal
tern
ativ
eas
sets
are
defi
ned
inS
ecti
on
4.2
.LQ
den
ote
sth
equ
anti
lelo
ssfu
nct
ion
of
23
an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal
.(2
015)
give
nin
(24)
.T
heLQ
andLFZG
valu
esare
mu
ltip
lied
by
104
an
d103,
resp
ecti
vel
y,to
faci
lita
tep
rese
nta
tion
.L
ower
valu
esco
rres
pon
dto
sup
erio
rp
erfo
rman
ce.
Bol
dnu
mb
ers
ind
icat
eb
est
met
hod
sin
each
colu
mn
.
Pan
elA
:1-
day
hori
zon
Pan
elB
:5-d
ayh
ori
zon
Panel
C:
10-d
ayh
ori
zon
1%5%
1%
5%
1%
5%
Mod
els
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
His
tSim
0.73
63.
656
2.0
33
2.1
90
1.4
75
7.1
08
4.2
12
4.4
24
2.0
40
9.6
22
5.7
91
5.9
62
GA
RC
H-F
hs
0.52
22.
594
1.7
01
1.8
27
1.0
73
5.1
72
3.7
36
3.9
04
1.4
42
6.7
63
5.1
27
5.2
35
GA
RC
H-E
vt
0.52
42.
605
1.7
02
1.8
28
1.0
78
5.1
95
3.7
22
3.8
90
1.464
6.8
87
5.1
06
5.2
12
GJR
-Fh
s0.
523
2.60
11.6
91
1.8
16
1.0
96
5.2
74
3.7
34
3.9
02
1.4
78
6.9
60
5.1
28
5.2
36
GJR
-Evt
0.51
72.
568
1.6
91
1.8
17
1.0
98
5.2
94
3.7
30
3.8
98
1.4
73
6.9
31
5.1
26
5.2
37
Sav
-AL
0.51
72.
558
1.6
72
1.7
96
1.1
51
5.5
38
3.7
40
3.9
08
1.6
45
7.7
42
5.2
67
5.3
87
Sav
-Evt
0.52
12.
589
1.6
95
1.8
21
1.1
39
5.4
96
3.8
42
4.0
18
1.760
8.6
11
5.5
38
5.7
26
As-
AL
0.51
52.
541
1.6
48
1.7
70
1.0
82
5.2
21
3.7
06
3.8
73
1.561
7.3
76
5.1
45
5.2
55
As-
Evt
0.51
82.
575
1.6
84
1.8
09
1.1
12
5.3
90
3.8
30
4.0
08
1.6
26
8.0
95
5.3
07
5.5
38
Mid
as-A
L0.
513
2.54
81.6
86
1.8
11
1.0
44
5.0
37
3.5
97
3.7
56
1.376
6.4
94
4.9
17
5.0
21
Mid
as-E
vt
0.51
92.
580
1.6
93
1.8
18
1.0
67
5.1
44
3.6
56
3.8
18
1.436
6.9
80
5.0
11
5.1
39
Mid
asA
s-A
L0.5
08
2.5
23
1.6
68
1.7
92
1.0
35
5.0
04
3.5
65
3.7
21
1.3
24
6.2
72
4.8
36
4.9
37
Mid
asA
s-E
vt
0.51
72.
568
1.6
80
1.8
04
1.0
42
5.0
32
3.6
30
3.7
90
1.4
32
6.9
87
5.0
13
5.1
76
44
Tab
leA
.6M
od
el
Con
fid
en
ce
Set
Resu
lt-
Alt
ern
ati
ve
Ass
ets
Th
ista
ble
rep
orts
the
resu
lts
ofth
e5%
Mod
elC
on
fid
ence
Set
(MC
S)
for
alt
ern
ati
veass
ets
at
the
1%
an
d5%
qu
anti
lele
vel
sfo
r1-,
5-
an
d10-d
ayfo
reca
sth
oriz
ons,
resp
ecti
vely
.T
he
alte
rnat
ive
ass
ets
are
defi
ned
inS
ecti
on
4.2
.T
he
entr
yin
each
colu
mn
pre
sents
the
nu
mb
erof
tim
esou
tof
43
ind
ices
,th
atth
em
odel
inro
wis
excl
ud
edfr
omth
e5%
MC
S.LQ
den
ote
sth
equ
anti
lelo
ssfu
nct
ion
of
(23)
an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal
.(2
015)
give
nin
(24)
.T
he
ran
ge
stati
stic
in(2
5)
isu
sed
toth
eeq
uiv
ale
nce
test
of
the
MC
S.
Low
erva
lues
corr
esp
on
ds
tosu
per
ior
per
form
ance
.
Pan
elA
:1-
day
hori
zon
Pan
elB
:5-d
ayh
ori
zon
Pan
elC
:10-d
ayh
ori
zon
1%5%
1%
5%
1%
5%
Mod
els
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
His
tSim
2021
22
22
20
20
15
14
21
20
15
14
GA
RC
H-F
hs
44
15
15
35
33
910
33
GA
RC
H-E
vt
44
15
15
34
33
11
11
33
GJR
-Fh
s2
213
13
35
43
13
12
33
GJR
-Evt
22
15
15
56
33
1313
33
Sav
-AL
22
66
99
11
10
83
3S
av-E
vt
44
14
14
910
77
17
17
78
As-
AL
11
44
35
22
97
22
As-
Evt
22
14
14
57
43
13
12
33
Mid
as-A
L1
112
12
33
00
87
11
Mid
as-E
vt
33
14
15
44
22
1413
12
Mid
asA
s-A
L0
07
82
21
17
50
0M
idas
As-
Evt
11
12
12
46
11
78
11
45
A.4
.R
obu
stn
ess
Che
cks:
Diff
eren
tC
oun
try
Gro
ups
Tab
leA
.7O
ut-
of-
Sam
ple
Fore
cast
Loss
es
-D
iffere
nt
Cou
ntr
yG
rou
ps
Th
ista
ble
pro
vid
esth
eaver
age
ou
t-of-
sam
ple
fore
cast
loss
esfo
rd
iffer
ent
cou
ntr
ygro
up
sat
the
1%
an
d5%
qu
anti
lele
vel
sfo
r1-,
5-
an
d10-d
ay
hori
zon
s,re
spec
tivel
y.LQ
den
ote
sth
equ
anti
lelo
ssfu
nct
ion
of
23
andLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
eLQ
an
dLFZG
valu
esare
mu
ltip
lied
by
104
an
d103,
resp
ecti
vel
y,to
faci
lita
tep
rese
nta
tion
.L
ow
ervalu
esco
rres
pon
dto
sup
erio
rp
erfo
rman
ce.
Bold
nu
mb
ers
ind
icate
bes
tm
eth
od
sin
each
colu
mn
.
1-d
ayhori
zon
5-d
ayhori
zon
10-d
ayhori
zon
1%
5%
1%
5%
1%
5%
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
Panel
A:DevelopedStock
Markets
His
tSim
0.6
81
3.4
08
1.9
90
2.1
58
1.6
27
7.9
43
4.5
96
4.8
98
2.1
74
10.4
36
6.0
86
6.4
04
GA
RC
H-F
hs
0.4
65
2.3
30
1.6
25
1.7
60
1.2
18
5.9
72
3.8
78
4.1
24
1.5
73
7.5
80
5.4
15
5.6
77
GA
RC
H-E
vt
0.4
65
2.3
29
1.6
25
1.7
60
1.2
03
5.8
96
3.8
74
4.1
19
1.5
74
7.5
84
5.4
19
5.6
82
GJR
-Fhs
0.4
63
2.3
18
1.6
10
1.7
43
1.2
14
5.9
47
3.8
91
4.1
37
1.6
08
7.7
30
5.4
85
5.7
50
GJR
-Evt
0.4
63
2.3
17
1.6
10
1.7
43
1.1
91
5.8
34
3.8
69
4.1
14
1.6
02
7.7
02
5.4
87
5.7
52
Sav
-AL
0.4
72
2.3
62
1.6
35
1.7
71
1.3
78
6.7
26
4.1
57
4.4
21
2.1
46
10.2
91
5.9
72
6.2
87
Sav
-Evt
0.4
73
2.3
70
1.6
36
1.7
72
1.3
44
6.5
67
4.1
14
4.3
75
1.9
90
10.1
46
5.7
80
6.2
61
As-
AL
0.4
67
2.3
37
1.6
08
1.7
41
1.3
10
6.4
01
4.1
36
4.4
02
2.0
82
9.9
93
6.1
46
6.4
79
As-
Evt
0.462
2.314
1.6
06
1.7
39
1.2
92
6.3
14
4.0
57
4.3
14
1.9
52
9.9
35
5.7
16
6.1
32
Mid
as-
AL
0.4
70
2.3
52
1.6
34
1.7
70
1.1
99
5.8
64
3.9
27
4.1
75
1.5
82
7.6
13
5.5
27
5.7
95
Mid
as-
Evt
0.4
71
2.3
59
1.6
33
1.7
69
1.2
12
5.9
41
3.8
85
4.1
31
1.6
27
7.9
02
5.3
24
5.5
99
Mid
asA
s-A
L0.4
65
2.3
28
1.6
08
1.7
41
1.160
5.676
3.8
72
4.1
17
1.461
7.043
5.323
5.582
Mid
asA
s-E
vt
0.4
62
2.3
15
1.605
1.738
1.2
02
5.9
12
3.833
4.078
1.6
12
7.9
20
5.3
41
5.6
36
Panel
B:EmergingStock
Markets
His
tSim
0.7
15
3.5
70
2.0
83
2.2
53
1.7
98
8.7
02
5.2
05
5.5
08
2.4
05
11.4
02
6.9
09
7.2
03
GA
RC
H-F
hs
0.5
23
2.6
11
1.7
67
1.9
07
1.3
82
6.7
22
4.4
92
4.7
43
1.8
38
8.7
53
6.1
26
6.3
63
GA
RC
H-E
vt
0.5
24
2.6
15
1.7
67
1.9
08
1.3
74
6.6
86
4.4
65
4.7
15
1.8
29
8.7
11
6.1
20
6.3
58
GJR
-Fhs
0.512
2.554
1.7
51
1.8
90
1.3
48
6.5
59
4.4
93
4.7
42
1.8
35
8.7
36
6.1
35
6.3
72
GJR
-Evt
0.5
12
2.5
56
1.750
1.888
1.3
41
6.5
24
4.4
64
4.7
11
1.8
39
8.7
54
6.1
45
6.3
82
Sav
-AL
0.5
37
2.6
82
1.7
93
1.9
36
1.5
89
7.6
94
4.8
94
5.1
72
2.3
45
11.1
70
6.9
60
7.2
63
Sav
-Evt
0.5
36
2.6
73
1.7
96
1.9
39
1.5
59
7.5
67
4.8
27
5.1
00
2.3
27
11.6
74
6.7
19
7.1
42
As-
AL
0.5
20
2.5
93
1.7
59
1.8
98
1.5
04
7.2
92
4.8
11
5.0
82
2.2
56
10.7
12
6.8
73
7.1
73
As-
Evt
0.5
14
2.5
67
1.7
56
1.8
95
1.4
70
7.1
67
4.7
06
4.9
70
2.2
08
11.5
63
6.5
99
7.0
97
Mid
as-
AL
0.5
35
2.6
70
1.7
92
1.9
35
1.3
66
6.6
36
4.5
29
4.7
81
1.8
95
9.0
14
6.2
90
6.5
36
Mid
as-
Evt
0.5
34
2.6
64
1.7
93
1.9
36
1.3
76
6.7
25
4.4
84
4.7
36
1.8
95
9.3
25
6.1
38
6.4
49
Mid
asA
s-A
L0.5
13
2.5
60
1.7
58
1.8
97
1.312
6.368
4.5
02
4.7
52
1.790
8.515
6.1
71
6.4
16
Mid
asA
s-E
vt
0.5
16
2.5
74
1.7
55
1.8
94
1.3
46
6.6
24
4.431
4.684
1.9
06
9.4
10
6.074
6.372
46
Tab
leA
.8M
od
el
Con
fid
en
ce
Set
-D
iffere
nt
Cou
ntr
yG
rou
p
Th
ista
ble
rep
ort
sth
ere
sult
sof
the
5%
Mod
elC
on
fid
ence
Set
(MC
S)
for
diff
eren
tco
untr
ygro
up
sat
the
1%
an
d5%
qu
anti
lele
vel
sfo
r1-,
5-
an
d10-d
ay
fore
cast
hori
zon
s,
resp
ecti
vel
y.T
he
entr
yin
each
colu
mn
pre
sents
the
nu
mb
erof
tim
esou
tof
43
ind
ices
,th
at
the
mod
elin
row
isex
clu
ded
from
the
5%
MC
S.LQ
den
ote
sth
equ
anti
lelo
ss
fun
ctio
nof
(23)
an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
era
nge
stati
stic
in(2
5)
isu
sed
toth
eeq
uiv
ale
nce
test
of
the
MC
S.
Low
ervalu
es
corr
esp
on
ds
tosu
per
ior
per
form
an
ce.
1-d
ayhori
zon
5-d
ayhori
zon
10-d
ayhori
zon
1%
5%
1%
5%
1%
5%
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
Panel
A:DevelopedStock
Markets
His
tSim
24
24
24
24
66
77
10
96
7G
AR
CH
-Fhs
11
76
22
00
34
22
GA
RC
H-E
vt
21
66
11
00
34
11
GJR
-Fhs
00
11
11
00
45
11
GJR
-Evt
00
11
11
00
45
11
Sav
-AL
11
10
10
65
22
15
16
55
Sav
-Evt
32
11
11
66
11
811
44
As-
AL
11
00
55
22
12
13
44
As-
Evt
00
00
33
11
610
12
Mid
as-
AL
11
910
22
22
66
22
Mid
as-
Evt
22
88
44
11
55
00
Mid
asA
s-A
L0
00
00
01
10
00
0M
idasA
s-E
vt
00
11
11
00
23
00
Panel
B:EmergingStock
Markets
His
tSim
18
18
18
18
56
88
11
87
7G
AR
CH
-Fhs
66
87
11
10
00
00
GA
RC
H-E
vt
55
77
22
10
00
00
GJR
-Fhs
22
11
11
10
22
00
GJR
-Evt
22
11
11
10
22
00
Sav
-AL
77
98
33
55
43
12
Sav
-Evt
87
10
93
35
41
51
2A
s-A
L4
40
02
23
24
43
3A
s-E
vt
44
11
33
11
26
01
Mid
as-
AL
66
98
22
21
11
32
Mid
as-
Evt
87
10
91
10
00
30
1M
idasA
s-A
L0
02
21
10
00
00
0M
idasA
s-E
vt
44
22
11
00
14
00
47
A.5
.R
obu
stn
ess
Che
cks:
Alt
ern
ativ
eE
stim
atio
nW
indo
ws
Tab
leA
.9O
ut-
of-
Sam
ple
Fore
cast
Loss
es
-A
ltern
ati
ve
Est
imati
on
Win
dow
s
Th
ista
ble
pro
vid
esth
eaver
age
ou
t-of-
sam
ple
fore
cast
loss
esfo
ralt
ern
ati
ve
rollin
gw
ind
ow
len
gth
at
the
1%
an
d5%
qu
anti
lele
vel
sfo
r1-,
5-
an
d10-d
ay
hori
zon
s,re
spec
tivel
y.
LQ
den
ote
sth
equ
anti
lelo
ssfu
nct
ion
of
23
an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
eLQ
an
dLFZG
valu
esare
mu
ltip
lied
by
104
an
d103,
resp
ecti
vel
y,to
faci
lita
tep
rese
nta
tion
.L
ow
ervalu
esco
rres
pon
dto
sup
erio
rp
erfo
rman
ce.
Bold
nu
mb
ers
ind
icate
bes
tm
eth
od
sin
each
colu
mn
.
Panel
A:
1-d
ayhori
zon
Panel
B:
5-d
ayhori
zon
Panel
C:
10-d
ayhori
zon
1%
5%
1%
5%
1%
5%
Model
sLQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
Panel
A:1500-observtionEstim
ationW
indow
His
tSim
0.6
53
3.2
59
1.9
54
2.1
08
1.6
36
7.9
13
4.7
29
4.9
83
2.1
85
10.3
81
6.3
84
6.6
10
GA
RC
H-F
hs
0.4
82
2.4
06
1.6
46
1.7
72
1.2
53
6.0
89
4.0
90
4.2
97
1.6
86
8.0
11
5.7
21
5.8
94
GA
RC
H-E
vt
0.4
81
2.4
00
1.6
45
1.7
72
1.2
48
6.0
63
4.0
80
4.2
86
1.6
74
7.9
59
5.706
5.879
GJR
-Fhs
0.5
12
2.5
55
1.7
12
1.8
44
1.3
28
6.4
46
4.2
42
4.4
59
1.8
44
8.7
72
5.8
88
6.0
76
GJR
-Evt
0.4
74
2.3
64
1.6
30
1.7
54
1.229
5.967
4.0
71
4.2
76
1.7
12
8.1
27
5.7
61
5.9
35
Mid
as-
AL
0.4
81
2.4
00
1.6
47
1.7
74
1.2
67
6.1
46
4.0
51
4.2
55
1.7
02
8.1
03
5.7
71
5.9
50
Mid
as-
Evt
0.4
98
2.4
82
1.6
71
1.8
00
1.3
29
6.4
82
4.2
32
4.4
56
1.9
03
9.5
78
6.0
27
6.3
67
Mid
asA
s-A
L0.467
2.329
1.616
1.740
1.2
45
6.0
40
4.022
4.225
1.664
7.909
5.7
21
5.8
87
Mid
asA
s-E
vt
0.4
83
2.4
11
1.6
42
1.7
67
1.3
51
6.6
40
4.2
62
4.4
99
1.9
95
10.1
87
6.0
77
6.4
37
Panel
B:2000-observationEstim
ationW
indow
His
tSim
0.6
65
3.3
20
1.9
69
2.1
27
1.6
63
8.0
56
4.7
48
5.0
14
2.2
08
10.4
78
6.3
73
6.6
17
GA
RC
H-F
hs
0.4
82
2.4
05
1.6
42
1.7
70
1.2
55
6.1
07
4.0
56
4.2
71
1.6
58
7.8
99
5.622
5.811
GA
RC
H-E
vt
0.4
80
2.3
96
1.6
42
1.7
70
1.2
54
6.0
99
4.0
48
4.2
62
1.6
80
7.9
99
5.6
29
5.8
20
GJR
-Fhs
0.5
08
2.5
36
1.7
02
1.8
35
1.3
21
6.4
22
4.2
06
4.4
31
1.8
20
8.6
68
5.8
13
6.0
17
GJR
-Evt
0.4
74
2.3
64
1.6
26
1.7
52
1.2
37
6.012
4.0
51
4.2
65
1.6
95
8.0
65
5.6
82
5.8
74
Sav
-AL
0.4
84
2.4
01
1.8
88
2.0
39
1.4
04
6.8
05
4.4
53
4.6
97
2.0
03
9.5
55
6.0
11
6.2
30
Sav
-Evt
0.4
81
2.3
98
1.6
28
1.7
55
1.3
48
6.5
44
4.1
77
4.3
99
1.9
33
9.8
09
5.7
54
6.0
87
As-
AL
0.4
72
2.3
42
1.8
96
2.0
49
1.2
83
6.2
21
4.5
12
4.7
63
1.8
94
9.0
37
6.0
20
6.2
42
As-
Evt
0.4
68
2.3
34
1.598
1.722
1.2
75
6.2
05
4.0
55
4.2
71
1.8
55
9.6
14
5.6
23
6.0
67
Mid
as-
AL
0.4
81
2.3
99
1.6
46
1.7
75
1.2
76
6.1
97
4.0
35
4.2
49
1.6
72
7.9
80
5.7
20
5.9
16
Mid
as-
Evt
0.4
99
2.4
88
1.6
71
1.8
02
1.3
36
6.5
25
4.2
22
4.4
56
1.8
84
9.5
28
5.9
60
6.3
27
Mid
asA
s-A
L0.467
2.331
1.6
16
1.7
41
1.251
6.0
76
4.009
4.221
1.646
7.848
5.6
76
5.8
68
Mid
asA
s-E
vt
0.4
85
2.4
18
1.6
41
1.7
69
1.3
60
6.6
97
4.2
60
4.5
10
1.9
87
10.2
01
6.0
48
6.4
40
48
Tab
leA
.10
Mod
el
Con
fid
en
ce
Set
-A
ltern
ati
ve
Est
imati
on
Win
dow
s
Th
ista
ble
rep
orts
the
resu
lts
ofth
e5%
Mod
elC
on
fid
ence
Set
(MC
S)
for
alt
ern
ati
vele
ngth
sof
roll
ing
win
dow
at
the
1%
an
d5%
qu
anti
lele
vels
for
1-,
5-an
d10
-day
fore
cast
hor
izon
s,re
spec
tive
ly.
Th
een
try
inea
chco
lum
np
rese
nts
the
nu
mb
erof
tim
esou
tof
43
ind
ices
,th
at
the
mod
elin
row
isex
clu
ded
from
the
5%M
CS
.LQ
den
otes
the
qu
anti
lelo
ssfu
nct
ion
of
(23)
an
dLFZG
isth
eF
ZG
loss
funct
ion
of
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
era
nge
stat
isti
cin
(25)
isu
sed
toth
eeq
uiv
alen
cete
stof
the
MC
S.
Low
erva
lues
corr
esp
on
ds
tosu
per
ior
per
form
an
ce.
1-d
ayh
oriz
on
5-d
ayh
ori
zon
10-d
ayh
ori
zon
1%5%
1%
5%
1%
5%
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
LQ
LQS
Panel
A:1500-observtionEstim
ationWindow
His
tSim
4343
43
43
30
30
25
25
26
27
99
GA
RC
H-F
hs
1010
17
18
00
00
34
11
GA
RC
H-E
vt
87
15
15
11
11
34
00
GJR
-Fh
s17
1727
27
33
88
89
01
GJR
-Evt
55
77
00
00
89
11
Mid
as-A
L8
817
17
22
00
56
00
Mid
as-E
vt
2827
32
33
11
13
17
17
1321
10
15
Mid
asA
s-A
L0
00
00
00
01
12
1M
idas
As-
Evt
1312
20
20
14
16
20
20
1228
917
Panel
B:2000-observationEstim
ationWindow
His
tSim
4343
43
43
23
25
32
33
31
31
20
18
GA
RC
H-F
hs
64
28
27
22
31
22
00
GA
RC
H-E
vt
77
28
29
11
21
32
00
GJR
-Fh
s18
1730
30
44
99
54
22
GJR
-Evt
22
19
19
11
13
22
11
Sav
-AL
87
43
43
89
13
15
17
15
24
Sav
-Evt
99
17
17
55
88
917
22
As-
AL
32
43
43
11
14
14
97
44
As-
Evt
22
55
11
11
716
32
Mid
as-A
L5
530
30
12
00
22
11
Mid
as-E
vt
2524
40
40
913
14
16
1219
10
13
Mid
asA
s-A
L0
011
10
00
11
11
00
Mid
asA
s-E
vt
910
30
30
12
14
19
21
1226
10
14
49
A.6
.R
obu
stn
ess
Che
cks:
Alt
ern
ativ
eM
CS
proc
edu
res
Tab
leA
.11
Mod
el
Con
fid
en
ce
Set
Resu
lts
-10%
MC
S
Th
ista
ble
rep
orts
the
resu
lts
ofth
e10
%M
od
elC
on
fid
ence
Set
(MC
S)
at
the
1%
an
d5%
qu
anti
lele
vels
for
1-,
5-
an
d10-d
ayfo
reca
sth
ori
zon
s,
resp
ecti
vely
.T
he
entr
yin
each
colu
mn
pre
sents
the
nu
mb
erof
tim
esou
tof
43
ind
ices
,th
at
the
mod
elin
row
isex
clu
ded
from
the
5%
MC
S.LQ
den
otes
the
qu
anti
lelo
ssfu
nct
ion
of(2
3)an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
era
nge
stati
stic
in(2
5)
isu
sed
toth
eeq
uiv
alen
cete
stof
the
MC
S.
Low
erva
lues
corr
esp
on
ds
tosu
per
ior
per
form
an
ce.
1-day
hor
izon
5-day
hor
izon
10-d
ayhor
izon
1%5%
1%5%
1%5%
Model
sLQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
His
tSim
4343
4343
1516
2121
2728
1716
GA
RC
H-F
hs
1111
1718
55
10
45
23
GA
RC
H-E
vt
1111
1718
44
10
44
12
GJR
-Fhs
44
34
22
10
98
23
GJR
-Evt
44
34
22
10
109
33
Sav
-AL
1313
2627
1312
1210
2827
1212
Sav
-Evt
1414
2729
1110
1110
2026
713
As-
AL
109
22
1011
97
2727
1315
As-
Evt
66
22
77
76
1828
510
Mid
as-A
L13
1324
254
45
310
113
4M
idas
-Evt
1314
2323
66
11
912
12
Mid
asA
s-A
L2
14
41
12
10
00
0M
idas
As-
Evt
77
77
33
00
78
01
50
Tab
leA
.12
Mod
el
Con
fid
en
ce
Set
-A
ltern
ati
ve
Eli
min
ati
on
Ru
le
Th
ista
ble
rep
orts
the
resu
lts
ofth
e5%
Mod
elC
on
fid
ence
Set
(MC
S)
at
the
1%
an
d5%
qu
anti
lele
vels
for
1-,
5-
an
d10-d
ayfo
reca
sth
ori
zon
s,
resp
ecti
vely
.T
he
entr
yin
each
colu
mn
pre
sents
the
nu
mb
erof
tim
esou
tof
43
ind
ices
,th
at
the
mod
elin
row
isex
clu
ded
from
the
5%
MC
S.LQ
den
otes
the
qu
anti
lelo
ssfu
nct
ion
of(2
3)an
dLFZG
isth
eF
ZG
loss
fun
ctio
nof
Fis
sler
etal.
(2015)
giv
enin
(24).
Th
eu
sin
gth
ese
mi-
qu
ad
rati
c
stat
isti
cin
Han
sen
etal
.(2
011)
isu
sed
toth
eeq
uiv
ale
nce
test
of
the
MC
S.
Low
erva
lues
corr
esp
on
ds
tosu
per
ior
per
form
an
ce.
Pan
elA
:1-
day
Pan
elB
:5-
day
Pan
elC
:10
-day
1%5%
1%5%
1%5%
Model
sLQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
LQ
LFZG
His
tSim
4343
4343
99
1112
811
46
GA
RC
H-F
hs
66
33
11
00
22
11
GA
RC
H-E
vt
55
44
11
00
22
11
GJR
-Fhs
11
11
11
00
23
11
GJR
-Evt
33
11
11
00
23
11
Sav
-AL
98
1415
33
44
810
55
Sav
-Evt
98
1313
33
23
38
14
As-
AL
33
11
23
22
1111
34
As-
Evt
33
00
22
20
311
14
Mid
as-A
L6
614
151
10
02
30
0M
idas
-Evt
77
1414
11
00
13
00
Mid
asA
s-A
L1
10
00
00
00
00
0M
idas
As-
Evt
33
00
11
00
13
00
51
Table A.13 Model Confidence Set - Alternative Bootstrapping Methods
This table reports the results of the 5% Model Confidence Set (MCS) at the 1% and 5% quantile levelsfor 1-, 5- and 10-day forecast horizons, respectively. The entry in each column presents the number oftimes out of 43 indices, that the model in row is excluded from the 5% MCS. LQ denotes the quantileloss function of (23) and LFZG is the FZG loss function of Fissler et al. (2015) given in (24). Therange statistic in (25) is used to the equivalence test of the MCS. The test statistic is constructed usingalternative bootstrapping methods. Lower values corresponds to superior performance.
1-day horizon 5-day horizon 10-day horizon
1% 5% 1% 5% 1% 5%
LQ LQS LQ LQS LQ LQS LQ LQS LQ LQS LQ LQS
Panel A: Use of stationary bootstrapping
HistSim 43 43 43 43 10 10 16 16 13 16 9 10Garch-Fhs 6 6 3 3 0 0 0 0 2 2 1 1Garch-Evt 5 5 4 4 0 0 0 0 2 2 1 1GJR-Fhs 1 1 1 1 1 1 0 0 3 3 1 1GJR-Evt 3 3 1 1 1 1 0 0 3 3 1 1Sav-AL 10 9 17 16 6 6 7 6 13 12 6 7Sav-Evt 9 9 14 15 5 5 3 3 4 11 4 8As-AL 3 3 1 1 2 2 4 4 10 11 4 5As-Evt 3 3 0 0 1 1 2 2 6 13 1 5Midas-AL 7 7 14 13 0 0 1 1 3 3 0 0Midas-Evt 8 8 13 13 1 1 1 1 1 5 0 0MidasAs-AL 1 1 0 0 0 1 0 0 0 0 0 0MidasAs-Evt 3 3 0 0 0 0 0 0 2 6 0 0
Panel B: Block bootstrapping of length 2
HistSim 43 43 43 43 9 10 14 14 8 12 6 7Garch-Fhs 6 6 3 3 1 1 0 0 1 2 1 1Garch-Evt 5 5 4 4 1 1 0 0 1 2 1 1GJR-Fhs 1 1 1 1 2 1 0 0 2 3 1 1GJR-Evt 3 3 1 1 2 1 0 0 2 3 1 1Sav-AL 8 10 18 17 5 5 6 6 9 12 6 7Sav-Evt 8 8 13 14 3 3 3 4 2 10 4 7As-AL 3 3 1 1 2 3 3 3 10 12 4 5As-Evt 3 3 0 0 1 2 2 2 4 13 1 5Midas-AL 6 6 16 16 1 1 1 1 2 3 0 0Midas-Evt 8 8 14 14 2 1 0 0 0 4 0 0MidasAs-AL 1 1 0 0 0 0 0 0 0 0 0 0MidasAs-Evt 3 3 0 0 1 1 0 0 1 5 0 1
Panel C: Block bootstrapping of length 6
HistSim 41 41 43 43 7 7 11 11 8 10 5 7Garch-Fhs 6 6 3 3 0 0 0 0 2 2 1 1Garch-Evt 5 5 4 4 0 0 0 0 2 2 1 1GJR-Fhs 1 1 1 1 1 1 0 0 2 2 1 1GJR-Evt 3 3 1 1 1 1 0 0 2 2 1 1Sav-AL 7 7 14 14 3 3 5 4 8 10 7 5Sav-Evt 7 7 15 14 2 2 1 1 4 8 2 4As-AL 3 3 1 1 1 0 2 2 12 12 3 4As-Evt 3 3 0 0 1 1 1 0 5 12 1 5Midas-AL 5 6 13 13 0 0 0 0 2 2 0 0Midas-Evt 6 7 13 13 1 1 0 0 1 3 0 0MidasAs-AL 1 1 0 0 0 0 0 0 0 0 0 0MidasAs-Evt 3 3 0 0 0 0 0 0 1 2 0 0
52
Recommended