Economic Modelling 38 (2014) 627–632

Contents lists available at ScienceDirect

Economic Modelling

j ourna l homepage: www.e lsev ie r .com/ locate /ecmod

State dependent asymmetric loss and the consensus forecast of real U.S.GDP growth

Matthew L. Higgins a, Sagarika Mishra b,1

a Department of Economics, Western Michigan University, 1903 West Michigan Ave., Kalamazoo, MI 49008, USAb Centre for Financial Econometrics, Deakin University, 221 Burwood Hwy., Melbourne, Victoria, 3125, Australia

1 We would like to thank C. James Hueng and Kevin Cments and suggestions. Any remaining errors are ours.

http://dx.doi.org/10.1016/j.econmod.2014.02.0160264-9993/© 2014 Elsevier B.V. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history:Accepted 14 February 2014Available online 11 March 2014

JEL classification:C53D83

Keywords:Survey forecastsAsymmetric lossTime-varying bias

It has been well documented that the consensus forecast from surveys of professional forecasters shows a biasthat varies over time. In this paper, we examinewhether this biasmay be due to forecasters having an asymmet-ric loss function. In contrast to previous research, we account for the time variation in the bias bymaking the lossfunction depend on the state of the economy. The asymmetry parameter in the loss function is specified to de-pend on set state variables which may cause forecaster to intentionally bias their forecasts. We consider boththe Lin–Ex and asymmetric power loss functions. For the commonly used Lin–Ex and Lin–Lin loss functions,we show the model can be easily estimated by least squares. We apply our methodology to the consensus fore-cast of real U.S. GDP growth from the Survey of Professional Forecasters.We find that forecast uncertainty has anasymmetric effect on the asymmetry parameter in the loss function dependent upon whether the economy is inexpansion or contraction. When the economy is in expansion, forecaster uncertainty is related to an overpredic-tion in themedian forecast of real GDP growth. In contrast, when the economy is in contraction, forecaster uncer-tainty is related to an underprediction in the median forecast of real GDP growth. Our results are robust to theparticular loss function that is employed in the analysis.

© 2014 Elsevier B.V. All rights reserved.

1. Introduction

Significant resources are spent each year by businesses and govern-ments for obtaining forecasts of a variety of economic variables. Numer-ous surveys of professional forecasters repeated over time provideresearchers with rich data sets to evaluate the forecasting accuracyof the forecasting profession. In spite of significant improvements ineconomic data collection and public availability, and advances in linearand nonlinear forecasting techniques, research over the last thirty yearshas consistently demonstrated that professional forecasters produce bi-ased forecasts. Among others, Carlson (1977), Urich and Wachtel(1984), Caskey (1985), Zarnowitz (1985), Frankel and Froot (1987),Croushore (1993, 1997), Jeong and Maddala (1996) and Souleles(2002) found that forecasts of the key variables such as inflation,interest rates and GDP growth reported by professional forecasters arebiased. Apparently, there are episodes where forecasters are systemati-cally overly optimistic or overly pessimistic. These findings seem to callinto question the value of professional forecasting serves.

Recently, researchers have argued that biased forecasts may be theresult of rational behavior if forecasters face asymmetric loss functions.Extending the work of Granger (1969), Christoffersen and Diebold(1997) showed that under asymmetric loss, the optimal forecast is the

order for their insightful com-

conditional mean plus a bias that depends on the parameters in theloss function and the second and higher order moments of the condi-tional distribution of the variable being forecasted. Assuming that fore-casters have an asymmetric exponential loss function and that interestrates follow a GARCH process, Batchelor and Peel (1998) showed thatrationality cannot be rejected for forecasts of three month treasury billyields. Elliott et al. (2005) proposed a generalized method of momentstest for forecast rationality assuming forecasters have an asymmetricpower loss function. Using their test, they showed that the bias in theIMF and OECD forecasts of the budget deficits of G7 countries is consis-tent with the forecasters having asymmetric loss functions. Capistran(2006) also used their test to show the U.S. Federal Reserve's GreenBook forecasts of inflation are biased and consistent with the Fed fore-casters having an asymmetric power loss function.

The above studies all assume that the asymmetric loss functions offorecasters remain constant over time. Time variation in the optimalbias is assumed to be due to time variation in the higher ordermomentsof the variable being forecasted. However recent studies by Krane(2003), Batchelor (2007) and Patton and Timmermann (2007) sug-gested that the loss function depends not only on the forecast errorbut also on the state of the economy. In that case, the bias in agents'forecasts can be explained either by the time-varying asymmetry pa-rameter or by time-varying higher ordermoments or by both. Althoughthere is extensive evidence for conditional heteroskedasticity ininflation and interest rates, there is little evidence of time-varying

2 Ifσ2 is time-varying then the estimation has to be done in two stages. In thefirst stagewe can estimate the time-varying variance of the series being forecasted using a GARCHmodel and then we can replace σ2 with their estimates (σ2

tþ1;t) and estimate using Eq. (2).

628 M.L. Higgins, S. Mishra / Economic Modelling 38 (2014) 627–632

conditional variances or higher order moments in quarterly real eco-nomic variables such as GDP or its individual components. If this is thecase, time variation in the optimal forecast bias of such variables mustbe due to time variation in the loss function itself. To our knowledge,the only paper which formally tests state dependence in the loss func-tion of forecasters is the recent paper by Patton and Timmermann(2007). They estimated a quadratic spline approximation to the impliedloss function of the U.S. Fed Green Book forecasts of real GDP growth bythe generalized method of moments. The estimated loss function wasallowed to depend on the current level of real GDP growth. Theyfound that the level of GDP growth significantly affected the shape ofthe loss function. When GDP growth was high, the loss function ap-peared to be symmetric. When GDP growth was low, the loss functionplaced greater loss on over predicting growth than on under predictinggrowth.

In this paper we test whether forecasters have time-varyingasymmetric loss in a parametric framework. Unlike Patton andTimmermann (2007), our model can easily incorporate multiplestate variables and is easy to estimate. We assume that forecastershave an asymmetric loss functionwhere the parameter that determinesthe asymmetry is a function of a linear combination of variables thatrepresent the state of the economy. For two of the most commonlyused loss functions, we show that the asymmetry parameter can be es-timated by least squares.We use ourmodel to test whether one quarterahead forecasts of real GDP growth from the Survey of ProfessionalForecasters have a time-varying bias that depends on the state of theeconomy. We consider factors such as duration of the business cycle,uncertainty of forecasts and type of government, which may causeagents to intentionally bias their forecasts. The paper proceeds asfollows. In Section 2 we present the theoretical results. In Section 3we describe the data and variables. In Section 4we analyze the empiricalresults. Finally, Section 5 concludes the paper.

2. State dependence in forecast bias

Suppose yt is the series to be forecasted. For simplicity, we assumethe series is to be forecasted with one period horizon. Our results canbe generalized to any horizon. The series in period t + 1 can bedecomposed as yt + 1= μt + 1,t+ ϵt + 1, where μt + 1,t is themean in pe-riod t + 1, conditional on the information set available in time period tand ϵt + 1 is an innovation which has mean zero and is uncorrelatedwith elements in the information set.

Let ytþ1;t be the one period ahead forecast of yt using all informationavailable in timeperiod t.When forecasters have quadratic loss, i.e., theyare indifferent between positive and negative errors, the optimal pre-dictor is ytþ1;t ¼ μ tþ1;t . Therefore, the forecast error under quadraticloss (ytþ1;t−ytþ1;t) is the same as the innovation ϵt + 1.

Standard forecast efficiency tests, which assume quadratic loss, arebased on regressing the forecast error et + 1 on variables in the informa-tion set and testing for significance. The underlying idea behind thesetests is that if forecasts are efficient, the forecast errors are uncorrelatedwith the determinants of the forecast.

If forecasters have asymmetric loss however, the properties of fore-cast errors are very different. Granger (1969) and Christoffersen andDiebold (1997) showed that if agents have asymmetric loss func-tions, the optimal predictor is the conditional mean plus a biasytþ1;t ¼ μ tþ1;t þ λtþ1;t, where the bias λt + 1,t depends on the loss func-tion and the second and higher order conditional moments of yt + 1.The forecast error becomes

etþ1 ¼ −λtþ1;t þ ϵtþ1: ð1Þ

The analytic expression for the bias λt + 1,t depends on the assumedparametric forms of the loss function and the conditional distribution ofyt + 1. If the loss function or the second or higher moments of yt + 1 arefunctions of the information set, the bias is a time-varying function of

the information set. As a consequence, standard efficiency tests will re-ject forecast rationality.

2.1. Lin–Ex loss

In this section we introduce a time-varying asymmetry parameterinto the commonly used Lin–Ex loss function introduced by Varian(1974). If the asymmetry parameter in the loss function is time-varying, the Lin–Ex loss function can be written as

L ytþ1−ytþ1;t

� �¼ exp αt ytþ1−ytþ1;t

� �n o−αt ytþ1−ytþ1;t

� �−1

whereαt can be any real number.Whenαt N 0, the loss is approximatelylinear for over prediction and approximately exponential for under pre-diction. On the other hand, if αt b 0, the loss is approximately linear forunder prediction and approximately exponential for over prediction.We specify the asymmetry parameter αt as a linear combination of aset of state variables zt so thatαt ¼ zt

′γ. The state variables are assumedto be part of the forecaster information set and represent informationthat alters the forecaster relative loss in over predicting versus underpredicting.

As shown by Christoffersen and Diebold (1997), assuming that theforecast errors are conditionally normal, the optimal predictor underLin–Ex loss is ytþ1;t ¼ μ tþ1;t þ σ2

tþ1;tαt=2. The forecast error becomes

etþ1 ¼ −σ2

tþ1;t

2αt þ ϵtþ1

¼ −σ2

tþ1;t

2zt′γ� �þ ϵtþ1:

Under Lin–Ex loss the time-varying bias in the forecast can be dueto the time-varying asymmetry parameter αt or due to the conditionalvariance σt + 1,t

2 of the series being forecasted or due to both.2 In the ab-sence of time-varying higher order moments, the bias can only be ex-plained by the time-varying asymmetry parameter. As we describedabove, empirically it is found that the one quarter ahead GDP growthrate forecasts do not have higher order moment dynamics. In this case,the bias can only be explained by the time-varying asymmetry parame-ter αt = zt′γ. If we assume a constant conditional variance σt + 1,t

2 = σ2,the forecast error becomes

etþ1 ¼ −σ2

2zt′γ� �þ ϵtþ1

¼ zt′γ� þ ϵtþ1:

ð2Þ

Given a sequence of observed forecast errors, the maximum like-lihood estimator of γ∗ can be computed by the least squares regres-sion of et + 1 on zt. The significance of the state variables can betested using standard t-tests. We can estimate the time-varyingbias with the negative of the predictions from this regressionλtþ1;t ¼ −z′tγ

� . We can estimate the time-varying asymmetricparameter by αt ¼ 2λtþ1;t=σ

2, where σ2 is the LS estimator of the var-iance of ϵt + 1 in Eq. (2). The regression (2) takes the form of the stan-dard regression test for forecast efficiency. From the above analysis,such a test can be interpreted as a test for time-varying asymmetryunder Lin–Ex loss.

Because αt represents a behavioral parameter in the forecaster's lossfunction, it would be useful to construct a confidence interval for αt ineach time period t. It is straight forward to present an asymptoticsampling theory for αt ¼ −2z′tγ

�=σ2. From standard linear regression

629M.L. Higgins, S. Mishra / Economic Modelling 38 (2014) 627–632

theory, γ� and σ2 are jointly asymptotically normal with elements of

their variance–covariance matrix tV given by var γ�� � ¼ σ2 Z′Z� �−1

, var

σ2� �

¼ 2σ4= T−1ð Þ and cov γ�; σ2

� �¼ 0, where Z is a T × k matrix

whose tth row is zt′. Let θ = (γ∗′, σ2)′ and f t ¼ dαt=dθ ¼

−2σ−2zt ;2σ−4z′t γ

�h i′. By the delta method [see, for example, Greene

(2008)], αt is asymptotically normal and a consistent estimator of its as-

ymptotic variance isavar αtð Þ ¼ f t′

t V f t, where V is the usual estimator ofthe variance–covariance matrix of the LS estimators of γ* and σ2 in theregression (2). Then, a 95% confidence interval forαt in each period t can

be calculated as αt � 1:96√ f′

t V f t .

2.2. Asymmetric power loss

In this section we introduce a time-varying asymmetry parameterinto the asymmetric power loss function used by Christoffersen andDiebold (1996, 1997) and Elliott et al. (2005, 2008). When α is time-varying, the loss function L can be written as,

L ytþ1;t ;αt

� �¼ αt þ 1−2αtð ÞI ytþ1;t−ytþ1;tb0

� �h iytþ1;t−ytþ1;t jp;�� ð3Þ

where 0 b αt b 1 is the asymmetry parameter and I(.) is the indicatorfunction that takes the value 1 when the condition in the argument(ytþ1;t−ytþ1;tb0) is true. The loss function is asymmetric for αt ≠ .5.In applications, the power p is usually assumed to be known. Thisfamily of loss functions includes many of the loss functions common-ly used in the literature. When αt≠ 0.5 and p=1, the loss function isasymmetric Lin–Lin. When αt ≠ 0.5 and p = 2, the loss function isasymmetric Quad–Quad. Because the asymmetry parameter is re-quired to be in the interval (0, 1), we employ a probit transformationand assume αt =Φ(zt′γ), whereΦ(.) is the standard normal cumulativedistribution function. Assuming conditional normality, the optimal pre-dictor is

ytþ1;t ¼ μ tþ1;t þωtσ tþ1;t :

where the time varying parameter ωt solves the equation

1−αtð ÞZ ωt

−∞z−ωtj jp−1ϕ zð Þdz−αt

Z ∞

ωt

z−ωtj jp−1ϕ zð Þdz ¼ 0; ð4Þ

and ϕ(z) is the standard normal density [see Christoffersen and Diebold(1996)].

If we assume that p is known and that the conditional standarddeviation is constant, the coefficients on the state variables can be esti-mated from the observed forecast errors by the method of maximumlikelihood. The forecast error is

etþ1 ¼ −ωtσ þ ϵtþ1:

The log-likelihood function of the forecast errors is

lT γ;σð Þ ¼ − T2log 2πð Þ− T

2log σ2

� �−1

2

XTt¼1

etþ1 þωtσ� �σ2 ;

where ωt is a function of γ through Eq. (4). For any given value of γ,Eq. (4) can be solved numerically for each ωt and the log-likelihoodfunction can, in turn, be evaluated. The log-likelihood function canthen be numerically maximized using the standard methods. Giventhe MLE's of γ and σ, we can estimate the time-varying asymmetryparameter by αt ¼ Φ z′tγ

� �and the time-varying bias by

λtþ1;t ¼ ωtσ .The above estimation is greatly simplified when p = 1 and the loss

function is Lin–Lin. The optimal predictor is easily seen to reduce to

ytþ1;t ¼ μ tþ1;t þΦ−1 αtð Þσ . Because we assume αt = Φ(zt′γ) to insurethat αt ∈ (0, 1), the optimal predictor becomes simplyytþ1;t ¼ μ tþ1;t þ σz′tγ. The forecast error is

etþ1 ¼ −σz′tγ þ ϵtþ1

¼ z′tγþ þ ϵtþ1:

ð5Þ

In this case maximum likelihood estimation is equivalent to line-ar regression. The regression (5) is identical to the Lin–Ex regres-sion (2). The only difference lies in the parameterizations γ∗ =−σ2γ/2 and γ+ = −σγ. Therefore, the standard regression test forforecast efficiency can be interpreted as a test for time-varying asym-metry under either Lin–Ex or Lin–Lin loss. As with Lin–Ex loss, wecan estimate the time-varying bias with the negative of the predic-

tions from this regression λtþ1;t ¼ −z′tγþ. A consistent estimator of

the time-varying asymmetry parameter is αt ¼ Φ λtþ1;t=σ� �

, where σ

is the LS estimator of the standard deviation of ϵt + 1 in Eq. (5).Using theMLE's, we can construct confidence intervals for the time-

varying asymmetry parameter αt in the asymmetric power loss func-tion. By the asymptotic normality of the MLE and the delta method,αt ¼ Φ z′t tγ

� �is asymptotically normal and a consistent estimator of

it's asymptotic variance is avar αtð Þ ¼ ϕ z′tγ� �2z′tΣzt , where ϕ(.) is the

standard normal density function and Σ is any of the conventional es-timators of the asymptotic variance–covariance matrix of the MLE γ.

A 95% confidence interval for αt is αt � 1:96ϕ z′tγ� �

√z′tΣzt . If it isassumed that p = 1 and the regression (5) is estimated by LS, the

estimator of αt is αt ¼ Φ −z′t γþ=σ

� �. In this case, let θ = (γ∗′σϵ

2)′

and gt ¼ dαt=dθ ¼ −ϕ −z0t γ

þ=σ

� �σ−1z′t ;−ϕ −z′tγ

þ=σ

� �z′tγ

þ=2σ3

h i′.

Then, again by the delta method, αt is asymptotically normal and a con-

sistent estimator of its asymptotic variance is avar αtð Þ ¼ g′t V gt where Vis the usual estimator of the variance–covariance matrix of the LS esti-mators of γ+ and σϵ

2 in Eq. (5). Then a 95% confidence interval for αt is

αt � 1:96√g′t V gt .

3. Data and variables that may explain bias

In this section we test for state dependence bias in the one quarterahead real U.S. GDP growth rate forecasts from the Survey of Profession-al Forecasters (SPF). Startingwith the first quarter of 1968, the NationalBureauof Economic Research, togetherwith the American Statistical As-sociation, began conducting surveys of forecasts of important economicvariables produced by private sector economists. The forecastedvariables includedmeasures of output, inflation, unemployment and in-terest rates. In the early years the survey averaged about 50 participantsin each quarter. The number of respondents had dwindled to about 20participants by the late 1980s. The Federal Reserve Bank of Philadelphiarevived the survey in 1990 after it was discontinued by the ASA and theNBER. The survey once again averages about 50 participants. Croushore(1993) provides a very detailed description of the SPF. In the third quar-ter of 1981, the scope of the survey was expanded to include, amongother forecasts, forecasts of the one quarter ahead real GDP growthrate. These are the forecasts we use in this paper. We use the medianconsensus forecast. We replicated all of our work using the mean con-sensus forecast and the conclusions were the same. Our sample periodbegins with the third quarter of 1981 and ends with the fourth quarterof 2007.

There are many reasons why forecasters may have asymmetric lossfunctions and intentionally report biased forecasts. Some authors havesuggested that agents may bias their forecasts based on whether theeconomy is in expansion or contraction. The ability to predict turningpoints in economic activity is very valuable in economic planning. Fore-casters may seek to build a reputation for their ability to forecast

Table 1Determinants of time-varying asymmetry parameter in the one quarter ahead GDPgrowth rate, 1981:3–2007:4.

Variables Lin–Ex/Lin–Lin loss Quad–Quad loss

General Final General Final

Intercept −0.49(0.78)

−0.03(0.36)

−0.37(0.52)

−0.07(0.27)

RECDM 1.27(1.50)

0.69(1.28)

POLIDM 0.61(0.46)

0.52(0.38)

UNCERT −0.63⁎⁎

(0.33)−0.57⁎⁎

(0.29)−0.49⁎⁎

(0.24)−0.39⁎⁎

(0.19)DURATION 0.006⁎

(0.02)0.003(0.01)

REC_UNCER 3.43⁎⁎

(1.79)1.70⁎⁎

(0.40)2.21⁎⁎

(0.63)1.19⁎⁎

(0.23)REC_DUR −1.61

(1.30)−0.91(0.58)

Adj Rsq 0.21 0.20 0.16 0.16SIC 4.43 4.30 0.37 0.20Q(4) 0.161 0.123 0.209 0.140Q(8) 0.443 0.264 0.398 0.179Q2(4) 0.899 0.852 0.650 0.407Q2(8) 0.950 0.676 0.784 0.143Jarque–Bera 0.499 0.854 0.395 0.379

Standard errors are in parenthesis. For residuals, squared residuals and normality testp-values are given.⁎⁎ Denotes significant at 5% level.⁎ Denotes significant at 10% level.

630 M.L. Higgins, S. Mishra / Economic Modelling 38 (2014) 627–632

turning-points. Failure to predict an actual turning-point may be morecostly than incorrectly predicting ones that do not occur. This suggeststhat when the economy is in expansion, forecasters may intentionallybias their forecast of GDP growth downward. Similarly, when the econ-omy is in recession, they may intentionally bias their forecast upward[see Loungani and Trehan (2002) and Zarnowitz and Braun (1993)].We test whether expansion/recession in the economy can causebias in agents' forecasts by using a expansion/recession dummyin our regression. We construct an expansion/recession indicator fromthe NBER's U.S. business cycle chronology. We define our dummyvariable (RECDM) to take the value 1 if the economy is in recessionand zero otherwise. The NBER dates the beginning of a recession asa significant decline in economic activity lasting more than fewmonths. The NBER considers not only a decline in real GDP, but alsoa broad decline in real income, employment, industrial productionand wholesale–retail–sales. The NBER usually announces theturning-points several quarters after they occur. We use the date ofthe turning-point and not the announcement date because profes-sional forecasters are likely to be aware of the general economic de-cline shortly after they occur and prior to the official announcementby the NBER.

In addition to the current phase of the business cycle, duration of thecurrent phase may also cause forecasters to bias their forecasts. As thelength of the expansion becomes longer, agents may become increas-ingly optimistic and bias their forecasts upward. Similarly, as the lengthof a contraction becomes longer, agentsmay become increasingly pessi-mistic and bias their forecast downward. Batchelor andDua (1990) sug-gested that forecasters may, in fact, find it beneficial to develop areputation as optimists or pessimists. To test this source of bias, we in-clude duration (DURATION) measured as the number of quarters sincethe last turning point as a variable in our regression. We also includean interaction between the expansion/recession dummy and duration(REC_DUR) to allow for asymmetric effects between the phase of thebusiness cycle and the business cycle duration.

We also test whether the political parties of the current administra-tion produces a bias in the forecasts of real GDP growth. There has beena long literature that documents the difference in the two U.S. politicalparties emphasis on the inflation/output trade-off in the economy. Tra-ditionally, the Democrats are viewed as more concerned with stimulat-ing growth and reducing unemployment in the short-run. Whereas,Republicans are more concerned with keeping inflation low to promotegrowth and stability in financial markets. Hibbs (1977, 1986) andAlesina et al. (1993) gave time series evidence for these stylized notions.More recently, Snowberg et al. (2007) used election prediction marketdata to show that the market's expectations concerning equity returns,interest rates and oil prices were directly impacted by the probabilitiesof which party would be elected. Hence the political party in officemight explain the bias so we introduce a political party dummy(POLIDM) variable. The dummy variable for type of governmenttakes the value 1 if a Republican is in office and zero if a Democrat isin office.

In addition to the above variables, we consider whether forecasteruncertainty affects forecaster bias. Fildes and Stekler (2002) suggestedthat the relationship between forecaster bias and uncertainty is impor-tant, but the effectmay be ambiguous.When there is a lack of consensusamong forecasters, forecasters may have a stronger incentive to try andbuild a reputation as an optimist or a pessimist. It is also possible that inthe presence of high uncertainty, forecasters may place less reliance ontheir formal or informal models and bias their forecast towards thelong-run historical average of real GDP growth. Our measure of uncer-tainty (UNCERT) is the standard deviation of the individual one quarterahead forecasts of real GDP growth from the individual level SPF. Aswith business cycle duration, we also include an interaction term be-tween our uncertainty measure and the business cycle dummy(REC_UNCER) to allow uncertainty to have an asymmetric effect overthe business cycle.

4. Empirical results

In this empirical section, we show that time varying bias in the SPFforecasts of one quarter ahead real GDP growth may be explained bycertain state variables. As we argue above, under asymmetric loss atime-varying forecast bias could be due to either the time variation inthe conditional variance or time variation in the asymmetry parameter.We begin our empirical analysis by testing for time-varying secondorder moments in the forecast error of GDP growth rate. We first con-duct LM tests for ARCH through lag eight. All of the tests fail to rejectthe null hypothesis of no ARCH. We also conduct a variety of tests tosee if the conditional variance of the forecast error is determined bylagged levels or first differences of real GDP growth. These tests werealso insignificant, further indicating a constant conditional variance forthe forecast error. The previous tests are conducted assuming a constantconditional mean. As described in Batchelor and Peel (1998), if the con-ditional variance of the forecast error is time-varying, but the asymme-try parameter is constant, the forecast errors should follow a GARCH-Mprocess under Lin–Ex and Lin–Lin loss. Therefore, we also estimateGARCH(1,1)-Mmodels andGARCH(1,1)-Mmodels that included laggedlevels and first differences of real GDP growth. For all of the models, thevariables in the conditional variances are insignificant. Also, the GARCH-M coefficients on the conditional variances are themselves insignificant.The above tests suggest that the forecast errors for GDP growth rate donot have time-varying variances nor biases that depend on them.

Specifying the conditional variance as constant, we estimate modelswhich allow the asymmetry parameter to depend on RECDM, POLIDM,UNCERT and DURATION. To allow for possible asymmetric effects overthe business cycle, we also include the interactions between the reces-sion dummy RECDM and the continuously varying variables UNCERTand DURATION. We estimate the models assuming agents forecastunder both Lin–Ex and asymmetric power loss. For asymmetric powerloss, we consider both a Lin–Lin specification with p = 1 and a Quad–Quad specification with p = 2. Recall that in Lin–Ex and Lin–Lin loss,the linear regressions used to estimate the asymmetry parameter areidentical. Therefore, we present two sets of models. One for Lin–Ex/Lin–Lin and one for Quad–Quad. The Quad–Quad model is estimatedby maximum likelihood.

Fig. 2. Estimated time-varying asymmetry parameter and 95% confidence bands underLin–Lin loss.

631M.L. Higgins, S. Mishra / Economic Modelling 38 (2014) 627–632

The estimated models (regressions (2) and (5)) and accompanyingdiagnostic statistics are shown in Table 1. In the first column ofTable 1 we present the general model for Lin–Ex and Lin–Lin loss. Thedependent variable in Table 1 is the forecast error, the difference be-tween the real value of GDP growth and the forecast of GDP growth pro-duced by the SPF. We find the intercept and the coefficients on RECDM,POLIDM, DURATION and REC_DUR to be insignificant. A zero interceptindicates that there is no time-invariate systematic bias in the forecasts.Insignificance of the recession dummy indicates that forecasters haveno propensity to over or under forecast real GDP growth dependingonwhether the economy is in expansion or contraction. This is contrarytomost findings in the literature. The insignificance of POLIDM suggeststhat agents do not appear to be optimistic nor pessimistic based onwhich party is in office. The coefficients on UNCERT and REC_UNCERare significant. The sign of the coefficient on UNCERT is negative andthe sign of the coefficient on REC_UNCER is positive. Furthermore, thecoefficient on UNCERT is less in absolute value than the coefficient onREC_UNCER. This implies that when the economy is in recession, in-creasing uncertainty causes forecasters to introduce a positive bias.Whereas, when the economy is in expansion, increasing uncertaintycauses forecasters to introduce a negative bias. Uncertainty causes fore-casters to make conservative forecasts. As uncertainty increases, fore-casters bias the forecast towards it's historical average. Unlike thefinding by McNees (1976, 1988, 1997), McNees and Ries (1983), andZarnowitz and Braun (1993) that during recession agents over predict,our results suggest that in the presence of recession, it is increasing un-certainty that causes agents to over predict. In other words, our resultssuggest that it is not just recession that causes agents to over predict.It is the increased uncertainty during recession that causes agents toover predict. The results for the Quad–Quad model are very similar.Themagnitude, signs and level of significance of the estimated parame-ters are similar between the Lin–Ex/Lin–Lin and Quad–Quad specifica-tions. The finding that forecaster uncertainty and the current phase ofthe business cycle determine the time-varying forecast bias does notappear to dependon the specificationof the parametric formof the fore-caster loss function.

We drop the insignificant variables and re-estimate the models toobtain final specifications. These results are also shown in Table 1. Themagnitude, signs and level of significance of the coefficients in thefinal models are similar to the general models. In both specifications,the SIC is minimized by the final model. There is no evidence of mis-specification in the models. Q-statistics through lags four and eightbased on the residuals and squared residuals are insignificant and indi-cate that there is nodependence in the errors through thefirst or secondmoments. The parametric form of the estimatedmodels depends highlyon the forecast errors being normally distributed. The Jarque–Bera tests

Fig. 1. Estimated time-varying asymmetry parameter and 95% confidence bands underLin–Ex loss.

for normality of the errors are also insignificant, suggesting that the nor-mality assumption is plausible.

For each of the final models, we estimate the time-varying asymme-try parameter αt for each time period. In Figs. 1, 2 and 3we show the es-timated asymmetry parameters with accompanying 95% confidencebands under Lin–Ex, Lin–Lin and Quad–Quad loss. Considering Fig. 1,the asymmetry parameter shows considerable variation over timeunder Lin–Ex loss. The parameter is predominantly negative, being as-sociated with periods when the economy is in expansion. During pe-riods of economic expansion, forecasters place a greater loss on overpredicting real GDP growth in the presence of forecaster uncertainty.During the 1982, 1990 and 2000 recessions, the asymmetry parameteris positive and much larger in magnitude. During these periods of con-traction, forecasters place a larger loss on under predicting real GDPgrowth in the presence of forecaster uncertainty. Recall that for Lin–Exloss, the loss function is symmetric when αt = 0. For the majority ofthe time periods, the confidence bands exclude 0, indicating that theasymmetry is statistically significant. In Figs. 2 and 3, the pattern of var-iation over time of the asymmetry parameter under Lin–Lin and Quad–Quad loss is similar to that under Lin–Ex loss shown in Fig. 1. For Lin–Linand Quad–Quad, the loss function is symmetric when αt = .5. For themajority of the time periods, the confidence bands exclude .5, again in-dicating that the asymmetry is statistically significant. Under Lin–Linloss, the asymmetry parameter is slightly less in magnitude than theparameter under Quad–Quad loss. The figures clearly convey that thepattern of variation in the asymmetry of the loss function does notappear to depend on the specification of the parametric form of theloss function.

Fig. 3. Estimated time-varying asymmetry parameter and 95% confidence bands underQuad–Quad loss.

Fig. 4. Estimated time-varying forecast biases under Lin–Ex, Lin–Lin and Quad–Quad.

632 M.L. Higgins, S. Mishra / Economic Modelling 38 (2014) 627–632

In Fig. 4, we show the estimated time-varyingbias implied by each ofthemodels. Recall that the estimated bias under Lin–Ex and Lin–Lin areidentical. The magnitude of the estimated biases is slightly less underQuad–Quad loss than under Lin–Ex and Lin–Lin loss. The over patternsof the biases are similar. During periods of economic expansion, fore-casters bias their growth forecasts downward in the presence of fore-caster uncertainty with the biases ranging between − .5 and −2% atan annualized rate. During the three recessions in our sample period,the biases become positive. During the 1982 recession, the biasedreach 2.6% for Quad–Quad and 5.2% for Lin–Ex and Lin–Lin loss. Duringthe 1990 and 2000 recessions, the biases are smaller in magnitude andrange between .8 and 1.9%.

5. Conclusion

Although there have been important advances in time series model-ing, professional forecasters still produce forecasts that have a bias thatvaries over time. We examine whether the bias in the SPF forecast ofreal U.S. GDP growth can be explained by a time-varying asymmetricloss function.We propose amethod for estimating the time-varying de-gree of asymmetry and forecast bias for the asymmetric loss functionsmost commonly considered in the forecasting literature. For the impor-tant Lin–Ex and Lin–Lin loss functions, all of the inference can be basedon the LS estimator of a linear regression. Contrary to previouswork, ourempirical results show that the direction and magnitude of the bias inforecasts of real GDP growth are not simply dependent on the phaseof the business cycle. Rather, the direction andmagnitude of the bias de-pends on an interaction between forecaster uncertainty and the phaseof the business cycle.

References

Alesina, A., Londregan, J., Rosenthal, H., 1993. A model of the political economy of theUnited States. Am. Polit. Sci. Rev. 87, 12–33.

Batchelor, R., 2007. Bias in macroeconomic forecasts. Int. J. Forecast. 23, 189–203.Batchelor, R., Dua, P., 1990. Product differentiation in the economic forecasting industry.

Int. J. Forecast. 6, 311–316.Batchelor, R., Peel, D.A., 1998. Rationality testing under asymmetric loss. Econ. Lett. 61,

49–54.Capistran, C., 2006. Bias in Federal Reserve Inflation Forecasts: Is the Federal Reserve

Irrational or Just Cautious? Manuscript, Banco De Mexico.Carlson, J.A., 1977. A study of price forecasts. Ann. Econ. Soc. Meas. 6, 27–56.Caskey, J., 1985. Modeling the formation of price expectations: a Bayesian approach. Am.

Econ. Rev. 75, 768–776.Christoffersen, P.F., Diebold, F.X., 1996. Further results on forecasting and model selection

under asymmetric loss. J. Appl. Econ. 11, 561–571.Christoffersen, P.F., Diebold, F.X., 1997. Optimal prediction under asymmetric loss.

Econom. Theory 13, 808–817.Croushore, D., 1993. Introducing: the survey of professional forecasters. Fed. Res. Bank

Phila. Bus. Rev. 3–13.Croushore, D., 1997. The Livingston survey: still useful after all these years. Bus. Rev.

15–27.Elliott, G., Komunjer, I., Timmermann, A., 2005. Estimation and testing of forecast rationality

under flexible loss. Rev. Econ. Stud. 72, 1107–1125.Elliott, G., Komunjer, I., Timmermann, A., 2008. Biases in macroeconomic forecasts:

irrationality or asymmetric loss? J. Eur. Econ. Assoc. 6, 122–157.Fildes, R., Stekler, H., 2002. The state of macroeconomic forecasting. J. Macroecon. 24,

435–468.Frankel, J.A., Froot, K.A., 1987. Using survey data to test some standard propositions

regarding exchange rate expectations. Am. Econ. Rev. 133–153.Granger, C.W.J., 1969. Prediction with a generalized cost of error function. Oper. Res. Q.

20, 199–207.Greene, W.H., 2008. Econometric Analysis, 6'th Edition. Prentice Hall Upper Saddle River,

NJ.Hibbs, D., 1977. Political parties and macroeconomic policy. Am. Polit. Sci. Rev. 71,

1467–1487.Hibbs, D., 1986. Political parties and macroeconomic policies and outcomes in the United

States. Am. Econ. Rev. 76, 66–70.Jeong, J., Maddala, G.S., 1996. Testing the rationality of survey data using the weighted

double-bootstrapped method of moments. Rev. Econ. Stat. 78, 296–302.Krane, S.D., 2003. An evaluation of real GDP forecasts: 1996–2001. Econ. Perspect. 1, 2–20.Loungani, P., Trehan, B., 2002. Predicting when the economy will turn. FRBSF Econ. Lett.

1–4.McNees, S.K., 1976. The Forecasting Performance in the 1970s. Federal Reserve Bank of

Boston.McNees, S.K., 1988. How accurate are macroeconomic forecasts? N. Engl. Econ. Rev. 36,

15–21.McNees, S.K., 1997. The 1990–91 recession in historical perspective. Publ. Budg. Financ.

103.McNees, S.K., Ries, J., 1983. The track record of macroeconomic forecasts. N. Engl. Econ.

Rev. 5, 18.Patton, A., Timmermann, A., 2007. Testing forecast optimality under unknown loss. J. Am.

Stat. Assoc. 102, 1172–1184.Snowberg, E., Wolfers, J., Zitzewitz, E., 2007. Partisan impacts on the economy: evidence

from prediction markets and close elections. Q. J. Econ. 122, 807–829.Souleles, N.S., 2002. Consumer sentiment: its rationality and usefulness in forecasting

expenditure — evidence from the Michigan micro data. NBER Working Paper.Urich, T., Wachtel, P., 1984. The structure of expectations of the weekly money supply

announcement. NBER Working Paper.Varian, H.R., 1974. A Bayesian approach to real estate assessment. Stud. Bayesian Econ.

Honour LJ Savage (North Holland) 195–208.Zarnowitz, V., 1985. Rational expectations andmacroeconomic forecasts. J. Bus. Econ. Stat.

3, 293–311.Zarnowitz, V., Braun, P.A., 1993. Twenty-two years of the NBER-ASA quarterly economic

outlook surveys: aspects and comparisons of forecasting performance. Bus. CyclesIndic. Forecasting 28, 11–84.