Katerina Simons

Economist, Federal Reserve Bank ofBoston. The author wishes to thankRichard Kopcke and Peter Fortune forhelpful comments. Kevin Daly pro-vided valuable research assistance.

Model Error

Modern finance would not have been possible without models.Increasingly complex quantitative models drive financial inno-vation and the growth of derivatives markets. Models are

necessary to value financial instruments and to measure the risks ofindividual positions and portfolios. Yet when used inappropriately, themodels themselves can become an important source of risk. Recently,several well-publicized instances occurred of institutions suffering signif-icant losses attributed to model error. This has sharpened the interest inmodel risk among financial institutions and their regulators.

In March of 1997, NatWest Markets, an investment banking subsid-iary of National Westminster Bank, announced a loss of £90 million dueto mispriced sterling interest rate options. Shortly thereafter, BZW, aninvestment banking subsidiary of Barclays, sustained a £15 million losson mispriced currency options and Bank of Tokyo-Mitsubishi announceda loss of $83 million. In April of 1997, Deutsche Morgan Grenfell lost anundisclosed amount. Model errors have been blamed for all these losses.1

This article will describe various models and discuss model errorscharacteristic of two types—valuation models for individual securities,and models of market risk. The article will discuss the statistical issuesthat complicate the use of such models, namely the probability distribu-tions of asset returns and estimates of their volatility. It will also discussa number of practical issues related to model development and describethe approach taken by bank regulators to model risk.

I. Types of Models

Models can be roughly divided into four categories. The first type isthe macroeconomic forecasting model, which seeks to predict real output,employment, inflation, the unemployment rate, and the level of interestrates. Macroeconomic models range from naive one-equation models that

extrapolate the values of these variables from theirrecent past levels to elaborate ones containing hun-dreds of equations and thousands of variables.

A second, related type of model is a microeco-nomic model that seeks to explain relationships inparticular markets. Such models also use regressionanalysis, but to explain microeconomic variables, forexample, the effect of changes in interest rates onmortgage prepayment rates, or the effect of wintertemperatures on the demand for heating oil. Macro-economic models often use the results of these modelsas inputs.

Increasingly complex quantitativemodels drive financial

innovation and the growthof derivatives markets.

The third type is the valuation model, used bytraders to price derivatives instruments. Derivativesare instruments whose value depends on the price ofunderlying assets such as stocks, bonds, commodities,or foreign currencies. A simple example of a deriva-tive is a call option on a stock. It gives the buyer theright, but not the obligation, to purchase the stock atsome point in the future at a price agreed upon today(known as the strike price). A call option can be pricedwith the Black-Scholes model, which determines theoption’s value as a function of five factors: the strikeprice, the time to expiration, the current stock price, itsvolatility, and the risk-free interest rate. This model,developed by Fischer Black and Myron Scholes (1973)is often cited as the foundation of modern derivativesmarkets. For the first time, options could be pricedaccurately, and this gave rise to an explosion ofoptions trading in the 1980s.

In fact, many newer derivative pricing models aremodifications or extensions of Black-Scholes. Notethat the Black-Scholes model is different from a statis-tical model. It is an analytical formula—theoretically,given the five specified factors, one can determine thevalue of the option exactly. In practice, the results ofstatistical models often serve as inputs into valuationmodels. For example, to value a mortgage-backedsecurity, one needs an estimate of the mortgage pre-

payment rate, which has to come from a statisticalmodel. To value a call option with the Black-Scholesformula, one needs to know the volatility of the stockprice, which can also be statistically estimated. Statis-tical estimates are subject to errors and thus can leadto error in the valuation model for which they serve asan input.

The fourth type of model is a model of marketrisk, used to estimate how the value of an individualposition or the whole portfolio changes with a changein the prices of the underlying instruments. Currently,the standard for market risk measurement is a familyof models known as “Value at Risk” or VAR. VAR isdefined as the maximum amount the portfolio can losewith a given probability in a given period of time. Forexample, if the daily VAR is estimated to be $1 millionat the 95th percent confidence level, one would expectto lose no more than $1 million in one day 95 percentof the time.

To estimate VAR, one first needs to know theprobability distributions of the prices of the underly-ing assets. Second, one must use these prices to arriveat the values of the securities in one’s portfolio (that is,use valuation models); and third, one must aggregatethese values into one summary estimate of the valueat risk. Thus, it is clear that errors in valuation modelsfeed into errors in VAR. In many financial institutions,the valuation methods used in VAR are different andindependent of the valuation models used on thetrading desks. However, an accurate VAR is not asubstitute for accurate valuation models. Clearly, it isnot very useful to know with a great degree ofconfidence and precision how much money one’sportfolio can lose in a given period of time if one isseriously mistaken about what the portfolio is worthin the first place.

Statistical issues related to macroeconomic andmicroeconomic models are outside the scope of thisarticle. We will be concerned only with the last twotypes of models mentioned—individual valuation andValue at Risk.

II. Types of Errors

It can be useful to think of valuation models andValue at Risk as subject to two distinct types oferror—error in the inputs or data, and error in thestructure of the model itself. To return to the exampleof pricing a call option, one is unlikely to make amistake about the strike price and the expiration date,as these are specific to the contract and do not change

1 These events are discussed in more detail in Paul-Choudhury(1997).

November/December 1997 New England Economic Review18

during the life of the option. Similarly, while therecan be some disagreement about the exact numberthat should be used for the risk-free interest rate, theoption price is not very sensitive to small variations init. A more likely and serious error in the option valuecan result from an incorrect stock price. Price errorsare especially likely if the stock is illiquid, infrequentlytraded, or subject to abrupt price swings. Then theprice at the last transaction may be outdated and givethe wrong price for the option.

It is always a matter of judgmentwhich simplifying assumptionscan be made and whether theresulting model is sufficientlyaccurate for the purpose for

which it is being used.

Of the five factors in the Black-Scholes modelaffecting the value of the option, only the volatility isnot directly observable. It must be estimated from thehistorical data; alternatively, it can be “backed out”from prices of other options on the same underlyingsecurity. Different methods of volatility estimation canyield widely different results, and it is probably safe tosay that incorrect volatility estimates are the singlelargest cause of error in both valuation models andrisk management models.

The second type of error is specification error,that is, error in the structure of the model itself. Themodel is misspecified if, for example, an additionalfactor affecting the price of the derivative is not part ofthe model. This kind of error often occurs when amodel intended for one product is used for anotherwithout sufficient modification. A more complex issuearises with respect to the simplifying assumptionsused in the model. Such assumptions are necessarilyunrealistic, but this may not be a problem if theypreserve the important features of the market whilemaintaining analytical tractability. However, it is al-ways a matter of judgment which simplifying as-sumptions can be made and whether the resultingmodel is sufficiently accurate for the purpose forwhich it is being used. Simplifying assumptions, par-ticularly those made in modeling the stochastic pro-cess governing asset returns, will be discussed below.

III. The Perils of VAR

Perhaps no model is more fraught with contro-versy than Value at Risk, or VAR. First popularized bythe Group of Thirty report, Derivatives: Practices andPrinciples, VAR has become a de facto industry stan-dard for measuring market risk, that is, the risk oflosses from the prices of financial instruments. VARforms the backbone of the “Internal Models Ap-proach” to allocating capital for market risk in banktrading accounts. This approach, endorsed by BasleCommittee on Banking Supervision of the Bank forInternational Settlements (BIS) and adopted by U.S.bank regulators, allows banks to use their own modelsto set aside capital for the risks in their tradingaccounts.

To be acceptable to regulators, banks’ modelsmust be based on VAR and fulfill certain other condi-tions. Specifically, the confidence level for the VARcalculation must be 99 percent and the holding periodmust be two weeks. For instance, if VAR were foundto be $1 million under these conditions, the bankwould expect to lose more than $1 million 1 times outof 100 over a two-week period. Figure 1 shows theprobability of possible profits and losses based on thenormal distribution and the VAR at the 99 percent

November/December 1997 New England Economic Review 19

confidence level, expressed as a number of standarddeviations from the mean.

Many market participants consider the 99 percentconfidence level and the two-week interval to be tooconservative; the industry practice is to limit theconfidence level to 95 percent and the holding periodto one day. However, since the purpose of the regu-latory VAR calculation is to set aside enough capitalto protect the bank against failure due to marketrisk, the regulators consider that a conservative VARis justified.

Recently, there has been something of a backlashagainst VAR. Its detractors argue that VAR is mislead-ing and too often wrong to be useful and that, byproviding an illusion of scientific precision, it creates afalse sense of security, thereby leading traders to carrylarger positions and take more risk.2 The followingexample is a demonstration of how one can go wrongwith VAR.

The Mexican Peso Crisis

Suppose a bank trading desk held a position of$10 million in Mexican pesos on December 19, 1994and wanted to calculate the VAR of this position based

on the historical behavior of the peso/dollar exchangerate. Following the BIS guidelines for internal models,the bank uses a 99 percent confidence interval and atwo-week holding period. Figure 2 shows a plot of thepercentage changes in the exchange rate from Novem-ber 1993 through the first half of December 1994. Thebank calculates the standard deviation, or volatility, ofthe daily return on its peso holdings during theprevious year starting on November 9, 1993. (BeforeNovember 9 the peso was pegged to the dollar, sothere was no variation in the exchange rate.) The dailyreturn on the peso position can be calculated as thedaily percentage change in the exchange rate, namely:

Rt 5Et 2 Et21

Et21z 100%.

Rt is the daily return and Et is the exchange rate on dayt. The daily volatility of returns over a period of Ndays, with all observations weighted equally, can becalculated as follows:

s 5 ÎOi51

N

~Ri 2 m!2,

where s is the volatility of daily returns and m is themean return.2 Taleb (1997) eloquently summarizes the case against VAR.

November/December 1997 New England Economic Review20

The volatility during the sample period wasfound to be 0.4704 percent. The VAR calculation isshown in Table 1. To calculate the VAR of this positionfor two weeks (or 10 trading days), we multiply thedaily volatility by =10. Then we multiply the result-ing two-week volatility by 2.33 for the 99th percentconfidence level and by the dollar value of the port-folio ($10 million). VAR 5 .4704% p =10 p 2.33 p $10million 5 $347,000. This means that the bank wouldexpect to lose no more than $347,000 within the nexttwo weeks, 99 times out of a 100.

The subsequent events are shown in Figure 3.Within the next eight days, the peso depreciated 65.7percent, when the exchange rate went from 3.47 pesosto the dollar on December 19 to 5 pesos to the dollaron December 27. The bank would have lost $6.5million on its $10 million position, compared to its$347,000 VAR estimate.

IV. Fat Tails

Why did the risk estimate prove so inadequate inthe case of the Peso Crisis? Having first verified thatthe exchange rate data were correct and that thearithmetic calculations were also done correctly, weare left to examine the assumptions underlying themodel. To arrive at this VAR estimate, we adopted themost commonly used model of asset returns. Weassumed that returns are independently distributedover time and identically distributed over time (IID),and that they are normally distributed. These assump-tions make the model analytically tractable but theyare not necessarily close to reality. The assumption oftemporal independence allowed us to scale the dailyvolatility by the square root of time. Furthermore, theassumption of the normal distribution allowed us toinfer that a loss 2.33 standard deviations away fromthe mean would happen no more than one time outof a hundred. The Peso Crisis was a 44-standard-deviation event, something that might happen to anormally distributed random variable once in manybillions of years.3 Nevertheless, it did happen. Nor

3 The exact probability of such an event is not reported herebecause none of the statistical programs tried by the author coulddistinguish such a small number from zero.

Table 1Calculation of VAR

PositionDaily

Volatility10-dayVolatility

99%Confidence

2-weekVAR

$10 million .4704% 1.5% 3.47% $347,000

November/December 1997 New England Economic Review 21

was the Peso Crisis of 1994 unique. Crashes happenregularly in financial markets, and they do so muchmore frequently than would be predicted by thenormal model.

While the normal distribution is used routinely inmodeling asset returns, it has been widely recognizedfor many years that financial markets exhibit signifi-cant non-normalities. In particular, asset returns ex-hibit “fat tails,” meaning that more of their probabilityis to be found at the tail ends of the distribution andless at the center. A fat-tailed distribution and anormal distribution are illustrated in Figure 4. Afat-tailed distribution makes extreme outcomes suchas crashes relatively more likely than does the normaldistribution.

How to Measure Fat Tails

The degree of tail-fatness of a distribution can bemeasured in two ways. The first is kurtosis, or thefourth moment of a distribution. Kurtosis for a sampleof daily returns can be estimated as follows:

K 51

Ts4 Ot51

T

~Rt 2 m!4,

where K is kurtosis, T is the number of days in thesample, m is the mean return, s is the standarddeviation of returns, and Rt is the return on day t. Thekurtosis of a normally distributed variable is 3. Bycontrast, the sample kurtosis of the daily returns onthe S&P 500 index between January 1, 1980 and March31, 1995 was 128.38. For the most part this highkurtosis can be explained by the stock market crash of1987. The kurtosis was 2.62 after the crash and 0.98before it. The high kurtosis was entirely due to theperiod between October 1 and December 31 of 1987(Fortune 1996, p. 33).

Another measure of tail-fatness is the frequencyof outcomes that are a given number of standarddeviations away from the mean. For example, for anormal distribution, outcomes that are farther than2.33 standard deviations away from the mean in eitherdirection have a 1 percent probability of occurringand events 2.58 standard deviations away from themean have only a 0.5 percent probability. Thus, onecan measure tail-fatness of returns of a given asset bycounting how often such extreme outcomes occurredin the sample.

The empirical evidence of fat tails in asset returnsis extensive. It was first demonstrated for daily stock-market returns by Fama (1965) and subsequentlyconfirmed in the academic literature for many assetreturns. Recently, Duffie and Pan (1997) measuredthe kurtosis and tail probabilities of daily returnsbetween 1986 and 1996 for 33 return series, including17 equity indexes representing 13 countries, exchangerates between the dollar and 12 foreign currencies,and 3 commodity series. They found evidence of fattails across all 33 series they examined. Interestingly,they found the returns on the U.S. dollar/Mexicanpeso exchange rate to have the fattest tails of all,with kurtosis of 217.5, or 20 moves beyond 5 stan-dard deviations and 5 moves beyond 10 standarddeviations.

Fat Tails and Options Pricing

Fat-tailed distributions not only play havoc withValue at Risk, but can also lead to errors in pricingoptions if the standard Black-Scholes model is used. Acritical determinant of the stock option price is thedistribution of the terminal price of the underlyingstock, that is, its price at the time the option expires. Ifthe returns on the stock are normal, then the terminalstock price has a lognormal distribution. If, on theother hand, the returns distribution has fatter tailsthan the normal, then the terminal price distribution

November/December 1997 New England Economic Review22

also has fatter tails than the lognormal. This intro-duces a bias into the Black-Scholes price of deeplyin-the-money and out-of-the-money options,4 in a waythat causes the model to underprice both.

Consider, for example, a deeply out-of-the-moneycall option (such as an option that gives the holder theright to buy a stock for $50 at the end of the month,while the stock is currently worth only $40). Thisoption will expire with a positive value only if thestock price increases by a large amount. The probabil-ity of such an increase depends on the right tail of theprice distribution at the end of the month. If the tail isfatter than assumed under the Black-Scholes model,then this probability is higher and the option is morevaluable than the Black-Scholes model would lead oneto believe.

Fat-tailed distributions not onlyplay havoc with Value at Risk,but can also lead to errors in

pricing options if the standardBlack-Scholes model is used.

An analogous argument applies to an out-of-the-money put, which will have a positive value only ifthe stock price drops significantly. The fatter the lefttail of the price distribution, the more likely the pricedrop. Thus, the Black-Scholes model underprices theout-of-the money put. The pricing bias also holds fordeep-in-the-money options because of the put–callparity, which is independent of the price distribution.If the call is out of the money, then the correspondingput is in the money, and vice versa. Thus, an in-the-money put will have the same pricing bias as anout-of-the-money call, and an out-of-the-money putthe same bias as an in-the-money call.

These pricing biases in option pricing exist if thedistribution of returns is fat-tailed but symmetrical,that is, the left and right tails are of equal thickness.However, this need not be the case. The returnsdistribution can be skewed, with one tail thicker thanthe other. In particular, it has often been observed that

in equity markets, crashes occur more frequently thansudden sharp increases in stock prices. This wouldimply that the stock returns distribution has a fat lefttail but a normal right tail. In this case, only out-of-the-money puts and in-the-money calls will be under-priced by the Black-Scholes model (Hull 1997, p. 494).

V. Three Recipes for Fat Tails

We can try to improve on the normal model byexplicitly modeling fat-tailed distributions and usingthese distributions for pricing options and for calcu-lating the probabilities of losses of certain magnitudes.The challenge is in choosing a fat-tailed distributionthat is both mathematically tractable and empiricallyrelevant. Among the most widely used approachesto modeling fat-tailed distribution are stable distribu-tions, jump diffusion, and stochastic volatility.

Stable Distributions

Stable distributions (also known as Pareto-Levyor stable-Paretian), first described by Levy (1924), area class of bell-shaped symmetrical distributions ofwhich the normal is a special case. Generally, non-normal stable distributions have narrower peaks atthe mean and fatter tails than the normal distribu-tion. The fat-tailed distribution illustrated in Figure 4alongside the normal is Cauchy distribution, whichcan be described as follows:

f~x! 5 S 1pD g

g2 1 ~x 2 d!2 ,

where g 5 1 and d 5 1.Stable distributions imply long-range correlations

in asset returns and are characterized by trends,cycles, and discontinuous changes. The application ofstable distributions to financial asset returns was firstproposed by Mandelbrot (1963) and gave rise to alarge literature on the subject.5

The most striking feature of stable distributions isthat the variance and all higher moments do not exist.This feature also makes their use in modeling verydifficult, because of the crucial role the concept ofvariance plays in finance theory. Moreover, the histor-ical record of asset returns does not fit some empiricalimplications of stable distributions. For example, sam-

4 An option is in the money if it would lead to a positive cashflow if it were exercised immediately. Similarly, an out-of-the-money option would lead to a negative cash flow if exercisedimmediately.

5 See Campbell, Lo, and MacKinlay (CLM) 1997, p. 17 for moredetail and a list of references.

November/December 1997 New England Economic Review 23

ple estimates for variance drawn from random vari-ables that follow stable distributions would not con-verge to a single number as the sample gets larger, butwould keep increasing without limit. In practice, esti-mates of variance of sample returns usually convergeas the sample gets larger. In addition, the use of astable distribution to model short-term returns impliesthat long-term returns would also follow the samedistribution. In practice, unlike short-term returns,long-term returns are often well-approximated by thenormal distribution (CLM 1997, p. 19). As a result, theuse of stable distributions in finance has always beencontroversial and recently has been supplanted byother models such as jump diffusion and stochasticvolatility, which produce fat tails while preserving thefinite variance.

Jump Diffusion

The jump diffusion model was introduced byMerton (1976). This model assumes only that returnsare IID. In particular, the returns usually behave as ifdrawn from a normal distribution but are periodically“jumped” up or down by adding an independentnormally distributed shock. The arrival of these jumpsis random and their frequency is governed by thePoisson distribution with a given expected frequency.

Fortune (forthcoming) estimates the jump diffu-sion process for the daily returns on the S&P 500. Hefinds that the volatility of returns arises both from thenormal process and from the jumps, with the jumpsaccounting for 80 percent of overall return volatility.He also compares intraweek daily returns with re-turns over weekends and finds that jumps occur morefrequently on weekdays and are, on average, positive.In contrast, weekend jumps are, on average, lessfrequent, of larger size, and negative. This is consistentwith companies’ announcing bad news over week-ends, hoping to minimize its impact on their stockprices when the markets next open.

The advantage of the jump diffusion model is thatit can make extreme events appear more frequently.Duffie and Pan (1997) illustrate the impact of jumps onrisk measurement by comparing a normal distributionof returns with a constant standard deviation of 15percent, to an otherwise identical process that hasjumps that occur with an expected frequency of once ayear and come from a normal distribution with a zeromean and a standard deviation of 10 percent. Despitethe low frequency of jumps, this is equivalent in riskto a standard normal distribution of returns with astandard deviation of 185 percent. The jump process

makes extreme events far out in the tail more likely.For example, in the above model, one can expect tolose overnight at least a quarter of the value of one’sposition. In contrast, as Duffie and Pan point out, inthe normal model “one would expect to wait farlonger than the age of the universe to lose as much asone quarter of the value of one’s position overnight.”6

Stochastic Volatility

Stochastic volatility means that the volatility isnot constant but undergoes random changes overtime. This results in a fat-tailed unconditional distri-bution of returns, while the conditional distribution(that is, the distribution of returns at time t conditionalon all previous returns) remains normal. The mostpopular class of stochastic volatility models isGARCH, and the most popular type of GARCH isGARCH (1,1). GARCH stands for Generalized Autore-gressive Conditional Heteroskedasticity. The model isautoregressive because it involves regressing the vari-ance on its own past values; heteroskedasticity simplymeans changing variance. The first version of this typeof model, known as ARCH, was introduced by Engle(1982) and the generalized, or GARCH, version, wasdeveloped by Bollerslev (1986).

GARCH (1,1) relates the variance of asset returnsat time t to the variance of asset returns at time t 2 1and the excess return at time t. The equation is:

s t2 5 v 1 ae t21

2 1 bs t212 ,

where st is the volatility of asset price at time t, et is theexcess return, that is, the difference between the actualreturn on the asset at time t and the average return,and v, a, and b are constants. This model is referred toas GARCH (1,1) because volatility at time t dependsonly on the return and the volatility at time t 2 1 andnot on the path they have taken in the past. In thismodel, b is the persistence parameter, which deter-mines how much carryover effect the previous peri-od’s volatility has on today’s volatility. The threeparameters can then be estimated from the historicalreturns data.

Implied Volatility—Smiles and Scowls

A different approach to volatility estimates calcu-lates them from option prices. This can be done

6 Duffie and Pan credit Mark Rubinstein with the use of thislanguage to describe the expected frequency of an overnight loss ofthis magnitude in the normal model.

November/December 1997 New England Economic Review24

because in the Black-Scholes model a one-to-one cor-respondence exists between the price of any optiontraded in the market and the volatility of the under-lying security that went into the calculation of thatoption price. Thus, if the option price is known, thevolatility can be “backed out” from the Black-Scholesformula. (The formula cannot be solved analytically,but the volatility can be found by plugging in differentvalues for it until one gets arbitrarily close to theoption price.)

The volatility calculated from the option price isknown as “implied” volatility. If one knows the priceof an option on a certain underlying security and itsimplied volatility, one could, in principle, calculate theprice of any other option on the same underlyingsecurity. Moreover, one could also use the impliedvolatility in value-at-risk calculations. The advantageclaimed for using implied volatility rather than histor-ical volatility in pricing options and measuring valueat risk is that it is forward-looking, reflecting actualexpectations of market participants rather than thepast. The advantages of implied volatility, however,are more apparent than real. The main reason for thisare the simplifying assumptions in the Black-Scholesmodel. Recall that the model assumes that the volatil-ity of the underlying security is constant. Thus, iftraders really were using the Black-Scholes to priceoptions, the implied volatility for any underlyingsecurity would be identical for all options on it. Inreality, nothing could be further from the truth. Op-tion prices on the same underlying security implydifferent volatilities, depending on the option’s expi-ration date and strike price. Implied volatility firsttends to decrease with the strike price and then toincrease, resulting in the so-called volatility “smile.”Figure 5 shows a hypothetical example of a volatilitysmile. Its symmetry is usually characteristic of cur-rency options (Hull 1997, p. 503).

Implied volatility plots for stock index optionsresemble crooked “scowls” rather than smiles. Stockindex option volatilities are highest for deeply out-of-the-money options and they keep decreasing, themore options get into the money. Figure 6 showsseveral implied volatility plots for call options on theS&P 500 index traded on June 10, 1997. The S&P 500index traded at 862.88 when these volatilities werecalculated. Each plot shows the variation in impliedvolatility for options with different strike prices butthe same expiration dates. The different curves repre-sent different months of option expiration. The steep-est slope is for the nearest expiration month (in thiscase, June) with the more distant expiration months

having progressively flatter slopes. The relationshipbetween implied volatility and time to expiration isknown as the term structure of volatility, by analogywith the term structure of interest rates.

The differences in implied volatilities among dif-ferent strike prices and expiration dates reflect notonly market views of volatility, but other consider-ations. They may depend, in part, on differences inoption premiums caused by variations in supply anddemand for different kinds of options. This is partic-ularly true for deeply in- and out-of-the-money op-tions, as they tend to be infrequently traded. Impliedvolatilities for different times to expiration tend toconverge for near “at-the-money” options because ofthe relative depth of their markets. Ultimately, how-ever, it must be kept in mind that despite theirubiquity, different implied volatilities for the sameunderlying securities calculated from the Black-Scholes formula do not have a well-defined economicmeaning. In fact, such volatilities reflect a logicalinconsistency, since they are calculated from a modelwhose fundamental assumptions are violated by theirvery existence. Implied volatilities can be useful, how-ever, if they are estimated from an options-pricingmodel that allows for time-varying volatility, pro-vided that we know which model was used to price

November/December 1997 New England Economic Review 25

the option, and provided that this model was correct.These conditions make implied volatilities more ap-propriate for trading options than for more generalvalue-at-risk calculations. Risk managers concernedwith value-at-risk often prefer historical data becauseof their desire to have independent models rather thanrelying on pricing models used by traders.

VI. Choosing the Model

The use of historical data to model fat-taileddistributions of asset returns has gained ground inmodeling value at risk. In particular, jump diffusionand stochastic volatility models are appealing becausethey can predict “tail events” more accurately than thenormal distribution. Nevertheless, the normal distri-bution continues to be used despite its frequentlyproclaimed and well-documented flaws. The appeal ofthe normal model lies in its simplicity. This may notseem like a good excuse, given its less-than-reassuringtrack record; however, the appeal of the normal be-comes more obvious once we consider the practicaldifficulties in implementing the alternatives. To usethe normal distribution, one needs to estimate onlyone unobserved parameter from the historical data,namely the variance. In contrast, the two alternative

models described above, jump diffusion and GARCH(1,1), each require estimation of three unobservedparameters. The GARCH model requires the estimatesof v, a, and b. The jump diffusion model requires theestimates of the variance of returns, the variance ofjumps, and the expected frequency of jumps. Thus, itis not enough to choose a model structure that hap-pens to describe the stochastic process generating thedata reasonably well. It is also necessary to estimate itsparameters correctly, and if one or more parameters ofthe model are misspecified, the resulting model is notgoing to be an improvement over the normal model.

VAR and the Risk of Catastrophic Events

Would a VAR measure based on a stochasticvolatility model such as GARCH have done any betterthan the normal model in estimating the probability ofa catastrophic event such as the Peso Crisis? Theanswer depends on the amount and appropriatenessof the historical data used for estimating the model. Itwill be recalled that the volatility of the peso/dollarexchange rate was very low during the year leadingup to the crisis. It was low no matter how it wasmeasured and what sort of structure was imposed onthe data. The peso/dollar exchange rate simply didnot change very much in the year leading to the

November/December 1997 New England Economic Review26

dramatic devaluation, and no model can find thevariability that is not there. The obvious choice is touse historical data further in the past. In particular,Mexico has had currency devaluations since the mid-dle 1970s. Possibly, 20 years of daily exchange ratedata would provide an estimate of the probability ofthe latest devaluation, and one model might do sobetter than another. However, using 20 years of dailydata in every market in which a bank is trading isimpractical, and in general 20-year-old data usuallywould not be relevant.

The preceding observations are not meant tosuggest that no information was available in 1994 thatsuggested a possible devaluation. Such information,however, was not to be found in the volatility of theexchange rate but in the political climate and macro-economic policies of the Mexican government. Re-cently there has been renewed interest in the causes of

The basic model as applied tocurrency crises was developed byKrugman in 1979. It attributes

speculative pressures oncurrencies to the inconsistencyof the government’s monetary

and fiscal policies with itsexchange rate target.

currency crises, spurred in part by the events inMexico. The basic model as applied to currency crises,however, was developed by Krugman in 1979. Itattributes speculative pressures on currencies to theinconsistency of the government’s monetary and fiscalpolicies with its exchange rate target. Specifically, suchinconsistency can arise if a country runs budget defi-cits while its central bank finances them by expansion-ary monetary policy. The expanding supply of localcurrency relative to the U.S. dollar tends to depress thevalue of the local currency. If the central bank wants todefend the value of its currency and keep it pegged tothe U.S. dollar, it must buy it with its foreign currencyreserves. But its foreign currency reserves are notinfinite, and once they are exhausted, the currency canno longer be defended. Rational investors will foreseethat the reserves will eventually be exhausted and

seek to profit immediately by selling the currencybefore the reserves are depleted. Krugman’s theorypredicts that the collapse of the exchange rate peg willoccur as soon as investors become convinced that thegovernment’s policies are unsustainable and mount aspeculative attack on the currency.

Mexican experience in 1994 is consistent withKrugman’s model of a speculative attack. At the time,Mexico had a large and growing current-accountdeficit and relied on an overexpanding money supplyand short-term borrowing. These weaknesses weremasked for a time by large capital inflows fromforeign investors which allowed Mexico to maintainthe value of the peso and maintain its foreign ex-change reserves. In 1994, however, Mexico experi-enced a great deal of political instability, notably thepeasant uprising in Chiapas and the assassination ofpresidential candidate Luis Donaldo Colosio. Politicalturmoil led to a loss of confidence by foreign investors,who withdrew their money. As the Mexican govern-ment sought to defend the value of the peso, itsforeign exchange reserves were rapidly depleted, forc-ing the devaluation.

Catastrophic Events and Stress Testing

While no method of modeling catastrophic eventslike the Peso Crisis is universally accepted, it is clearthat VAR is not sufficient and must be supplementedby other methods. These usually involve stress testingusing scenario analysis. Stress testing involves creat-ing a set of realistic scenarios and evaluating theirimpact on all current trading positions. Besides thePeso Crisis, other examples of catastrophic marketevents in the 1990s include the European currencycrisis precipitated by the decision of Great Britain andItaly to leave the Exchange Rate Mechanism (ERM) inSeptember of 1994, the sharp fall in the U.S. andEuropean bond markets in April of 1994, and the Kobeearthquake in February of 1995. In addition to usingscenarios based on actual past catastrophic events,stress testing can be supplemented by views of man-agement about possible future events or based oneconometric models of market trends. Stress testingcan then be used to define the limits on marketpositions the traders are permitted to take, and todetermine the level of diversification necessary tobring the impact of catastrophic events to acceptablelevels.

Thus, it is useful to make a distinction betweenthe risk of events that occur somewhat rarely (like 1out of 100 times) but are still a part of business as

November/December 1997 New England Economic Review 27

usual, and catastrophic events that happen very rarelyand whose probability cannot be estimated with accu-racy. The former risk is measured by VAR and thelatter by stress testing. These approaches are com-plementary, and both are necessary for modeling risk.The main difference between VAR and stress testingis that, unlike VAR, stress testing does not assignprobabilities to the scenarios being tested, whichmakes it difficult to compare results to VAR or to eachother.

VII. Conclusion

At the beginning of this article, we distinguishedbetween four types of models—macro models, micromodels, valuation models, and risk management mod-els such as Value at Risk. The bulk of the article hasfocused on valuation and risk management modelsand their crucial first step—modeling the statisticalprocesses of asset returns. We observed that modelingasset returns is complicated by changes in the param-

eters of the distribution, such as the volatility. Theseparameter shifts can be due to changing economicconditions, technical innovations, and fiscal and mon-etary policies. These structural shifts can inform ourstress tests and scenario analyses, but the real chal-lenge is formally incorporating them into the modelsof asset returns. Up to now, such attempts have notresulted in a significant improvement in the predictivepowers of the models of asset returns. The mostfruitful approaches so far have been the modeling ofasset returns as fat-tailed distributions, including sta-ble distributions, jump diffusion, and stochastic vola-tility models. However, a trade-off almost alwaysexists between the realism and the analytical tractabil-ity of the model. Striking the right balance in the faceof this trade-off, and maintaining it through changingmarket conditions for different financial instruments,is more art than science and requires considerableexperience and judgment. Bank regulators recognizethis and as a result place more emphasis on qualitativeaspects of risk management than on the content ofmodels themselves.

References

Black, Fischer and Myron Scholes. 1973. “The Pricing of Options andCorporate Liabilities.” Journal of Political Economy, vol. 81 (May–June), pp. 637–59.

Bollerslev, Tim Peter. 1986. “Generalized Autoregressive Condi-tional Heteroskedasticity.” Journal of Econometrics, vol. 31, pp.307–27.

Campbell, John Y., Andrew W. Lo, and A. Craig MacKinlay. 1997.The Econometrics of Financial Markets. Princeton, NJ: PrincetonUniversity Press.

Duffie, Darrel and Jun Pan. 1997. “An Overview of Value at Risk.”The Journal of Derivatives, vol. 4, no. 3 (Spring), pp. 7–49.

Engle, Robert F. 1982. “Autoregressive Conditional Heteroskedas-ticity With Estimates of the Variance of UK Inflation.” Economet-rica, vol. 50, pp. 987–1008.

Fama, Eugene F. 1965. “The Behavior of Stock Market Prices.”Journal of Business, vol. 38 (January), pp. 34–105.

Fortune, Peter. 1996. “Anomalies in Option Pricing.” New EnglandEconomic Review, March/April, pp. 17–40.

———. Forthcoming. New England Economic Review.Group of Thirty. 1993. Derivatives: Practices and Principles. Washing-

ton, DC.Hull, John C. 1997. Options, Futures, and Other Derivatives. Third

Edition. Upper Saddle River, NJ: Prentice Hall.Krugman, Paul. 1979. “A Model of Balance of Payments Crises.”

Journal of Money, Credit and Banking, vol. 11, pp. 311–25.Levy, P. 1924. “Theorie des Erreurs. La Loi de Gauss et Les Lois

Exceptionelles.” Bull. Soc. Math., vol. 52, pp. 49–85.Mandelbrot, Benoit. 1963. “The Variation of Certain Speculative

Prices.” Journal of Business, vol. 36, pp. 394–419.Merton, Robert C. 1976. “Option Pricing When Underlying Stock

Returns Are Discontinuous.” Journal of Financial Economics, vol. 3(March), pp. 125–44.

Paul-Choudhury, Sumit. 1997. “This Year’s Model.” Risk, vol. 10, no.5 (May), p. 19.

Taleb, Nassim. 1997. Interview in Derivative Strategies, vol. 36,December/January, pp. 37–40.

November/December 1997 New England Economic Review28