Looking beyond thecatastrophe model

THESE days, Karen Clark frequentlyfinds herself having to explain to journal-ists thathercriticismofcertainaspectsofthe catastrophe-modelling sector is notdirected at either the catastrophe modelsor at the risk modellers themselves. Thesituation largely reflects the rise in theprofiles of both Clark and her Boston-based catastrophe risk consultancy,Karen Clark & Company (KC&C) overthe past four years. It does, however,matter a great deal to her that she is notperceived to be disparaging an industryshevirtuallycreated fromscratchin1987when she devised the world’s first hurri-cane cat model and founded the first cat-modelling company, Applied InsuranceResearch (AIR), now AIR Worldwide.

She served as chief executive of AIRuntil 2007, when she left to set up KC&C.From the very beginning, the companyadopted a distinctly independent stancein relation to the risk-modelling indus-try; it is, notably, not slow to challengesome of the industry’s most deeply heldassumptions. It is an attitude that verymuch informs KC&C’s annual reports onthe performance of the near-term (overthe five-year period 2006 through 2010)hurricane models produced by the threemajor cat modellers: AIR, Eqecat andRisk Management Solutions (RMS).

Near-term models, Clark says, wereintroduced in 2006, following thedestructive and costly 2004 and 2005hurricane seasons. In the wake of hurri-cane Katrina there was a lot of “modelbashing”, she notes: “It was felt therewere a lot of factors the models wereunderestimating. At the same time, alot was being written on global warmingand its relationship to hurricanes. Sothe dynamics of the environment weresuch that the modellers really felt theyhad to look inside their models andreview the different components, andthat contributed to introduction of thesenear-term models in 2006 by all of themodelling companies.”

It is a development that reflects theimportance of catastrophe risk manage-ment for the US insurance industry.Catastrophe claims make up the largestcomponent of property losses today. “Inthe US homeowners’ insurance market,30% of the premium dollar is taken up byactual and expected catastrophe losses,”Clark explains.

The third and final KC&C report,which was published in January of thisyear, found the near-term models,designed to project insured losses inthe US from Atlantic hurricanes, havesignificantly overestimated losses forthe period. The two earlier reports came

to the same conclusion for the 2006through2008andthe2006through2009seasons. Interestingly, all the modellingcompanies more or less had the sameprojections for the firstyearof the2006to2010 near-term period. “All the modelspretty much said hurricane activityand losses were going to be 40% aboveaverage,whichisahugeamount,”shesays.

Little harddata

As it turnedout,hurricaneactivity for theprojectionperiodwaswellbelowaverage.For example, hurricanes making landfallin the US between 2006 to 2010 were 53%below average. The losses, Clark notes,were 70% below average. “We had mini-mal losses in every year except 2008. Sohow well did the models do? Our conclu-sion is they have not demonstrated anyskill. They have not shown any ability toaccurately project near-term hurricanelosses. And that is not surprising becausethere is very little hard scientific data tosupport these near-term projections.”

She has no doubt the modellers areusing the best science available. “But wealso know science can lead to numbersthat go awry and that is because there isnot a lot of data underlying that science.Scientists have the same problem actuar-ies do – if you don’t have a lot of data, youhave high uncertainty. So our approachat KC&C is to look at the numbers and atthe facts. There are so few facts we at leastshould be looking at those facts.”

One of Clark’s central arguments isthe problem is not with the models. “Themodels are great and we are not criticis-ing the modellers for developing thenear-term models, which have thepurpose of providing their clients withdifferent views of hurricane risk. Wetotally think that is the right thing to do.However, models are just models. Theydo the best they can. What is inappropri-ate is the extent to which the marketinghype oversold the near-term models andthe science underlying them.”

Global warmingAccording to Clark, an important factornot being taken into account is that themore sophisticated climate models areprojecting a decrease in hurricanes as aresult of global warming. “And that is astatement that comes from the Intergov-ernmental Panel on Climate Change[IPCC] in its most recent report. Now,thereissomeevidencestormsmaybecomemore intense over time because of risingsea-surface temperatures. And if that doesoccur, it is going to be a gradual increaseover time. It is not going to be a 40%increaseoverthecourseofayear.Sowecan

allagreethisisadevelopmentweshouldbemonitoring, but we have to be careful howweimplementit inourmodels.”

Clark says all the marketing hypearound the near-term models gave thefalse impression there was a generalscientific consensus hurricane losseswere increasing and were going to besignificantly above average for the period2006 to 2010. “RMS most aggressivelymarketed this new model as a replace-ment for the standard model. Ratherthan an alternative view, it said all com-panies should use this model.”

For its part, the industry is more orless compelled to take note of Clark’sinterventions. She is by far the mosthigh-profile and decorated figure inthe sector. In addition to a number ofindustry awards, in 2009 the NationalAssociation of Insurance Commissionersappointed KC&C as the lead consultantin developing a recommendation on thescope, timeline and potential costs ofbuilding a national catastrophe multi-peril model for personal lines risks in theUS. On the international front, she waspresented with an award certificate forthe 2007 Nobel Peace Prize bestowed onthe IPCC, an organisation with whichshe has worked since 1995. The IPCCchairman, RK Pachauri, said the IPCChad provided certificates of the awardonly to those who contributed sub-stantially to the work of the organisationover the years since its inception.

NoveltyThis is a far cry from the 1980s, whencatastrophe risk modelling was such anovelty one of Clark’s key roles was toexplain its objectives to the wider world,particularly to the insurance industry.

Clark recalls when she was at univer-sity in the early to mid-1980s, computershadjuststartedtobeusedforfinancialandeconomic modelling. “I loved using thecomputer to build models to generatefinancial information that could be usedtomakedecisions,” shesays.

After graduation, she worked with asmall group of about six or seven peoplein the research and development depart-ment of an insurance company in Bos-ton. “We were internal consultants. Ourrole was to come up with ways to help thiscompany deal with problems that werenot being addressed by traditional actu-arial and underwriting techniques.Catastrophe risk, of course, came underthat category. This company had signifi-cantcoastalexposuresandIwasgiventheproject of finding a better way of calculat-ing the losses it could sustain from a hur-ricane. That was how it all started. I just

fell in love with catastrophe modelling.One thing led to another and I ended updevoting my whole career to it.”

Not surprisingly, it was tough goingfor the first few years after she set up AIRin1987.Oneof the firstplaces shewent towas Lloyd’s. At that time, there were noprominent Bermuda companies andalthough there were some very big UScompanieswritingpropertycatbusiness,it was mainly written from London bythe Lloyd’s syndicates. One of Clark’sfirst contracts was with the reinsurancebroker EW Blanch (subsequentlyabsorbed into Aon Benfield) to develop acatastrophe risk model the companycould use for its clients.

WayoffClark says before she developed the firsthurricane model in the 1980s, insurersandreinsurersweregrosslyunderestimat-ing their potential losses by about a factorof10.“Theywerewayoff,”shenotes.

There were a number of reasons forthis. First, there had not been a majorstorm in a highly populated area for dec-ades.Second,inthemid-1980s,thelargestloss the insurance industry had experi-enced was slightly more than $1bn fromhurricane Alicia in 1983. Third, in 1986,there was a highly influential study by theUSAllIndustryResearchAdvisoryCouncil(AIRAC) which focused on the potentialfor a $7bn insured hurricane loss. “So thatnumber became the industry benchmarkfor a worst-case scenario,” Clark says. “Atthe same time, our hurricane model saidthe insured lossescouldreach$60bn.This

wasverydifferent fromwhat therestof theindustrywasthinking.”

TrackingexposuresBut, according toClark, themajor reasonthe industry was underestimating theirpotential losses by a factor of 10 wasbecause companies had stopped trackingtheir exposures in hazardous areas,particularly along the coastline. “Therehad been decades when there were nocatastrophe events and the propertyvalues had grown exponentially, so by thetime hurricane Andrew came in 1992,there were literally trillions of dollars ofexposurealongtheGulfofMexicoandtheUS East Coast and insurance companieswere simply not aware of the magnitudeof their exposures.”

Uniquely for the time, Clark’s catas-trophe model could simulate events andestimate what the damages could be atthe present time based on contempora-neous property values. “That was a reallyimportant component of the model. Asyou know, even today that is an issue interms of the under-evaluation of poten-tial losses,” she says.

HurricaneAndrewThe insurance market finally embracedcatastrophe modelling after hurricaneAndrew hit. “I remember it like it wasyesterday. It made landfall at about 5 amon August 24, 1992, a Monday morning.We ran scenarios with our hurricanemodel to try and give our clients someestimate of what the losses could be andby 9 am that morning we issued a state-ment that insured losses could exceed$13bn. Our clients simply did not believeit. Of course, the storm’s total lossesended up coming in at more than that,between $15bn and $16bn.”

Clark was besieged by phone calls,especially from underwriters in the Lon-donmarketwhowereconvincedthemax-imum loss figure would not be more than$6bn, particularly as Andrew had madelandfall south of Miami. “The responsewas: ‘A fewmobilehomesandanAirForcebase, how much can it be?’” According toClark, it would take nearly a year after thehurricane made landfall for the industryto fully appreciate the potential of catas-trophe modelling. “But it eventuallyclicked. The industry realised these mod-

The Japanese earthquake exposed the limitations of cat models. Here, theindustry’s founding figure talks toRASAADJAMIEabout how catastrophe-modelling companies can improve the accuracy of their loss projections

els were telling us something very valua-ble and we needed to wake up and figureout how we can best make use of them.”

RatingagenciesThe over-reliance on cat models by therating agencies, particularly their reli-ance on point estimates, is anotherimportant themeforClark. “The industryhas become wedded to these one-in-100and one-in-250-year numbers. So manydecisions are hanging on these point esti-matesandthen,ofcourse,whenthepointestimates change by 100%, by 200%, eve-rybody isat sea.Theratingagencies thinkthey are being consistent because theyare using a modelling approach. But dif-ferent models, different model versionsand different levels of data quality all leadto very different numbers.

The rating agencies claim they makeadjustments for these differences, butthey can’t really be sure they are makingthe right adjustments to be able to com-pare likewith like.Sooneof themessagesof KC&C is the rating agencies certainlyneed a different approach to be able toeffectively compare the financialstrength of different insurers.” The ten-dency of the rating agencies, sheexplains, is to be conservative. “Theirresponsibility is to give ratings on thefinancial health of companies, so they aregoing to be much more focused on thedownside, on the question of how badlycan this go wrong.”

Clark would strongly advise ratingagencies to adopt a robust set of trans-parent scenarios around characteristiccatastrophe events (such as the GreatNew England Hurricane of 1938 in thenorth-east of the US) to represent catas-trophe risk in each peril region.

These benchmark scenarios could beapplied to every company’s portfolio, sothe rating agencies are truly comparinglike with like. “Another characteristicevent for the New England region couldbe created by increasing the intensity ofthe 1938 New England Hurricane by 10%orby theamountscientists think iscredi-ble.” This is an approach Clark had rec-ommended to the rating agenciespreviously, but she now thinks, given theturmoil caused by recent model updates,theymightbea littlebitmoreopento thisidea. “What we have is a set of scientifi-cally defensible scenarios. So whilenobody knows what the right answer is interms of the numbers, we have a set ofcharacteristic events for each perilregion that credibly represents the risk,”she says.

Clark’s view is these characteristicevents are robust and are not going tochange frequently like the models do.

“And you can monitor the model changesrelative to the characteristic event sets.KC&C believes this is the future and isbetter than what the rating agencies aredoing now. At present, a rating agencycould be getting information that tells itone company is 100% higher risk thananother, when in reality that could bereversed,” she claims.

Over-specificationFor Clark, the cat-modelling industry isgoingdown the road of over-specification.Intryingtomodel thingsthatcannotevenbe measured, the loss estimates end upbeing highly volatile. “The cat modellerstalk about scientific knowledge – aboutwhat we know. At KC&C, we talk aboutwhat scientists don’t know. The cat mod-ellersneedtodothesamesothosewhousethe models can get a sense of the uncer-tainty in the data on which the models arebased. What scientists know is minusculerelative to what they don’t know, but it isnot necessarily in the modellers’ intereststotalkaboutwhat isnotknown.”

ButClarkbelieves tomanagecatastro-phe risk effectively, you need to have atleast an idea of what the range of uncer-tainty is in different peril regions. Shesees the Japan earthquake as a goodexample of why the industry should lookat other information beyond the models.She points out the modelling companiesdid not have a magnitude 9 event in theirJapan earthquake models in the seismicregion where the main event occurred.“Nor did they have a large-magnitudeearthquake combined with a major tsu-nami. And they certainly did not have anuclear disaster,” she adds.

“But if you had taken a few smartunderwriters before the event and putthem in a room for a few days to come upwith the most extreme and worst-casescenarios for Japan and given them thehistorical data relevant to Japan, theyprobably would have come up with amagnitude 9 or higher magnitude eventwith an associated tsunami. They wouldhaveknownapproximatelyevery15yearsthere is a magnitude 8 or greater earth-quake in or around Japan. And whilesomethinglikethishasnothadhappenedin Japan, there have been four earth-quakes of magnitude 9 or greater since1950alongtheso-calledRingofFire–themost seismically active region in theworld, of which Japan forms a part – thegreatestonebeingamagnitude9.5quakethat happened in Chile in 1960.”

Clark’s underwriters would also havebeen told large-magnitude earthquakesin Japan have caused tsunami waves of 30metres high and greater in at least threehistorically events. “If we gave our group

of non-scientists those facts, I am prettysure they would have come up with atleast a magnitude 9 earthquake, com-bined with a large tsunami wave, as aworst-case scenario. They may even havethought of the possibility of that [sub-sequent] nuclear disaster,” she says.

UpdatesBut many model users, she says, are notdoing this type of thinking because theyhave been lulled into this false sense ofsecurity, believing the models have fig-ured it all out for them. “The irony of thewhole thing is, now we know there couldbe a magnitude 9 earthquake, what do wedo? Are we going to wait for two or threeyears for the new earthquake models tocome out while the modellers updatetheir models? Why can’t we have a moreopen approach, so now we know we canhave a magnitude 9 earthquake, we canimmediately adjust our risk-manage-ment decisions to accommodate thatfact. Why do we have to wait several yearsfor the new models to come out?”

One reason why it takes the modellingcompanies so long to update their modelsis the models are overly complex. “Thereare only four major model componentsbut there are a great many variables, so anumber of experts and scientists aregoing to have to do more research. Thenthey need to get the results of theirresearchincorporated intothesecomplexmodels and they need to test it. For exam-ple, one of the things the scientists aresupposed to tell us is what the probabilityof this magnitude 9 earthquake is. Which,of course, is something they don’t know.It’s going to be a scientific guesstimate.They could call it a one-in-100-year, one-in-500-year or one-in-1,000-year event.So anything the scientists give us will justbebestguessbecause theydon’tknow.”

When you think about it, it is a bitcrazy. The models, she explains, are inone sense backward-looking toolsbecause they are constantly being cali-brated to the last major event, which isusually a couple of years old by the timethe new models are released.

MesmerisingFor Clark, the main issue for the industryis the science underlying these modelssounds so impressive. “You can go tothese presentations and get mesmerisedby all the scientific jargon. Companiesjustget lulled into this false senseof secu-rity the modellers have it all figured outto the extent that even when companieslook at numbers coming out of the mod-els that are obviously way off, they stillfeel compelled touse theminmanycases.So at KC&C, we inform companies aboutwhat reallyunderlies themodels in termsof the hard data versus research.”

She says the modelling companiescreate so much marketing hype aroundevery model update they make eachupdate sound like a major scientificbreakthrough. “So, there are all thesethings about what the scientists knowbut many updates are not based on newfactual knowledge. The updates incorpo-rate new research but typically it isresearch that has about the same level ofuncertainty as the previous research. Sothere is the overselling of the science andthe overselling of what scientists know.”

Clark says KC&C is helping compa-nies to understand two things: the truenature of catastrophe risk and the limita-tions of the catastrophe models (ie, whatmodelscanbeused forandwhat theycan-not be used for). The firm is also focusedon helping companies to access informa-tion generated outside the cat models sothey can be more informed about thescope and potential of their catastrophelosses.The idea is forcompanies tobe less

dependent on the seemingly endlesscycle of model updates and loss estimatesthat swing widely up and down.

To illustrate this, she refers to thelatest RMS model update. “One of the big-gest changes is the US inland hurricanelosses are much higher – of the order of200% to 300% in some cases. So this is anissueforalotofcompanies.Buttherearealot of companies saying they had knownfor a long time the RMS inland hurricaneloss estimates were too low and they hadbeen adjusting the numbers themselves.So, you have to ask yourself, if so manypeople in the market knew the RMSinland numbers were too low, did RMSknowthat?AndifRMSknewthattoo,thenwhydid it take it somanyyears to fix it?”

She cites another example in Massa-chusetts, where a previous hurricanemodel update dramatically changed thewind footprints for hurricanes in north-east storms. This model change signifi-cantly raised the cost of insurance incoastal areas, such as Cape Cod. Reinsur-ance costs soared. Most of the companiespulled out of Cape Cod, so today mosthomeowners are in the Fair Access toInsurance Requirements (Fair) Plan andnot being written by the private market.

“There are cases of homeowners onCape Cod who used to pay $800 for theirinsurance and are now paying more than$2,000 for the same property. Now thenew model says it is not really as bad as wethought and the coastal numbers aredecreasing. What does that say to thehomeowners in coastal areas?”

Cat models, she says, ultimately meanreal money to real people and that is whatpeople in the industry are missing if theyregard a model update as merely anotherchange in the numbers. “It may be finefor reinsurers, but if you are a primaryinsurer, it is no way to run a business.”

Outside the blackboxClark very much sees her present role asthinking outside the ”black box” of catmodelling. As far as she is concerned,catastrophe models have been takenaboutas farastheycangoandtheindustryis at a point where fresh insight is needed.“While cat models are great, they havesome significant limitations. We need todevelopotherapproachesandmethodolo-gies.”Thisuseofcredible informationandtools other than the cat models is anothercritically important theme for Clark.“There is nothing that says a cat model isalwaysthebesttoolsowehavetouseit.Wedon’t need to limit ourselves to one toolandwecanhaveotherapproaches.”

Cat models, she says, are very good forreinsurers that tend to write a globalbook of business. “Reinsurers are typi-cally not into the minutiae of every com-pany’s portfolio, so they can use thesemodels to obtain a general assessment oftheir risks. The cat models are very goodfor that because they are comprehensive.But the problem is they are a one-size-fits-all approach, which gives a verygeneral indication of risk, which may notprovide the most credible view of risk foraregionalorspecialisedbookofbusiness.But with the cat model everybody isstuck with the same solution. So if youare a primary insurer and you have a verylocalised or a specific book of business,the model generated loss estimates canbe way off. And, at present, there is no wayto fix the numbers.

Here she cites the real-life example ofFM Global, which writes a portfolio ofindustrial and commercial facilities.“The catastrophe models just don’t workfor its business, so it commissioned amodeller to create a special model for it.Now, most companies cannot afford topay the modellers millions of dollars tocreate their own model. So why don’t we

justhaveanapproachthatmakes it easierfor companies to be able to use their owndata and then tailor their risk assessmentaccording to their books of business.That is one area where modelling compa-nies can do better.”

The model estimates can also be off incertain peril regions. Since the modelsare based on historical data, Clarkbelieves insurance companies shouldhave access to the fundamental informa-tion about the historical events in theperil regions inwhichtheywritepropertybusiness. They should have a very visualand very scientific representation of howmany significant events have actuallyhappened in the regions.

“Everyone should know those statis-tics. It is not hard to know them. Andthese statistics should be generatedseparately from the models so they can beused to benchmark the output from themodels. For example, if I know histori-callysomethinghashappenedthatwouldgive me a loss of $2bn today and themodel is telling me my largest loss isonly $1bn, obviously the model is off. Or,vice versa, if the largest loss has been$100m and the model is telling me it isgoing to be $10bn, you know it could beup the other way. So we need to look atother information.”

In the ballparkGiven Clark’s scepticism, just how usefulhave catastrophe risk models been to theinsurance and risk-management indus-tries over the past five years?

Clark says when cat models werefirst introduced, the industry was way offin its estimation of catastrophe losses.“We were not even in the ball park. Thegreat thing about the cat models is theyhave got companies in the ball park andthat has been very valuable. But over thepast fiveyears,wehavenotbeenusingthecat models as if we are just in the ballpark. The way I have expressed it before isthat over the past 20 years, the use of themodels have improved from a handsaw toa chainsaw. A chainsaw is a great tool butit is not a surgical instrument. And youwould not try to do brain surgery with it.”

In Clark’s view, that is exactly what theindustry is trying to do with the modelsnow. “We think we have a surgicalinstrument with which to make pinpointcalculations. We don’t. We have a chain-saw, which is still a saw. So that is theproblem. We are trying to push the usageof the model beyond where it can go. It’stime we think about other approaches.”

Underwriter judgmentShe dismisses the argument the modelsare needed otherwise the insuranceindustry will be forced to go back to theold way of just relying on seat-of-the-pants underwriting judgment. That, shesays, is a big misconception. “If we don’thave a cat model, it does not mean wehave to go back to the old ways. What hashappened over the past 20 years is wehave gone from no models and all under-writer judgment to the other extreme: allmodels and no underwriter judgment.Neither extreme is optimal.”

Underwriter judgment, Clark notes,has earned itself a bad name but there aremany things about individual risks andaccountsunderwritersknowthatamodelwillneverknow.“Underwritershavevalu-able knowledge and expertise that shouldbe part of the risk assessment and man-agement process. It does not have to be aneither/or situation. We believe the idealapproach would use the discipline, struc-ture and scientific basis of the cat modelsbut, where appropriate, the knowledge ofunderwriters,engineers, losscontrolspe-cialists should be included. Science candoa lotbut it can’tdoeverything.”

New dawn: Japan earthquake models did not have amagnitude 9 event plus tsunami modelled – despitesuch an event being well within the realms of possibility

AP PHOTO/SERGEY PONOMAREV

Global Marketsincorporating Alternative Insurance Capital, The ReReport and World Insurance Report

Insurance Day ID GLOBAL MARKETS