Simon Willis - OrX

7/28/2019 Simon Willis - OrX

1/31

Taking value from

sharing data

4 June 2009


2/31

Page 2

ORX is a not-for-profit industry association headquartered in Zurich, Switzerland

ORX was founded primarily as a platform for securely sharing high quality operational risk

loss data

This is still what we do but ORX also works with its members to:

develop operational risk management practice set common standards for the industry develop professional networks

conduct leading edge research

ORX now has 52 members from 18 different countries

The ORX Global Banking Database now contains approx. 124,000 loss events to a value of

approx. 40 Billion

What is ORX?


3/31

Page 3

Banks have always managed operational risk

Operational risk management began in 1999 (maybe)

What have we spent the last 10 years doing and what have we achieved:

Better definitions

Better data

Better tools

Better measures and models

Better management

How as a discipline have we added value to our industry and our firms?

How as a discipline will we add value to our industry and our firms?

What is operational risk management for?


4/31

Calculate

Operational Risk Evolution

Op Risk first discussed asidentifiable risk class

Basel II initiative begins

Early movers createORM function

Industry working groupsformed IIF, ISDA, ITWG

Governing principlesestablished

Vision created to betterunderstand op risk

1999 -

2001

The Aspirational Years

Regulatory rules finalized

AMA qualification process

begins

Complexity of challengebecomes real

Firms struggle with costs,implementation and value

Value-added analytics

Efforts overshadowed byfinancial crisis

2005 -

2008

Pursuit of Value Begins

More banks form ORM function

Implementation begins

Basel II gains momentum

QIS provides insights

Sarbanes Oxley + / -

Significant losses incurred

ORX formed

Compliance vs. riskmanagement debate emerges

Risk quantification begins

2002 -

2004

The Development Years


5/31


6/31

Page 6

Risk management and risk measurement are fundamental activities

Risk models are only as good as the decisions that get made based upon them

Models only answer questions, dont ask them

Risk managers need to:

Think more broadly and challenge assumptions

Look at the specific but also understand the links and interdependencies

Learn from the past but know that the future can be very different

Recognise the importance of communication

Operational risk needs to think about its own role and mandate

Continuing value in improved risk measurement, opportunity for improved risk management

What might change in the future?


7/31

Is there a role for external data

Fundamental value of external loss data is that it offers a larger sample than your own

experience

To make no mistakes is not in the power of man; but from the errors and mistakes ofothers the wise and the good learn wisdom for the future

Plutarch

Banks can and do incorporate external loss data

To supplement internal data in quantitative models

Inform scenario analysis

Benchmark performance relative to peers

Validate the adequacy of internal data and capital

ORX now looking at risk data not just loss data


8/31

Calculate

Loss Data Has Been a Catalyst of Change

Data collected sporadically, if at all

No requirements or standard

practices

No loss profile or time series ofdata

Anecdotal reporting

Culture of blame

Limited transparency andawareness

Limited engagement of businessexecutives in control environment

No mechanism for data sharing

Before

Firm-wide loss data collection

Standard definitions and

recording standards established

Time series of data developing

Incorporated in risk reporting and

business MIS

Culture shift to risk management

Greater transparency, escalationand accountability

Significant attention by regulatorycommunity

After


9/31

Calculate

Loss Data Enables a More Analytic Approach

ExternalLoss Data

(ORX)

InternalLoss Data

BaseCapital

StatisticalModel

Risk-basedCapital

QualitativeAdjustment

1. CALCULATE BASE CAPITAL 3. ASSESS CAPITAL

APPROPRIATENESS

2. QUALITATIVE

ADJUSTMENT

Internal Losses

Exter nal Losses

Int ernal and

External

Benchmarks

Data helps eliminates the challenges of subjectivity, repeatability and statisticalincorporation of results

Scenario

Analysis


10/31

Overall Summary of ORX Annual Data

10


11/31

ORX Global Membership (May 2009)

ABN Amro

Banco Bilbao VizcayaArgentaria

Banco Pastor

Banco Popular

Banco Portugus de Negcios,Banc Sabadell

Bank Austria Creditanstalt

Bank of America

Bank of Ireland

Bank of Nova Scotia

Barclays Bank

BMO Financial Group

BNP Paribas

Bradesco

Caja Laboral

Cajamar

Caixa Catalunya

Lloyds TSB Bank plc

National Australia Bank

Northern Trust

PNC

Postbank

Rabobank

Royal Bank of Canada

Royal Bank of Scotland

Santander

Skandinaviska EnskildaBanken

Standard Chartered

State Street

TD Bank Financial Group

US Bancorp

Wachovia Corporation

Wells Fargo

WestLB

Caixanova

Capital One

Commerzbank AG

Credit Agricole

Danske Bank A/S

Deutsche Bank AG

Dresdner Bank AG

Erste Bank

Euroclear Bank

First RandFortis

Grupo Banesto

Hana Bank

HSBCHBOS plc

ING

Intesa SanPaolo

JPMorgan Chase


12/31

Total Number and Value of Losses by Year

12

Total Number and Value of Losses by Last 6 Quarters


13/31

Loss Events Frequency (2002-2008)

13

Internal

Fraud

External

Fraud

Employment

Practices &

Workplace

Safety

Clients,

Products &

Business

Practices

Disasters &

Public Safety

Technology

&

Infrastructure

Failures

Execution,

Delivery &

Process

Management

Malicious

Damage Total

% of

Total

Corporate Finance 21 112 141 308 1 5 330 0 918 0.74%

Trading & Sales 95 268 401 645 18 670 11,091 0 13,188 10.64%

Retail Banking 4,153 39,725 8,101 7,822 844 1,201 16,279 194 78,319 63.16%

Commercial B anking 185 4,207 393 1,669 59 261 4,440 1 11,215 9.04%

Clearing 52 530 123 105 3 156 1,768 0 2,737 2.21%

Agency Services 16 55 96 159 5 61 2,412 0 2,804 2.26%

Asset Management 52 110 141 586 10 76 2,206 1 3,182 2.57%

Retail Brokerage 209 161 515 1,856 10 55 1,222 1 4,029 3.25%

Private Banking 152 414 165 1,541 25 66 2,651 2 5,016 4.05%

Corporate Items 34 301 667 315 215 76 974 10 2,592 2.09%

Total 4,969 45,883 10,743 15,006 1,190 2,627 43,373 209 124,000

% of Total 4.01% 37.00% 8.66% 12.10% 0.96% 2.12% 34.98% 0.17%

1%5% 5%10% >10%


14/31

Gross Loss / 100 Gross Income 2002-2008

14

Gross Loss / 100 Gross Income by Last 6 Quarters


15/31

Business Line Ranking

15


16/31

Page 16

The challenges of using external loss data

Fundamental challenge of external loss data is that it is not your own experience

Key challenges when using external loss data include:

Banks are different: size, location, business mix, control environment

Banks collect data differently: categorisation, truncation point, currency

Confidentiality limits information data is anonymous, detail is limited

Important however not to overstate the differences between banks


17/31

Page 17

Loss data homogeneity

What similarities exist in the size and shape of the loss distributions from Members

Similarity is measured in terms of:

Statistical measures of goodness-of-fit among loss distributions

Reduction of error in predicting large losses as a result of using pooled data rather thaninternal data alone

Overall ORX data showed the following results:

A high level of homogeneity was evident in the shapes of various loss distributions

across all levels in the sample

Simple scaling relations were effective in aligning many loss distributions

Pooling losses among banks with similar loss distributions can result in (estimated at

20-30%) error reductions when estimating high quantiles of the loss severity distribution


18/31

Dealing with heterogeneity among data sources

How can we compare losses across ORX banks?

If the Bank of America reports a $1 million loss in External Fraud / Retail Banking,does that mean that the Bank Austria faces the same probability of such a loss?

The value of consortium data increases with banks ability to translate otherslosses into their own

Solution: development of loss scaling models

We determine the degree of similarity among distributions of various categories of lossesand adjust for differences in business line, location ,size of bank and size of loss:

Same distribution pool the data

Same distribution after applying a simple scaling relation scale the data, thenpool

Different distributions do not pool the data, build separate loss models

We have developed loss severity models for each loss category based on scaled and

pooled data


19/31

Corporate Finance Internal Fraud

Sample Analysis

Differences in loss scale were often evident across regions


20/31

Example: Scaling helps compare losses from

different regions

Distributions of (log)-losses occurring in North America and Western Europe for Private

Banking losses in Clients, Products and Business Practices. Applying a simple scale

factor to North American losses brings the two distributions into close alignment.

(Taken from Cope, Eric, and Simon Wills. External Loss Data Helps: Evidence from the ORX Database.

OpRisk &

Compliance. March 1, 2008.)

4 5 6 7 8

0.0

0.2

0.4

0.6

0.8

1.0

Clients, Products , and Business Practices / Private Banking

Log LOSS (Euro)

EmpiricalCDF

North AmericaWestern Europe

4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5

0.0

0.2

0.4

0.6

0.8

1.0

Clients, Products , and Business Practices / Private Banking

Log LOSS (Euro)

Emp

iricalCDF,

ScaledData

North America (Scaled)Western Europe

Clients, Products, and Business Practices / Private Banking

Log Loss (Euro) Log Loss (Euro)

Cumula

tiveProbability

Cumula

tiveProbability

Raw Losses

North America

Western Europe

Scaled Losses

North America

Western Europe

LowSeverity

HighSeverity

LowSeverity

HighSeverity


21/31

Observed correlations across loss categories are

low Observed correlations of total quarterly losses across loss categories are small

Avg correlation for business line pairs: 6.6% (st dev 18.3%)

Avg correlation for event type pairs: 5.8% (st dev 18.5%)

Avg correlation for business line / event type combination pairs: 5.9% (stdev 17.2%)

Over 80% of banks correlation matrices are not distinguishable from consortiumaverage

Most individual banks may use the consortium correlation matrix in place of theirown

Histogram of Kendall's Tau Severity Corrs

Correlation Value

Frequ

ency

-0.6 -0.4 -0.2 0.0 0.2 0.4 0.6

0

200

40

0

600

800

Curve shows a normaldistribution with matching

mean and variance

Standard deviation is

within theoretical expected

range, based on amount of

available quarterly data

values

Histogram of Correlations among Quarterly Total

Losses by Business Line / Event Type pair

Kendalls Tau

Frequency

.06

.17


22/31

ORX is taking these building blocks and lessons learnt and building out our capacity to

measure risk across our membership

Increasing value from ORX data

Empirical quantiles

%pointsoffitteddistribution

20,000 50,000 200,000 500,000 2,000,000 10,000,000

10

50

90

99

99.9

99.97

99.99

99.999

IF1

1) Fit Severity Distributions

9.0 9.2 9.4 9.6 9.8 10.0 10.2

5

6

7

8

9

EL 4: Lo g TOTAL INCOME: After QM

Log TOTAL INCOME

LogLosses

2) Apply Scaling Models

Num Losses

Density

5 10 15 20

0.00

0.05

0.10

0.15

3) Fit Frequency Models

5) Apply Correlation Models

4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5

0.0

0.2

0.4

0.6

0.8

1.0

IF1

x

Fn(x)

6) Estimate annual totalbank losses

4) Compute aggregate lossdistributions


23/31

Calculate

More progress can be made together

Industry-wide benchmarking

Critical event analysis

Business unit benchmarking

Trend analysis

Correlation with KRIs

Correlation with environment

Dynamic reporting

Industry-wide risk measure

Business unit / risk type measure

Op risk correlation analysis

Peer group / homogeneity analysis

Use of Scaling analysis

Forward looking measures

Cross risk-class correlation

Risk Measurement Risk Management


24/31

Our objective is to identify differences between firms and, at a granular level, identify andmeasure those factors that are driving the difference

What happened, to who, how much and why

Loss severity drivers such as jurisdiction, type of counterparty or claimant, role of the firm

ORX is supporting the creation of sub-sets of data and sub-sets of members

Looking to:

Improve ability to select relevant event data

Support improved peer group benchmarking

Support improved scenario analysis

Facilitate better discussion with the business

ORX Risk Management Tools


25/31

Page 25

Active review of the data requirement s for the Global Database

Product and Process

Require reporting per loss by end 2010

Exposure Indicators

Considering plans to expand / vary the Exposure Indicator data collected by Business Line

Large Loss Events

Work has begun on a review of data requirements for Large Loss Events (>10 million)

Seeking to establish process for the common categorisation of Large Loss Events

Developing Data Requirements


26/31

Large Loss Event Template - Illustration

Type:

Auction Rate Securities

Context: Bonds were issued where the coupon was periodically re-set based upon theyields set by the auction of reference instruments. At the time of issuance the reference

instruments auctions were heavily over-subscribed, but soon after were under-subscribed

raising concerns about yields and market liquidity.

Description: Investors claimed that conditions had materially changed and the basis upon

which they bought the bonds no longer prevailed. An unanticipated quality option, in theform of the market for the reference instruments, had materialised.

Resolution:

A number of issuing banks agreed to buy-back the issued bonds at par at the

next auction of the reference instruments.

Recordable Loss Amount:

Regulatory Fine

Categorisations:

Operational Risk Event, Operational Risk Loss

26

Business Line Corpor at e Finance/Corpor at e Finance

Event Type Cli ent s, Pr oduct s,. . /Pr oduct Flaw

Product Capi t al Rai si ng /Bond Issuance

Process Market Pr oduct s &

Services

Causes: Assumpt i on aboutst at us quo

Control Type Preventat ive

Control Failure Comp let eness of

Documentat ion

Scaling Vol ume of New

Products

Loss Severity Driver Tot al i ssuance si ze Business Environment Legal , Social

Claimant Type Professional

Invest ors

Impact Balance Sheet

Growth


27/31

ORX is seeking to establish National and Sector database from within current

membership and as a service to future members

Working to establish a Canadian national service, with 11 members to establish an ORX

Insurance Sector Service, with 8 members to establish Investment Banking Service and

soon will invite membership of a Global Custody Sector Service and Global Fraud Service

National and Sector Services use the ORX legal, security and system platform but have

the capacity to:

Define own loss data categorisation and standards including: loss attributes;

text fields; business metrics and KRIs

Define own frequency of loss submission and distribution

Set own quality assurance testing and reporting regime

Create own reports and benchmarking

Use ORX global analytical tools and routines

ORX objective is to develop bespoke National and Sector services as business level tools

Creating new trend and comparative data directly relevant to business units and directly in

support of business decisions

Developing risk management tools


28/31

Page 28

ORX has operated the Spanish National Service since 1 January 2007 on behalf of 10members:

Banco Bilbao Vizcaya Argentaria

Banco Pastor

Banco Popular

Banc Sabadell

Barclays Bank

Caja Laboral

Cajamar

Caixa Catalunya

Grupo Banesto

Grupo Santander

Spanish National Service has own governance determining who can participate, settingquality standards, monitoring quality standards and setting data requirements

Collected approximately 28,000 loss events

Participation in the Spanish National Service charged at 5,000 per annum

Spanish National Service


29/31

ORX is working with 11 members to establish an ORX Insurance Sector Service

The Service will be strongly based on the Global ORX Database National but vary in terms

of:

Retain ORRS base loss data reporting standards and format

Business Lines add 4 new insurance business lines

Products add approx. 16 new insurance product types

Exposure indicators define 2 additional exposure indicators

Retain quality assurance testing and reporting regime varying only forchanges made

Retain reports and benchmarking varying only for changes made

Retain standard timetable and data cycle

Hope to launch invitation plus specification before end 2008

Sector Service Example: Insurance


30/31

Summary

We have made a great deal of progress in the last 10 years

The current crisis is an challenge and an opportunity

We need to continue to invest and improve risk measurement

We need to add more value as risk managers

Sharing data can help us move forward

30


31/31

Documents

Simon Willis - OrX