Operational Risk:Operational Risk: The Case of Hedge Funds

Bing LiangUniversity of MassachusettsUniversity of Massachusetts

Shanghai Advanced Institute of Finance (SAIF)Finance (SAIF)

Definition of Operational Risk• Basel II: Operational risk is the risk of loss resulting from inadequate or failed processes,resulting from inadequate or failed processes, people and systems or from external events. 

• JP Morgan Chase: Operational risk is the risk ofJP Morgan Chase: Operational risk is the risk of loss resulting from inadequate or failed processes or systems, human factors or external events.

• European Commission: The risk of a change in value caused by the fact that actual losses, i d f i d t f il d i t lincurred for inadequate or failed internal processes, people and systems, or from external events (including legal risk) differ from theevents (including legal risk), differ from the expected losses.

Hedge FundsHedge Funds Proprietary trading and sophisticated research I t i l k d l ti l illi id k t Invest in less‐known and relatively illiquid markets Market prices sometimes not used/reliable Model‐based prices/self pricingModel based prices/self pricing

Active use of intermediaries Frequent trades (30% NYSE trading volumes) Leverage via prime brokerage (30 times for LTCM)

Lightly regulated Less regulatory oversight Less regulatory oversight Low transparency

– It is estimated that about half hedge fund failures are due to operational risk

AIMA Questionnaire for Due Diligence (DD)I t t i f tiInvestment manager informationInvestment researchExecution & tradingComplianceComplianceLegalAnti‐money laundering policyInsuranceInsuranceBusiness continuityFund informationData overviewInvestment strategyRiskInvestor service/reportingTaxationInsurance

Research QuestionsResearch QuestionsResearch QuestionsResearch Questions• What are the key variables (red flags) for• What are the key variables (red flags) for operational risk?

• Can we develop a score model for OR based onCan we develop a score model for OR based on red flags?

• Can the model predict fund performance andCan the model predict fund performance and failure, consistent with the Basel II definition?

• Should we use publicly available data or p yproprietary data?

Data ADVData ADVData ADVData ADV• Two main sources:• Two main sources:

– 9 versions of the Lipper TASS database – ADV forms from the SEC website during March and April 2006

• TASS Management companies were matched with ADV forms by both name and address y

• To verify a correct match, at least one fund from the ADV had to match a fund in TASSfrom the ADV had to match a fund in TASS

Summary Information (ADV)Summary Information (ADV)ADV TASSMean Median Mean Median Diff p-value

R t 0 96 0 83 0 93 0 79 0 03 0 22Return 0.96 0.83 0.93 0.79 0.03 0.22Std. Dev. 2.71 1.99 2.78 2.06 -0.07 0.11Autocorr. 0.14 0.14 0.12 0.13 0.02 0.00***Sharpe ratio 0.39 0.30 0.34 0.28 0.05 0.01**Mfee 1.38 1.50 1.44 1.50 -0.06 0.00***Ifee 17 13 20 00 16 27 20 00 0 89 0 00***Ifee 17.13 20.00 16.27 20.00 0.89 0.00***Assets 186.64 55.00 181.11 48.00 5.78 0.65Leverage 0.56 1.00 0.56 1.00 0.01 0.51Margin 0.46 0.00 0.44 0.00 0.03 0.18HWM 0.80 1.00 0.76 1.00 0.04 0.00***Lockup Period 4 36 0 00 3 55 0 00 0 87 0 00***Lockup Period 4.36 0.00 3.55 0.00 0.87 0.00***Sub. Freq. 36.02 30.00 34.43 30.00 1.76 0.01**Red. Freq. 83.00 90.00 69.21 30.00 14.34 0.00***

Tests and ResultsTests and ResultsTests and ResultsTests and Results• A “Problem” fund answered ‘yes’ to anyA  Problem  fund answered  yes  to any question on Item 11 of ADV.

f f d l ( )• Of 2,299 funds in our sample, 368 (16.0%) are defined as “problem.”– 126 of 879 (14.3%) management companies answered yesy

• Of the 10,295 total ADV registrations, 1,526 (14 8%) h d “ bl ”(14.8%) had a “problem.”

“Problem” vs. “Non“Problem” vs. “Non‐‐Problem” FundsProblem” Funds“Problem Funds” “Non-Problem” funds

Mean Median Mean Median Diff p-value

Avg. Return 0.77 0.68 0.91 0.79 -0.14 0.00***

Std. Dev. 2.50 1.66 2.71 2.02 -0.21 0.15

1 t O d AC 0 12 0 14 0 12 0 13 0 00 0 601st Order AC 0.12 0.14 0.12 0.13 0.00 0.60

Sharpe Ratio 0.28 0.25 0.36 0.26 -0.08 0.02**

AUM ($mm) 217 32 59 18 179 96 54 00 37 36 0 20AUM ($mm) 217.32 59.18 179.96 54.00 37.36 0.20

Age (Years) 5.60 4.50 4.96 3.83 0.64 0.01***

Min Invtmnt 0 96 0 50 1 28 0 50 -0 32 0 33Min Invtmnt 0.96 0.50 1.28 0.50 0.32 0.33

Mfee 1.37 1.50 1.38 1.50 -0.01 0.71

Ifee 15.25 20.00 17.49 20.00 -2.24 0.00***Ifee 15.25 20.00 17.49 20.00 2.24 0.00

HWM 0.69 1.00 0.82 1.00 -0.13 0.00***

Lockup 4.00 0.00 4.43 0.00 -0.43 0.21p

Conflict of InterestsConflict of InterestsProblem Non-Problem

Conflict of interests % Yes % Yes Diff p-valueBroker/Dealer 73 1 23 7 49 4 ***Broker/Dealer 73.1 23.7 49.4Investment Comp 50.3 15.8 34.5 ***Investment Advisor 73.9 41.6 32.3 ***Bank 40.5 9.8 30.7 ***Sponsor of LLP 56.8 21.5 35.3 ***BuySellYourOwn 30.7 8.3 22.4 ***BuySellYourselfClients 84.8 69.3 15.5 ***R S Y O 75 5 50 4 24 1 ***RecSecYouOwn 75.5 50.4 24.1 ***AgencyCrossTrans 31.2 2.3 25.1 ***RecSalesInterest 22 6 15 7 6 9 ***RecSalesInterest 22.6 15.7 6.9RecBrokers 46.7 38.0 8.7 ***OtherResearch 81.0 70.5 10.5 ***

Concentrated OwnershipConcentrated Ownership

Problem Funds Non-Problem funds

Mean Median Mean Median Diff p-valueMean Median Mean Median Diff p value

Direct Owners 9.96 9.00 7.33 6.00 2.63 0.00***

Controlling 8.28 7.00 5.97 5.00 2.31 0.00***Controlling 8.28 7.00 5.97 5.00 2.31 0.00

Percent 75% 0.73 1.00 0.50 0.50 0.23 0.00***

Domestic 0.80 1.00 0.49 0.00 0.31 0.00***Entity

Indirect Owners

2.33 1.00 1.37 0.00 0.96 0.00***Owners

Leveraged 0.51 1.00 0.57 1.00 -0.06 0.03**

Margin 0 35 0 00 0 49 0 00 0 14 0 00***Margin 0.35 0.00 0.49 0.00 -0.14 0.00***

Person Capital ($mm)

1.26 0.00 2.62 0.00 -1.36 0.02**

Leverage and Problem FundsLeverage and Problem FundsRaw Problem Non-Problem Diff p-valueLeverage 0.51 0.57 -0.06 0.03**Avg. Leverage 52.20 85.31 -33.11 0.00***Max Leverage 96.82 140.68 -43.86 0.00***

No FOFLeverage 0.61 0.61 -0.00 0.87Avg. Leverage 63.73 95.57 -31.84 0.01***Max Leverage 118.27 158.80 -40.53 0.04**

5% WinsorizedAvg Leverage 43 34 65 15 -21 81 0 00***Avg. Leverage 43.34 65.15 21.81 0.00Max Leverage 81.46 108.33 -26.87 0.00***

ImplicationsImplications• Problem funds are more likely to relate toProblem funds are more likely to relate to conflict of interest structures

bl f d h d• Problem funds have more concentrated ownership and less leverage

• Lenders are able to detect the problem funds and are unwilling to lendand are unwilling to lend

• Regulating hedge funds won’t help the lenders

Probability of Having Problems Probability of Having Problems Model 1 Model 2 Model 3

Log Assets 0.014 -0.022 -0.007HWM -0.199 ** -0.114 -0.149Mean Return 0.059Incentive Fee 0 037 *** 0 038 *** 0 036 ***Incentive Fee -0.037 *** -0.038 *** -0.036 ***Relationship 0.759 *** 0.652 ***AgencyCrossTrans 1.400 ***g yRecSecYouOwn 0.345 *** 0.374 ***BuySellYourOwn 0.695 ***Other Research 0.294 *** 0.226 **PercentOwner75 0.551 ***Direct Domestic 0 134 ***Direct Domestic 0.134 ***Pseudo R-squared 3.89% 16.59% 25.30%Num Obs 1986 1986 1969

Appraisal Ratio and Operational RiskAppraisal Ratio and Operational RiskProblem Non-problem Combined

Log Assets 0.073 *** 0.047 *** 0.011Log Assets 0.073 0.047 0.011Fund Age 0.009 0.016 *** -0.014Stdev. -0.018 -0.040 *** 0.010

Onshore 0.120 * 0.061 ** 0.151 *Incentive Fee 0.003 0.009 *** -0.023 **HWM -0.018 -0.056 -0.123Relationship -0.251 *** 0.023 -0.426 **Di t D ti 0 026 0 027 0 049Direct Domestic 0.026 -0.027 -0.049PercentOwner75 -0.081 -0.075 ** -265 **

Adj. R-squared 18.02% 6.76% 7.08%Num Obs 273 1,369 279

Observable Proxy for Operational RiskObservable Proxy for Operational RiskObservable Proxy for Operational RiskObservable Proxy for Operational Risk

• ADV operational risk variables were not• ADV operational risk variables were not available prior to 2006

• We attempt to use the observable TASS variables to proxy for the ADV variables p y

• The canonical analysis finds a rotation of both sets of variables that maximizes thesets of variables that maximizes the correlation between the two data sets.

A A UnivariateUnivariate Measure of Operational RiskMeasure of Operational RiskTASS ADVTASS ADV

Previous Returns -0.27 *** AgencyCrossTrans 0.06 **

Previous Std Dev -0 35 *** RelBrokerDealer 0 28 ***Previous Std. Dev. 0.35 RelBrokerDealer 0.28Fund Age -0.07 *** RelInvestComp 0.24 ***

Log of Assets 0.13 *** RelInvAdvisor 0.24 ***gReports Assets 0.12 *** RelPartSponser 0.30 ***

Incentive Fee -0.88 *** BuySellYouOwn 0.08 **

Margin -0.29 *** BuySellYourClient -0.08 ***

Audited -0.19 *** RecSecYouOwn 0.33 ***

*** ***Personal Capital -0.29 *** RecUnderwriter 0.26 ***

Onshor -0.05 *** RecSalesInterest 0.28 ***

OpenToInv 0 08 RecBrokers 0 33 ***OpenToInv 0.08 RecBrokers -0.33Accepts Mgd. Accts. -0.13 *** PercentOwner75 0.15 ***

Corr ADV & TASS 0.42 *** DirectDomestic 0.31 ***Co & SS 0 4 ect o est c 0 3

Operational Risk Predicts ReturnsB‐G Style Dummies

Year coefficient t‐value

1994 2 28% 2 20*1994 ‐2.28% ‐2.20*

1995 0.10% 0.12

1996 ‐3.27% ‐4.76**

1997 ‐2.61% ‐3.71**

1998 0.42% 0.60

1999 0 13% 0 141999 ‐0.13% ‐0.14

2000 ‐0.18% ‐0.25

2001 ‐0.42% ‐0.95

2002 ‐1.48% ‐4.43**

2003 ‐0.41% ‐1.12

*2004 ‐0.67% ‐2.45*

2005 ‐0.11% ‐1.31

Average Value ‐0.92% ‐2.66*g

Average. Adjusted R‐squared 40.17%

Average Number of Observations 1,027

Regression Results Based on the Cox Proportional Hazards ModelProportional Hazards Model

N ω‐score Chi‐sqr age Chi‐sqr

Convertible Arbitrage 491 0.04685 0.30 ‐0.00036 ‐0.15

Dedicated Short Bias 85 0.80538 2.73 ** 0.00583 1.34

Emerging Markets 778 0.33043 4.23**

‐0.00513 ‐2.07*

Equity Market Neutral 649 ‐0.07736 ‐0.99 ‐0.00690 ‐3.23 **

Event Driven 1196 0.16691 1.76 ‐0.00739 ‐4.79

**

Fixed Income Arbitrage 493 0.36735 2.36 * ‐0.01668 ‐4.03 **

Fund of Funds 2281 0.08577 1.36 ‐0.00729 ‐5.45 **

Gl b l M 506 0 16105 1 45 0 00440 1 75Global Macro 506 0.16105 1.45 ‐0.00440 ‐1.75

Long/short Equity 3936 0.16229 3.33 ** ‐0.00746 ‐7.86 **

Managed Futures 1046 0.19395 2.77 ** ‐0.00803 ‐6.66 **

All (ex FOF) 9180 0 17825 6 39 ** 0 00672 11 91 **All (ex FOF) 9180 0.17825 6.39 ‐0.00672 ‐11.91

Predicting Hedge Fund LifePredicting Hedge Fund Life

The ω scoreThe ω‐score 

• Developed by Brown Goetzmann Liang and• Developed by Brown, Goetzmann, Liang, and Schwarz (2008) to capture operational risk for h d f dhedge funds.

• Based on the relation between the SEC filing gdata (ADV) and TASS data

• Won the prestigious FAJ Graham and Dodd• Won the prestigious FAJ Graham and Dodd Award for 2009

Cover Page of the CFA Magazine (Nov./Dec. 2010)

Investor Flows and Operational RiskInvestor Flows and Operational RiskInvestor Flows and Operational RiskInvestor Flows and Operational Risk

• Previously we found lenders could• Previously we found lenders could differentiate problem and non‐problem funds.

• Another test is whether investors can differentiate these two fund typesyp

• We utilize the ω‐score variable and a flow analysis to investigateanalysis to investigate.

Operational Risk and investor FlowsOperational Risk and investor FlowsCoeff t-value Coeff t-value

Low Rank 0.596 6.10 *** 0.634 7.30 ***Mid Rank 0 977 6 89 *** 0 981 6 79 ***Mid Rank 0.977 6.89 0.981 6.79High Rank 0.905 11.69 *** 0.886 7.39 ***Std. Dev. -0.022 -5.44 *** -0.023 -5.79 ***Category Flows 0.685 9.80 *** 0.686 9.63 ***Log Assets -0.117 -6.45 *** -0.118 -6.52 ***

** ***Mgmt. Fees -0.045 -3.06 ** -0.044 -2.89 ***ω-score -0.010 -1.61 0.025 1.02Low Rank/ω -0 019 -0 15Low Rank/ω 0.019 0.15

Mid Rank/ω -0.202 -1.78 *

High Rank/ω 0.085 0.62g /

Avg. Adj. R-sq. 14.00% 14.16%

Avg. Obs. 966 966

Data‐DD and TASS/CISDMData DD and TASS/CISDM• Data provided by HedgeFundDueDiligence.com

• 444 due diligence reports (100‐200 pages)• 403 distinct advisors, 2003‐2008

• Data from• Offering document and marketing materials• Offering document and marketing materials• On‐site interviews with the manager• Questionnaire completed by the manager• Verification of facts with administrator• Check authenticity of audit with auditor• Background check on management and key staffBackground check on management and key staff

• Supplement data with TASS/CISDM

Correlation Analysis on TASS and DD VariablesCorrelation Analysis on TASS and DD Variables

TASS/CISDM Variables DD VariablesP i R t 0 19** Mi t t t 0 23**Previous Returns -0.19** Misstatements 0.23**Previous Std. Dev. -0.18** SignIQ 0.091st Order Auto-corr. -0.21** Big4Auditor -0.90**Fund Age -0.26** Pricing -0.39**Log of Assets -0.54** Internal Accounting 0.42**M t F 0 28** Mi t t t *Si IQ 0 19**Management Fee -0.28** Misstatements*SignIQ 0.19**Incentive Fee -0.24** Misstatements*Big4Auditor -0.06Leverage -0.50** Misstatements*Pricing -0.01Lockup 0.54** Misstatements*Internal Accounting 0.40**Advance Notice 0.25**

Correlation Between TASS and DD Panels 0.48**

ConclusionConclusionConclusionConclusion• The ADV data and DD data allow for quantifying operational risk for hedge funds to build scoreoperational risk for hedge funds to build score models.

• Find a strong statistical relation between problems andFind a strong statistical relation between problems and operational risk variables such as potential conflicts, concentrated ownership, leverage, internal pricing, auditing quality and misrepresentationauditing quality, and misrepresentation.

• Operational risk proxies had a more dramatic effect on problem fund returns.p

• Find ADV data seems to be redundant for lenders.• Hedge fund investors are unable to distinguish between 

h h d l l k f d h hhigh and low operational risk funds without the ADV filings. Hence the filings are useful for the investors.

Conclusion (Cont’d)Conclusion (Cont d)

• Operational risk can be used to effectively• Operational risk can be used to effectively predict hedge fund failure

• Compared with investment risk, operational risk is more important in predicting hedge p p g gfund failure