Alternative Intelligence Q

uotient Issue 33, Decem

ber 2009

7

The Performance of the SFA Score vs Traditional Risk-adjusted Performance MeasuresPeter Urbani, Infiniti Capital

Peter Urbani is Chief Investment Officer of Infiniti Capital, a Hong-Kong-based hedge fund of funds group.

Ever since the seminal work on Portfolio Theory by Harry Markowitz (1959) and the subsequent work of William Sharpe, the measurement of portfolio returns has been inextricably linked to the level of risk associated with achieving those returns.

This has led to the introduction of a number of risk-adjusted performance measures (RAPMs), most famously the reward-to-variability, or Sharpe Ratio.

Typically calculated as the portfolio returns in excess of those of the risk-free rate over the standard deviation of portfolio returns, the Sharpe Ratio embeds the concept of the variance, standard deviation squared, or volatility as the appropriate measure of ‘risk’ to use.

Over time, practitioners and academics alike have realised that this poses a number of problems for the accurate measurement of ‘risk’. In fact, the use of variance was largely an act of convenience to simplify the math in the days before computers. Markowitz himself has said that for some investors semi-variance might be a more appropriate measure to use.

The reason for this is simply that variance, or standard deviation as is more commonly used (the square root of the variance), is not a measure of ‘risk’ at all, but rather a measure of uncertainty. Standard deviation suffers from a number of well known deficiencies, most particularly the fact that it does not differentiate between good (upside) ‘risk’ and bad (downside) ‘risk’. Moreover, it is a symmetric measure that assumes both upside- and downside-variance are the same.

In recognition of these deficiencies, a number of other RAPMs have been developed to better address these issues. Probably the best known of these is the Sortino Ratio which replaces the standard deviation with the downside deviation or second lower partial moment (LPM2) as the denominator in the Sharpe Ratio.

Still others include the modified Sharpe Ratio where the denominator of risk is represented by the Cornish Fisher expanded or ‘modified’ Value at Risk (VaR).

More recently, Shadwick and Keating pioneered the use of the Omega Functio,n sometimes also used as the Omega Ratio. In this formula, the area under the probability curve in excess of some threshold return is taken over the area under the curve of the downside part of the distribution. This can be calculated in either a discrete form using empirical data, or a continuous form by fitting a distribution.

8

From a practitioner’s perspective, what we care most about is how well these measures predict the relative ranking from one period to the next and whether or not using one particular method produces superior returns to another. Although these RAPMs are typically used for calculating the relative ranking of funds they can also be, and often are, used as the objective function in direct portfolio optimisations. For instance, maximising the value of your portfolio’s Sharpe Ratio is the same as minimising its variance and gives the same set of weights as the classical Markowitz mean variance optimisation formulation. Minimising your ‘normal’ VaR will give the same solution. However, as mentioned previously, measures based on standard deviation such as Sharpe and the normal VaR calculation do not consider the asymmetry of returns.

In this article, we examine the performance of a new risk-adjusted performance measure called the Single Fund Analysis (SFA) score, developed by Infiniti Capital. This measure is a weighted average of a number of underlying statistics that has also been standardised to a reference data set making it both a relative and conditional measure. The SFA score can further be decomposed into risk, return and persistence sub-scores.

The study referenced below, compares the out-of-sample performance of a portfolio built using the SFA score as its objective function versus the performance of portfolios built from the same data set using the Sharpe, Sortino and Omega measures. For reference we also include a naive benchmark made up of an equally weighted continuously rebalanced portfolio of all of the 36 underlying hedge funds in the selection universe. The portfolios are re-optimised to the objective function and rebalanced on a quarterly basis.

The results of the study suggest that the SFA score is capable of generating annualised rates of returns (CAGR) of around 15% versus those of around 11.5% for the Sharpe Ratio, 13% for the Omega Ratio, and just 10% for the Sortino Ratio.

More importantly, although the Sharpe Ratio of the resultant time series remains better for both the Sharpe and Omega portfolios, the ratio of the annualised return to the absolute drawdown over the period, which is arguably a better measure of realised risk to return, remains highest for the SFA score portfolio.

Key statistics SFA Total Omega Sharpe SortinoEqually-

weighted fund

CAGR 15.09% 12.93% 11.44% 10.07% 10.36%

Annual return 14.34% 12.35% 10.97% 9.80% 10.08%

Annual SD 6.50% 5.07% 4.35% 5.72% 6.12%

Skew 0.78 -0.13 0.4 -0.21 -0.67

Kurtosis 0.00 -0.39 0.67 0.29 0.52

Normal VaR 95% -1.89% -1.38% -1.15% -1.90% -2.07%

Infiniti VaR 95% -1.25% -1.43% -1.15% -1.99% -2.30%

Sharpe Ratio 1.77 1.86 1.85 1.20 1.17

CAGR/ABS (Drawdown) 3.41 2.39 3.27 1.05 0.93

Max drawdown -4.43% -5.41% -3.50% -9.56% -11.10%

Best-fit distribution Gumbel (Max)Johnson

(Lognormal)Mixture of

normalsModified normal

Log normal (max)

Portfolio’s relationship to benchmark (equally-weighted fund )

Correlation 0.79 0.87 0.78 0.91 1.00

Beta 0.84 0.72 0.56 0.85

Alpha (monthly) 0.49% 0.42% 0.45% 0.10%

Information Ratio 1.03 0.76 0.23 -0.11

Alternative Intelligence Q

uotient Issue 33, Decem

ber 2009

9

The reason for the superior performance of the SFA scores is due to it not being a simple point estimate, but being calibrated relative to a reference data set of other hedge funds. This improves the predictive power of the method because it responds dynamically to market conditions. In order to ensure the availability of data for the SFA reference scores, the portfolios are also optimised with a one-month data lag. This means that January SFA scores which only become available in February are used to obtain the March opening portfolio weights.

Unlike traditional performance measures, the SFA score is both conditional on the time period being used and relative to a large reference data set of other hedge funds. Where other methods typically standardise everything back to a normal or Gaussian distribution, the IAS uses the best fitting distributions throughout. This has the effect of calibrating the range of scores more closely to real-world data.

Of course, the method is not perfect. The SFA scores will not provide the best returns over each and every single time period, however, over any meaningful length of time they will tend to out-perform.

We were somewhat surprised by the poor performance of the Sortino Ratio portfolio relative to that of the Sharpe Ratio portfolio. We believe this may have been due to the fact that the quality of the 36 underlying hedge funds used was very good. This enabled the computer to select portfolio weights that gave a portfolio with zero downside deviation in the in-sample optimisation periods. The low variance of these portfolios did not persist out-of-sample in the subsequent periods causing this portfolio to underperform.

This is a classical problem of over-fitting your data which results in there being little relationship between the in-sample period and the performance in the next period. The Sharpe Ratio suffers from a similar problem, but more because it is capturing only the linear effects of the portfolio whereas we know there are significant non-linear effects present in hedge funds.

The SFA score is able to capture some of these non-linear artifacts because of the statistics used in its calculation and the best-fitting non-normal distributions it uses. These have the effect both of improving the predictive power of the method and ensuring the resultant pay-off is positively skewed, or as close to positively skewed as possible. This translates into more upside risk than downside risk.

10

Subsequent studies, where we have used less well-performing hedge fund indices, have confirmed our beliefs and the intuitive expectation that the Sortino Ratio should out-perform the Sharpe Ratio.

The SFA score has been used by Infiniti to manage real portfolios over the past two years. All of the calculations used to obtain these results, the dataset used, and a trial version of the software used, are freely downloadable from www.infiniti-analytics.com for third parties to evaluate.

Peter UrbaniInfiniti Capitalpeter.urbani@infiniti-capital.comTel: 64 3 977 8811

References:

The Infiniti SFA score as a RAPM, Peter Urbani (2009), www.infiniti-analytics.com/kb/kb/article/infinitisfascorearapm.

Portfolio Selection: Efficient Diversification of Investments, Harry Markowitz (1959), http://cowles.econ.yale.edu/P/cm/m16/index.htm.

Sharpe Ratio, William Sharpe, http://en.wikipedia.org/wiki/Sharpe_ratio.

Sortino Ratio, Frank Sortino, http://en.wikipedia.org/wiki/Sortino_ratio.

Omega Ratio — A Universal Performance Measure, Keating and Shadwick (2002), www.performance-measurement.org/KeatingShadwick2002a.pdf.

Modified Sharpe Ratio, www.andreassteiner.net/performanceanalysis/?External_Performance_Analysis:Risk-Adjusted_Performance_Measures:Modified_Sharpe_Ratio.