8/4/2019 Perf1 Asset Allocation
1/44
Performance Analysis
Loriana Pelizzon
University of Venice
Overview
Asset allocation
Performance Analysis
Attribution analysis
Mutual Funds Analysis
Benchmarks
Peer group analysis
Style analysis
8/4/2019 Perf1 Asset Allocation
2/44
Asset allocation
how one decides to allocate assets among various
asset classes such as stocks, bonds, and cash. How do you decide how to allocate your assets?
The goal of asset allocation is to create adiversified portfolio with an acceptable level ofrisk and the highest possible return given that levelof risk.
A portfolio or asset allocation that maximizesreturn for the level of risk is called an efficientportfolio
Asset allocation
The most difficult aspect of this procedure is toaccurately predict expected returns.
In the case study we used long term historicalreturns, but is it reasonable to expect that historywill repeat itself?
There are a number of other ways to make suchpredictions.
One of the most promising methods is a modeldeveloped by Fischer Black and Robert Littermanwhile they were at Goldman Sachs.
8/4/2019 Perf1 Asset Allocation
3/44
Performance analysis
Risk-adjusted measures based on absolute
benchmark
Risk adjusted measures based on relative
benchmark
Risk adjusted measures based on
Customized Benchmark
Risk-adjusted measures based
on absolute benchmark
Sharpe ratio
Treynor ratio
returnexcessfundofdeviationstandard
returnexcessaveragesfund
RatioSharpe
'=
returnexcessaveragesfundRatioTreynor
'=
8/4/2019 Perf1 Asset Allocation
4/44
Risk-adjusted measures based
on absolute benchmark
Sortino ratio:
where:
deviationdownside
returnexcessfund'sRatioSortino =
( )[ ]returnexcessfund'sVARdeviationdownside ,0min=
Risk adjusted measures based
on relative benchmark
Morningstar Risk-Adjusted Rating.
Bill)TorReturnExcessCategory(AverageofHigher
BillTFundtheonReturnAdjustedLoadReturnrMorningsta
=
CategoryitsofrmanceUnderperfoAverage
rmanceUnderperfoAveragesFund'rRiskMorningsta =
8/4/2019 Perf1 Asset Allocation
5/44
Risk adjusted measures based
on relative benchmark
To calculate a funds summary star-rating,
the Morningstar Risk scores are thensubtracted from the Morningstar Return
scores.
Risk adjusted measures based
on Customized Benchmark
The information ratio, which is the ratio of excess
return to standard deviation of excess return (or
tracking error), is a measure of a managers skilland the consistency with which the manager has
been able to outperform
ErrorTracking
AlphaRationInformatio =
8/4/2019 Perf1 Asset Allocation
6/44
Manager performance: practice
One way to measure manager performance is
by looking at how an initial investment wouldhave grown over a specific time period
Manager performance: practice
8/4/2019 Perf1 Asset Allocation
7/44
Manager performance: practice
The rolling window graph is also helpful fordetecting structural changes in a portfolio.
A risk control process that was initiated bya manager several years ago, for example,would probably lower the portfolio'stracking error.
As a result, the symbols would shift to theleft as they do for Active Manager A (blue).
Manager performance: practice
8/4/2019 Perf1 Asset Allocation
8/44
Manager performance: practice
Manager performance:
practice
8/4/2019 Perf1 Asset Allocation
9/44
Manager performance:
practice
Manager performance: practice
8/4/2019 Perf1 Asset Allocation
10/44
Manager performance: practice
Attribution Analysis
To what do we attribute a managers
performance?
Is it stock picking, investing in the right
style, or market timing?
Were certain sectors over or
underweighted?
8/4/2019 Perf1 Asset Allocation
11/44
Attribution Analysis
Most of a managers returns are attributed to asset
class returns. A US equity managers returns depend mostly on
how well the US stock market does.
The second most important factor for an equitymanager is investment style.
Most growth stock managers perform well whengrowth stocks are in favor. Conversely theyperform badly when growth stocks are out offavor.
Attribution Analysis
The first goal is to find out how much of the
managers return comes from the general
market and investment style
We accomplish this using a technique called
style analysis
8/4/2019 Perf1 Asset Allocation
12/44
Attribution Analysis
Example.
Using only the monthly returns for The
Needham Growth Fund and the monthly
returns from the four Russell style indices
and T-Bills, we find the combination of
indices that best describes Needhams
behavior/style.
Attribution Analysis
8/4/2019 Perf1 Asset Allocation
13/44
Attribution Analysis
In Figure 2 the red portion of the pie chart shows
that these indices account for 77.5% of thevariance in Needhams return.
The variance of Needhams return that cant be
explained by the market and style is represented
by the green portion of the pie.
This residual variance or behavior is likely due to
the managers stock selection or sector bets.
Attribution Analysis
8/4/2019 Perf1 Asset Allocation
14/44
Attribution Analysis
The portion of the managers returns that are
explained by exposures to the style indices couldbe passively replicated by buying the appropriatepercentages of index funds or ETFs that representthe style indices.
The managers alpha is generated by the portion ofthe fund that we cannot passively replicate.
This represents the managers active bets.
They could be stock bets, sector bets, or evenmarket timing bets.
Attribution Analysis
In an attempt to identify these sources of returns, we startby constructing a custom benchmark called a stylebenchmark that is based on the index weights in Figure 1.
The performance graph and table (Figure 3) show thatNeedham beat its custom style benchmark by anannualized 16.92%.
This is the excess return Needham achieved over what wecould passively construct to represent Needhamsinvestment style.
This is the result of either manager skill or luck (how wedifferentiate between the two is explained in Mutual FundAnalysis). For now we assume manager skill.
8/4/2019 Perf1 Asset Allocation
15/44
Attribution Analysis
Attribution Analysis: stock
selection
8/4/2019 Perf1 Asset Allocation
16/44
Attribution Analysis
Should Needhams excess return be attributed to stock
selection, sector weightings, and/or market timing? To see the impact of sector bets we perform another style
analysis using sector indices rather than style indices.
Because we are using returns and a rolling window wedont expect to precisely identify the sector weights at anyspecific time but rather get an idea of what the sectorexposures have been over the life of the fund and how theyhave changed over time.
Attribution Analysis: sector
weightings
8/4/2019 Perf1 Asset Allocation
17/44
Attribution Analysis
Figure 4 above contains the results of the sector
analysis, which shows that Needham is heavilyweighted in technology, health care, and T-Bills.
T-Bills represent cash or anything that makes theportfolio behave like cash.
Based on the Needham Fund's prospectus the fundis run somewhat like a hedge fund. It shorts stocksand uses derivatives to reduce risk. Once again,using the exposure to the indices used in the style
analysis, we construct a custom style benchmark.
Attribution Analysis
8/4/2019 Perf1 Asset Allocation
18/44
Attribution Analysis
Figure 5 below shows that Needham
outperforms this benchmark by 13.84%annualized.
So of the 16.92% outperformance, about 3%
is from their sector bets.
The balance, 13.8%, is the result of either
stock selection or market timing
Attribution Analysis
8/4/2019 Perf1 Asset Allocation
19/44
Attribution Analysis
Market timing doesnt necessarily mean
moving from 100% stocks to 100% cash.
It can be as subtle as buying low beta stocks
when one perceives the market is over
valued.
It could also be a value manager building
cash because he cant find good valuations.
Attribution Analysis
8/4/2019 Perf1 Asset Allocation
20/44
Attribution Analysis
One way to evaluate the results of market timing is
to see how managers do in both up and downmarkets.
If a manager goes up more than the benchmarkwhen the benchmark goes up, the manager plotsabove the horizontal line in Figure 6 above.
If a manager goes down more than the benchmarkwhen the benchmark goes down, he plots to theright of the vertical line. If the manager goes down
less he plots to the left of the vertical line.
Attribution Analysis
Aggressive managers who go up more and downmore plot in the northeast quadrant. Defensive
managers who go up less and down less plot in thesouthwest corner.
Managers that go up more and down less, as is thecase with Needham, plot in the northwestquadrant. We believe that this is the result of goodmarket timing particularly if there is a consistentpattern of such behavior, as seen in Figure 7below.
8/4/2019 Perf1 Asset Allocation
21/44
Attribution Analysis
Attribution Analysis
Managers with bad market timing that go up
less and down more fall into the southeast
quadrant.
Needham went up 26% more than the
benchmark when the benchmark had a
positive return. When the benchmark went
down the fund declined about 22% less.
8/4/2019 Perf1 Asset Allocation
22/44
Mutual Fund Analysis
When investing in non-index mutual funds,
investors make two critical assumptions: 1) that skillful managers exist,
2) that they have the ability to recognize them.
If an investor is not willing to make these two
assumptions, they should invest in non-active
funds like index funds or exchange traded funds
(ETFs).
Mutual Fund Analysis
Mutual fund analysis, both qualitative and
quantitative, attempts to identify skillful active
managers. Qualitative analysis looks at factors such as the
background and experience of the manager and the
mutual fund company.
Here, we look only at the quantitative factors such
as manager performance, style, style consistency,
risk, risk-adjusted performance, etc.
8/4/2019 Perf1 Asset Allocation
23/44
Mutual Fund Analysis
What is the best way to analyze, and ultimately select,
mutual funds? Financial journalists are not equipped to analyze mutual
funds. In most cases they are simply reporting theperformance figures they received from the managersthemselves or the marketing/public relations people.
Mutual fund rating services are good data collectors butlack any real sophistication in fund analysis. Theseservices are oriented toward the retail fund investor.Consequently sophisticated advisors, plan sponsors andconsultants must perform their own mutual fund analysis.
Mutual Fund Analysis
The two biggest mistakes in quantitative
mutual fund analysis are improper:
benchmarking
end point bias.
How can you avoid these mistakes?
8/4/2019 Perf1 Asset Allocation
24/44
Benchmark
The most common error made when
measuring a managers performance is theselection of an improper benchmark.
Morningstars star ratings, for example, are
based on funds performance relative to a
broad group of fund returns, as opposed to a
more specific benchmark that reflects the
manager's true style.
Benchmark
Because of this, on February 28, 2000, at the very
peak of the growth stock bubble, most of
Morningstars five star funds were growth fundswhile there were no five star value funds.
Two years later, after the value funds did well and
the growth funds crashed, most of the five star
funds were value funds.
What makes a good benchmark?
8/4/2019 Perf1 Asset Allocation
25/44
Benchmark
At the heart of a quality manager analysis is a
good benchmark. In order for a benchmark to be avalid and effective tool for measuring a managersperformance, it must be:
Unambiguous
Investable
Measurable
Appropriate
Reflective of current investment opinions
Specified in advance
Benchmark
A benchmark with all of these characteristics is the stylebenchmark.
The style benchmark is the result of Nobel LaureateWilliam F. Sharpes returns-based style analysis.
The style benchmark is a custom benchmark produced byweighting a set of indices in a unique combination thatreflects the style of the manager.
The most important advantage of a custom stylebenchmark over a standard market benchmark is that itaccounts for the style characteristics of the manager.
8/4/2019 Perf1 Asset Allocation
26/44
Benchmark
If the manager specializes in small cap growth stocks thenthe benchmark should be made up of small cap growthstocks.
In fact, the ideal benchmark explains all of the returns ofthe manager that come from systematic factors such asstyle and market movements.
If this is the case, any performance over (or under) thebenchmark can be attributed to manager skill.
A benchmark that does not do a good job of capturing thestyle of the manager will always leave you wondering did the manager outperform because of style differenceswith the benchmark?
Benchmark
The investment industry uses a number of inappropriatebenchmarks, the most common of which is a manageruniverse or peer group.
Manager universes are not investable, not specified inadvance, and since they are made up of active managersthey are not the passive equivalent of an active manager.
Additionally, manager universes suffer from survivor bias(the poor performing managers drop out and / or aremerged with better performing funds).
Most importantly, they are usually too broadly defined toaccurately judge the skill of a specific manager.
8/4/2019 Perf1 Asset Allocation
27/44
Benchmark
Broad market indices such as the S&P 500,
Russell 3000, Wilshire 5000 etc. are notgood benchmarks for most active non-large
core managers.
Even style indices such as the Russell 1000
Growth, 1000 Value, 2000 Growth, or 2000
Value are not appropriate for the vast
majority of managers
Style Benchmarks vs. Market
Benchmarks
Style benchmarks are superior to singleindex benchmarks for the majority of
managers. Figures 1 and 2 show the result of a style
analysis of the Dodge & Cox Fund.
The combination of Russell style indicesand T-Bills that best defines the style of thisfund is shown in Figure 1.
8/4/2019 Perf1 Asset Allocation
28/44
Style Benchmarks vs. Market
Benchmarks
Style Benchmarks vs. Market
Benchmarks The Manager Style graph, shown in Figure 2, maps the
funds style relative to the four Russell style indices.
As explained earlier, the analytic technique that enables us
to determine the funds effective asset mix, create theManager Style Map, and build the custom style benchmarkis called returns-based style analysis.
The custom style benchmark is made up of the styles andweights shown on the bar chart.
For Dodge & Cox, the style benchmark is a compositereturns series made up of 2.1% in T-Bills, 76.5% in theRussell Large (1000) Value Index, and 21.4 % in theRussell Small (2000) Value Index.
8/4/2019 Perf1 Asset Allocation
29/44
Style Benchmarks vs. Market
Benchmarks
Can we prove that this is a
better benchmark?
A good test of a benchmark is to see howcorrelated the benchmarks returns are to themanagers returns.
The higher the correlation the better thebenchmark.
The red portion of the left pie chart on Figure 3shows the R-squared (correlation squared) of themanagers returns to the custom style benchmark,90.0%.
The pie chart on the right shows that the R-squared to the S&P 500 is only 67.5%.
8/4/2019 Perf1 Asset Allocation
30/44
Benchmark
The green portion of the pie chart measures the
variance in the funds returns that is not explainedby the benchmark. Notice for the S&P 500 it ismore than three times greater than for the stylebenchmark.
A good benchmark includes all of the systematicfactors (market, style) so that the unexplainedvariance is due exclusively to nonsystematic oridiosyncratic factors that are primarily the result of
the managers stock selection.
End Point Bias
The other common mistake made in performance analysisis called end point bias. Most of the funds recommended
by various financial publications are ones that recently
performed well.
When looking at cumulative statistics, recent performanceabove the benchmark creates the illusion that the fund hasconsistently outperformed.
Cumulative statistics are calculated through the mostrecent time period. Annualized return for one, three, five,and seven years, for example, is often used to evaluatemutual funds.
8/4/2019 Perf1 Asset Allocation
31/44
End Point Bias
Notice that the most recent year is included in all
of these periods. Due to the nature of thesestatistics, recent performance often hides pastperformance.
Here is an example.
The September 15, 2003 Forbes magazineheralded the Mairs & Power Growth fund as oneof the three best funds to own based on its longterm record. In Figure 1 the long term annualized
returns for this fund look quite good.
End Point Bias
8/4/2019 Perf1 Asset Allocation
32/44
End Point Bias
End Point Bias
Because this fund outperformed its benchmark (the S&P500 is a reasonable benchmark for this fund) for 2, 3, 5, 7,10, 15, 20, and 25 year periods, one would think that the
fund is a consistently good performer. Now look at Figure 2 below.
The red and green shaded area at the bottom of theperformance graph shows the cumulative return relative tothe benchmark.
If you had purchased this fund twenty five years ago youwould have spent all but the last couple of years belowyour benchmark.
It has only been the very good performance in the last fewyears that give it the high annualized rates of return found
in Figure 1.
8/4/2019 Perf1 Asset Allocation
33/44
End Point Bias
This is what we mean by an end point bias.
We could also call it the broken clock syndrome (abroken clock will be right twice a day).
Similarly, if a manager has been managing moneyfor twenty five years, even with no skill, there islikely to be several years of good performance.
If your end point (the date on which your analysisends) is particularly good, cumulative statisticsmay create the illusion of consistently goodperformance.
End Point Bias
8/4/2019 Perf1 Asset Allocation
34/44
End Point Bias
Would Forbes have recommended this fund
three years earlier?
We doubt it. Figure 3 shows us that through
February 2000 the fund had under
performed its benchmark by 831 percentage
points.
End Point Bias
8/4/2019 Perf1 Asset Allocation
35/44
End Point Bias
One way to avoid end point bias is to look at
rolling time periods. Figure 5 shows rolling three year periods of excess
returns.
Here you can see an almost equal amount of timeunderperforming and outperforming thebenchmark.
To have confidence that a manager is skillful andthat the skill will likely result in beating thebenchmark in the future, we prefer managers that
consistently outperform.
End Point Bias
8/4/2019 Perf1 Asset Allocation
36/44
End Point Bias
Lets take a look at a consistent
outperformer. Figure 6 shows theperformance of the Fidelity Low Priced
Stock Fund. It outperformed its benchmark
by 6.58% annually!
End Point Bias
8/4/2019 Perf1 Asset Allocation
37/44
End Point Bias
Mutual fund analysis
8/4/2019 Perf1 Asset Allocation
38/44
Mutual fund analysis
The bottom panel of Figure 6 contains some useful statistics. Onestatistical measure of consistency is tracking error, which is the
volatility (standard deviation) of excess return. All things equal, theless volatile the excess returns the greater the chance the manager isskillful rather than lucky.
Tracking error is used to calculate a risk-adjusted measure ofperformance called the information ratio.
The information ratio is the annualized excess return divided by thetracking error. The information ratio for the Fidelity fund is a very highat 1.48.
What are the chances that a manager could have achieved thisinformation ratio by being lucky? Part of the answer will depend onhow long she achieves a high information ratio.
The longer the good performance persists, the less chance of luck and
the more chance of skill.
Mutual fund analysis
Significance Level statistic measures the probability of luck vs.skill.
To have confidence that the manager was skillful and not just lucky thesignificance level should be at least 95%.
For the Fidelity Fund it is 100% (see the bottom panel of Figure 6).
The most important first step is to select the proper benchmark. If thatis not done all of the fancy statistics we have discussed will bemeaningless.
Investors can accurately measure a managers performance, evaluatethe consistency of the performance, and determine the probability thatthe managers performance is the result of skill.
Such an analysis dramatically improves the likelihood that our secondassumption our ability to pick skillful managers is true and in doingso that our selections may lead to superior future performance.
8/4/2019 Perf1 Asset Allocation
39/44
Peer Group Analysis
Investors often try to gauge a managers skill by
comparing the managers performance to theperformance of a group of similar managers.
This is typically called a universe or peergroup analysis. Many consider a universe oruniverse composite to be a good benchmark for amanager.
In fact, manager universes do not have thequalities that make a good benchmark.
They are not investable, they are not specified in
advance, and they are usually too broadly defined.
Peer Group Analysis
Universes also suffer from what is known assurvivor bias because many of the poorperforming managers drop out of the universe.
Poorly performing mutual funds, for example, areoften merged with more successful funds.
When this happens only the successful fundstrack record is maintained, so the poorperformance is not represented in any universethat includes the fund.
Once a product no longer exists, for whateverreason, it is dropped out of the universe.
8/4/2019 Perf1 Asset Allocation
40/44
Peer Group Analysis
Despite these limitations, peer group comparisons continue
to be popular with investors. Figure 1 is a peer group analysis using the standard
floating bar chart.
Notice that the floating bar graph shows where FidelityMagellan ranked in the universe for only six time periods.
The RHS graph shows where Magellan ranked everymonth for the last twenty five years.
Looking at the LHS graph one might conclude thatMagellan had never been in the bottom quartile of the
universe.
Peer Group Analysis
8/4/2019 Perf1 Asset Allocation
41/44
Peer Group Analysis
With the LHS graph we can see that there
are two such periods. Managers may not want to show all the
periods, so for them the floating bar graphmight be best.
Sponsors, consultants, and advisors shouldalways use the more comprehensive graphwhere no time periods can be hidden.
Peer Group Analysis
Another thing to be aware of is the
difference between cumulative time periods
(last one year, three years, five years etc)
and rolling time periods.
This relates to end point bias, which is
discussed in Mutual Fund Analysis.
8/4/2019 Perf1 Asset Allocation
42/44
Peer Group Analysis
Peer Group Analysis
There is no reason to limit peer group
comparisons to returns
8/4/2019 Perf1 Asset Allocation
43/44
Peer Group Analysis
Peer Group Analysis
8/4/2019 Perf1 Asset Allocation
44/44
Peer Group Analysis
A discussion of peer groups wouldnt be
complete without some discussion aboutuniverse construction.
One methodology is to create universes of
managers with similar styles.
Peer Group Analysis