Benchmark heresies
Heresy No 1: Benchmarks are not an appropriate way to measure investment skill
The idea of a benchmark, or a reference index that generates the return a hypothetical manager could make without any knowledge of the market, is deeply entrenched in the field of performance measurement. If you are a typical reader of this article, you probably have data available on thousands of benchmarks at the click of a mouse.
The reason benchmarks loom so large in most fund manager’s sights is simple; their reputation (and bonus) depends on the ability to construct portfolios of shares that show a higher return than a given benchmark. The Warren Buffets of the world who can do this consistently are said to have shown investment skill, whether in picking stocks or entire market sectors, and are rewarded accordingly.
Most equity benchmarks are put together by choosing stocks with the largest market capitalisation in the local marketplace. The weights in the benchmark are then made proportional to the company’s market capitalisation. It is as if the benchmark is set to be the entire stock market, excluding every company below a given valuation.
Stock indices are widely used as a convenient proxy to follow the gyrations of the stock market. As an objective way of measuring skill, however, the use of a benchmark as a target carries major disadvantages.
Specifically, it is easier to beat the benchmark at some times than at others, suggesting that outperformance on its own is not always real evidence of investor ability. For instance, suppose that a benchmark contains five small shares (10 per cent each of market cap) and one large one (50 per cent of market cap). Last year each small share rose by 20 per cent, but the large share fell by the same amount. The result: a market-weighted return of zero.
However, a manager who ignored these market weights and simply took equal weights in each share made a much healthier return of 13.3 per cent. In fact, this is the expected return of a portfolio where asset allocation can vary at random.
So where is the skill shown when investing in these shares? Surely a skilled manager ought to be able to make at least the return expected by chance?
This is the idea behind the random portfolio paradigm. (Try not to think of dartboards or monkeys at dealing desks at this point; it has a sound basis in statistical sampling theory.) The idea is to put together a representative sample of all possible portfolios that a manager could have built under their usual trading constraints. The average return of these virtual portfolios is then used as the benchmark performance, and the peer group is the distribution of all such portfolios.
Spreading the wealth
Why does the distribution matter? Suppose the expected return for two benchmarks was 15 per cent a year. For one benchmark, your random portfolio analysis tells you the standard deviation of returns was five per cent; for the other, it is one per cent.
Your fund manager now reports a 17 per cent annual return, two per cent above both benchmarks. That sounds impressive, but what sort of skill does this result indicate?
In the context of the first benchmark, two per cent outperformance is not particularly good. It is less than half a standard deviation above the mean, suggesting performance not much better than chance.
But the second result is much more impressive. In this case, a two per cent outperformance is two standard deviations above the return expected from chance, putting the manager in the top 2.5 per cent of all possible portfolios.
This second result was unlikely to be a fluke, and even less so if the manager can show a string of such successes. It is hard to think of a better way to get a true picture of your manager’s skills.
Throw away your benchmark
Looking good so far? It gets better.
Performance is just the tip of the iceberg these days. More and more, investors are clamouring for detailed information on risk and attribution, which conventionally rely heavily on stock or security-level benchmark data.
The problem here is that benchmarks are a nightmare to manage. Apart from their costs, they take a disproportionate amount of time and effort to maintain, particularly for smaller fund managers.
There are various reasons for this, including commercial pressures, data volumes and legacy programming issues. But the net outcome is that benchmarks remain a pain to use at anything other than the aggregate level. Nor does the situation look as if it will change at any time soon.
This is where the random approach really comes into its own. As long as we can agree on the set of stocks from which portfolios can be built, and the returns of these stocks, we can start measuring investment skill in a clear, logically consistent way without using a benchmark.
Think about it. No data recreation issues, stock splits, updates and corrections. No security classification problems. No reconciliation issues. (That rustling sound is your back office and IT managers pricking up their ears.) In short, using random portfolios for attribution replaces expensive, labour-intensive data requirements with cheap computer power.
Running a hedge fund? You can use exactly the same approach, even though your fund may be highly leveraged and have no sensible benchmark anyway.
What’s the downside? Apart from keeping a record of possible investment stocks and their returns, you also need to take a long, hard look at what sorts of positions are realistic, and how fast you can get in (or out) of a position in that stock. For instance, buying $1m of a stock with a $1bn market cap can probably be done over the phone, but buying the same amount in a company with a $5m market cap is a major legal undertaking.
Fortunately, none of this is particularly hard to include in a computer model. And remember, you do not have to deal with a benchmark.
Attribution with random portfolios is an idea whose time has definitely come. Expect to see this capability on offer from the more far-sighted performance analytics vendors very soon, maybe under a different name — Monte Carlo risk simulation, stochastic attribution or something similar.
Oh, and did I say you will not need a benchmark?
Heresy No 2: The Brinson approach to attribution is outdated and misleading
Brinson’s 1985 paper was a breakthrough in the way that investors look at the returns they make in equity portfolios. Rather than breaking down returns by sector, the insight of the paper was to highlight a fundamental split in the way that stocks are managed. The approach allowed a way to discriminate between returns made by which stocks were chosen (stock selection) as against how many stocks were bought (asset allocation).
Due to their global nature, asset allocation decisions are usually made by committees of fund managers. Whether they use quant-based models, read economic forecasts or consult the entrails of a chicken, the net result is the same: a set of decisions on which sectors (if any) in their portfolio should be overweight or underweight, and by how much.
The remainder of the investment decisions are now handed down to the stock analysts, who are each given their earmarked proportion of funds and told to invest in their specialist sectors. The return made by this non-asset-allocation return is pigeonholed under stock selection return.
This approach has percolated through the industry, and it would now be difficult to find a manager who doesn’t report returns broken down in this way.
Which is odd, because there are some major problems with the Brinson model.
The first major problem is that the user has to perform a fair amount of preparatory work before starting attribution:
· setting up buckets for the stocks, so that each stock falls into exactly one bucket;
· finding data on a benchmark, on which a similar process is performed.
To measure the return made by asset allocation decisions, the Brinson model looks at the effects of this investment skew. If you were overweight airline stocks and that sector performed well relative to the benchmark, you added value.
Less obviously, if you were underweight and that sector underperformed the benchmark, you also added value, because you reduced your exposure to a (relatively) falling market. Adding up all the excess value added by this over-or under-weighting measures the asset allocation return.
Dog, lemon or hot stock?
Let’s think about what is going on here. The Brinson approach represents a fundamental split in the types of investment decision. If you go to see a broker to invest some funds, you will probably have a range of stocks
suggested, and then be asked how you want to apportion your funds among
those stocks.
This is the second major problem with the Brinson approach. It does not really separate the returns made by the two types of investment decision.
Here is why. Most investors are familiar with share tipping newsletters, recommending ‘hot picks’. Everyone does it, even the major banks, although they might phrase the recommendation rather differently.
The advice given is of one particular type: stock selection, pure and simple. Share tipping is not asset allocation. No share tipper is ever going to tell you how much of a stock to buy. That is an entirely different type of decision, and depends on how much you have to invest, where your current portfolio exposures lie, and what your view is of the market.
A naïve investor who uses such a newsletter religiously would expect an equity attribution scheme to measure the value added by following these recommendations. They have outsourced the stock selection decisions, so their only discretionary input is asset allocation — how much to put into each stock.
The curious thing is that measuring the return made in this way is precisely what the Brinson scheme fails to do.
For example, suppose a manager with $100m under management decides to go overweight the benchmark in technology stocks by five per cent. The benchmark has 10 per cent of its investment in these stocks, so the manager has to place (10 per cent + five per cent) 100 million = $15m in this asset class.
The IT specialist now has a mandate to spend $15m on technology stocks. So he does, by purchasing $10m of Microsoft stock and $5m of Dell. All the return made from this choice is attributed to stock selection. But this is wrong. For instance, he could equally well spend $12m on Dell and $3m on Microsoft, leading to a quite different result.
While the stock selection is the same, the return will not be, because he has made different asset allocation decisions within his sector. Yet the Brinson attribution scheme puts all this return — which is clearly due to a mixture of asset allocation and stock selection decisions — into the stock selection pot.
An additional problem is that, despite the qualitatively different nature of the stock selection and the asset allocation decisions, the Brinson framework does not clearly discriminate between the two due to the appearance of an interaction term. To quote Carl Bacon (2004): “A flaw of both Brinson models is the inclusion of the interaction or other term. Interaction is not part of the investment process; you are unlikely to identify in any asset management firm individuals responsible for adding value through interaction.” In other words, the returns from both decision types can overlap.
I suggest a different approach. Skipping the technical details, the trick is to build an intermediate portfolio holding the same stocks as in the portfolio, but with the same weights that they have in the benchmark. The difference in return between this intermediate portfolio and the benchmark is assigned to stock selection return, and the difference in return between the intermediate portfolio and the portfolio is asset allocation return.
To illustrate, suppose you have a portfolio with exactly the same stocks as the benchmark, but with the holdings in different proportions. In this case the intermediate portfolio is exactly the same as the benchmark, so the return due to stock selection is zero — just as expected.
Secondly, suppose that the user only takes stock selection decisions without asset allocation — in other words, all return is due to stock selection, and none to asset allocation. This is not actually possible in this new scheme, because since the inclusion or omission of any stock has to affect asset allocation — which is also the case in reality.
However, it is possible to fake this situation in the Brinson framework if you hold sector allocations at the level of the benchmark, in which case the asset allocation return will be zero, irrespective of what stocks are actually held in the sector. But this can be both unrealistic and misleading.
The net result of this slight variation on the Brinson model leads to no interaction terms, no partition of stocks, and a clearer picture of which decisions affect return.
So why is the Brinson model so entrenched? I suspect it boils down to the same issues as Heresy No 1: benchmark data. Brinson-style attribution can be run even if you only have sector weights and returns. You do not necessarily have to know the security-level make-up and returns of your benchmark — a handy, but probably misleading, shortcut.
This restricted data approach may have been acceptable practice 20 years ago, but is it still the case today, with massive computer power and all the resources of the internet on everyone’s desk? Unfortunately, the answer appears to be ‘Yes’.
.jpg)
