Why Is It So Hard To Beat the Market?

Active fund managers want earn better returns than the market, while taking less risk.

I’ve talked about the shift toward index funds in Canada and the U.S., and how the prolonged poor performance of active funds is a driving force of that shift. While it seems logical that fund managers, some of the smartest, highest paid, and best equipped people on the planet, should be able to use their resources to create a favourable result for investors, the data simply does not support their ability to do so.

On average, active fund managers consistently underperform the market.

So then, what’s going on? Why is it nearly impossible to beat the market consistently?

William Sharpe’s 1991 paper The Arithmetic of Active Management, published in the Financial Analyst’s Journal, is a great starting point in explaining the challenge that active managers face. It’s pretty simple. Actively managed funds, on average, have significantly higher fees than passively managed funds.

Quick side note: Canadian mutual funds have some of the highest fund fees in the world, with an average of 2.35%. Contrast that with the fact that you could build a couch potato ETF portfolio for as low as 0.16%.

So back to William Sharpe. His 1991 paper said: Before costs, the return on the average actively managed dollar will equal the return on the average passively managed dollar. This makes perfect sense because there is one global market, and, in aggregate, all passive and active funds are investing in that same market. Sharpe went on to say that after costs, the return on the average actively managed dollar will be less than the return on the average passively managed dollar – same returns before fees, but lower returns after fees due to higher fees. If fees were not in the picture, you might expect about half of active managers to be able to beat the market. Add fees in, and that number drops to less than half at best.

Ok, so the average active manager is unlikely to beat the market in a given year due to fees. But who’s going to hire an average active manager?

The idea of manager skill has been the subject of a lot of research. In a 1997 paper published in the Journal of Finance, Mark Carhart explained that investment expenses and exposure to certain risks almost completely explain persistence in equity mutual funds returns. Based on his results, the existence of skilled or informed mutual fund managers cannot be supported. In a 2009 paper published in the journal of finance, Eugene Fama and Ken French found that, while there are some skilled managers out there, few of them produce returns high enough to cover their costs.

So then…If there is little to no evidence of manager skill, it means that the managers who have done well may have just been lucky in the past. Finding the manager who has done well in the past and will continue to do well is a gamble, not a guarantee. Are they skilled, or are you giving your money to someone who has been lucky?

An important distinction here is that when we are talking about skill, we only care about relative skill. Any mutual fund manager is likely to be intelligent, well-read, and competent, but they need to be more intelligent, well-read, and competent than the other manager on the other side of each trade. Any time a manager sells a stock that they think will do poorly, another manager is buying it because they think it will do well. Only one of them will be right. The investment management industry is full of PhDs, CFA charterholders, and MBAs. Currently, most of the world’s stocks are owned by institutions rather than individuals. Most trades are a bet between two highly skilled professionsls. The level of competition is exceptional.

As more and more skilled managers enter the active management business, luck becomes increasingly important as a determinant of success.

Supporting this idea, in a 2013 paper, Pastor, Stambaugh, and Taylor found that the increasing size of the actively managed fund industry has a negative impact on active funds’ performance. More skilled competition is making it increasingly challenging to generate higher returns.

Let’s say that you do find a skilled manager. They have had a few good years, but you can tell that they are skilled rather than lucky. You invest your money in their fund. So do a lot of other people, because the performance is good. Fairly quickly, the fund will grow to a point where the manager’s skill becomes less useful, or even useless, because there is so much money that needs to be invested. It’s one thing for a skilled manager to find a few good stocks to invest a few million dollars in, but if a fund has a few billion dollars to invest, the manager may have trouble being different from the market. When that happens, the investors in that fund end up with a really expensive index fund, known as a closet index fund.

On average, the math just doesn’t work out for the average active manager, and finding a manager that will be consistently above average is extremely difficult in today’s increasingly competitive environment. Even if a manager is successful in generating higher returns, their success will be their downfall.

There is another, less obvious reason that active managers have so much trouble beating the market. We know empirically that a small number of stocks in an index tend to drive the performance of that index. So trying to select stocks in an index dramatically increases the probability of underperforming. When trying to pick stocks in an index in an effort to beat the index, the likelihood of underperforming is much greater than the likelihood of outperforming.

This research was published in a 2017 paper by Heaton, Polson, and Witte titled Why Indexing Works, and it is yet another blow to active management.

If we look at data on market returns, the basis of this research is clear. Most of the market’s returns come from a small number of stocks.

Let’s look at an example using data from global stock markets between 1994 and 2016.

Over that time period, global stocks returned an average of 7.3% per year. Not bad at all. If we remove the top 10% of stocks in that global market portfolio, the average annual return drops to 2.9%. Excluding the top 25% results in the average annual return dropping to a much less exciting -5.2%. Of course it would be great if active managers could identify only the top performing stocks, but we have seen the data around their ability to do so – it doesn’t look good.

We know that, in most cases, active managers’ performance can be attributed to luck rather than skill. A lucky manager has been fortunate to randomly select stocks that have done well. In their paper, Heaton, Polson, and Witte lay out a very simple example explaining the challenge that active managers face based on what we know about market returns.

If we have an index consisting of five stocks, and assume that four of them will return 10% and one will return 50% over a given time period, and we suppose that active managers will create portfolios using an equally weighted subset of either one or two of those stocks, there will be a set of fifteen possible actively managed portfolios. Ten of the fifteen portfolios will earn a 10% return due to omitting the 50% stock, and five of the fifteen portfolios will earn either a 30% return or a 50% return due to holding the 50% stock, depending on whether it is a one or two stock portfolio. In this example, the average return of the stocks in the index, and all active fund managers, will be 18% (before fees).

Just like we would expect, all active managers together get the return of the market. But when we look at each portfolio individually, two-thirds of the actively managed portfolios will underperform the index due to their not holding the 50% returning stock, which is always included in the index.

Closet indexing is the result of active managers owning the market like an index fund, but charging active management level fees.

Some active managers will pitch themselves as having high conviction, or high active share, meaning that they are very different from the index. The implications of this research are that even if you are able to find an active manager that is truly active and has low fees, there is a relatively low probability that they will be able to deliver market beating performance. With this high probability of underperformance, finding a skilled manager, which we already know to be beyond challenging, becomes increasingly important.

Fees typically take the blame for the systematic underperformance of active managers, but this research demonstrates another big hurdle that needs to be overcome to beat the market. If active managers miss out on the relatively small proportion of top performing stocks, they are at significant risk of trailing the market.