24

From 2000 to 2020, the S&P 500's compounded annual growth rate (CAGR) is about 6-7% (assuming dividend reinvestment). With a CAGR of 6.5%, every $1 invested in 2000 grows to about $3.50 by 2020.

  • Scenario A: I have outperformed the S&P 500 nearly every year for the last 20 years. My CAGR is about 12%. I turned every $1 in 2000 into about $10 by 2020.

  • Scenario B: I have occasionally outperformed and occasionally under-performed the S&P 500 for the last 20 years. My CAGR is about 5%. I turned every $1 in 2000 into about $2.70 by 2020.

Questions:

  • In both of these scenarios, what method can I use to determine whether the performance was due to good luck, bad luck, good skill, or bad skill? What established methods do financial professionals use to separate luck from skill?

    I understand that the calculation will require much more information than I have provided above. I assume that the established methods will take into account the portfolio concentration, year-to-year variance of performance, etc.

  • Are these methods easily adopted by individual investors?

Relevance to personal finance

Suppose I have been managing my own finances and picking stocks for the last 20 years. With the data of my own financial performance available, it is time to take a cold, hard, and honest look at whether or not the time and effort spent on stock picking over the last 20 years was actually worth it. Going forwards, I will need to know whether or not my past performance was mostly due to luck or due to skill. If due to bad skill (or bad skill with good luck), I could buy index funds, quit picking stocks, and use the free time for leisure. If due to good skill (or good skill with bad luck), I shall continue picking stocks.

(Note: My finance background is relatively weak. I remember hearing about "alpha" and "sigma" many years ago. They seem to relate to my question, but I am not familiar with those concepts to know for sure)

Improve this question
9
  • 11
    You may want to reach over to the CrossValidated stack, as a big part of statistics is quantifying whether something occurred due to random chance (the "null hypothesis"), or if something meaningful is happening. Beating the market could probably be considered similar to the Lady Tasting Tea example, though I'm not exactly sure how it would work (the stock market is a fair bit more complicated, obviously)
    – PGnome
    Commented Jun 11, 2020 at 19:48
  • 1
    @NuclearWang OP says he puts time into picking the stocks, so I presume he does have some method to select his stocks, the question is then whether there's a statistical test to check if his method works or not. I think it's a very legitimate question to ask, even for experts, whether their expertise truly predicts reality, or if they were just lucky.
    – csiz
    Commented Jun 11, 2020 at 23:54
  • 1
    There are many stocks you could have been lucky on, any of Apple, Google, Tesla, Netflix would have brought you pretty good returns... But that's besides the point, you still want some answer to whether your method works or it's chance. There's no lower limit for data, you'll just get a wider error range. Up to the point where the error is so large the answer is useless, but it's still a useful endeavour to go through some statistical analysis.
    – csiz
    Commented Jun 12, 2020 at 0:31
  • 4
    @alephzero And yet at the same time, 20 years of data is WAY too much to collect before you decide whether you are good at picking stocks or not. If you spend more than a year or three deciding this I would argue that you might be missing out on a LOT of performance going with something like an index fund.
    – user12515
    Commented Jun 12, 2020 at 1:56
  • 2
    @PeteB. If that were true, then there would not be (a minority) of fund managers that do consistently achieve positive alpha. Berkshire Hathaway is an obvious and famous counter example to your assertion. It would be more accurate to say that it is very difficult to consistently make predictions, but to say that it is impossible is very plainly wrong.
    – JBentley
    Commented Jun 12, 2020 at 14:05

10 Answers 10

Reset to default
32

I've had that exact same question myself, though usually in the opposite direction (am I unlucky or an idiot :) )

One thing I've looked at is large components of return. If you have one or two investments that dominate your returns, that's more an indication of luck versus skill. If you remove the best performing stock or fund, how different is your return profile?

As an example, I've seen stock sites that say they consistently beat the market, but their highlighted picks consist of FANG stocks (Facebook, Amazon, Netflix and Google), which have increased by many multiples since 2010. My question has been - how have you done other than those picks? Not that picking those stocks early is completely luck - certainly one could look at their businesses and see potential, but do they get excess returns on other stocks as consistently?

Ask yourself the same question - do you have one or two stocks or indices that dominate your returns? What separates them from the other stocks/indices that would indicate skill versus luck?

If you remove the FANG stocks from the S&P 500, the average returns since 2010 drop from 10.2% to 8.9%. How much does removing the largest contributors from your portfolio change your returns?

At the end of the day it may not matter - sometimes being lucky is better than being good - but it can be hard to distinguish the two.

Improve this answer
3
  • 1
    "At the end of the day it may not matter - sometimes being lucky is better than being good" - I disagree. Luck can run out any moment, while skill (hopefully) does not. Skill is a predictor for future performance while luck is not. This is a very important distinction when making the decision whether to keep playing the stock market game or quit while you are ahead.
    – Philipp
    Commented Jun 13, 2020 at 12:12
  • 1
    I beg to differ. My biggest return is still by a single stock that I had managed to determine was really underpriced in 2008. I'm somewhere over 50% weight in it now, and while it doesn't return like it did in 2009 it still makes good steady gains.
    – Joshua
    Commented Jun 13, 2020 at 14:44
  • VC firms will also disagree. Their entire MO is to invest in a number of early-stage companies, most of which will go bankrupt, but the successes among which will more than pay off for the failed attempts.
    – chrylis -cautiouslyoptimistic-
    Commented Jun 14, 2020 at 5:20
13

If you bought S&P 500 with 3x Leverage in 2000, does a CAGR of ~20% outperform a S&P 500 with 1x Leverage (i.e. no leverage)? No. Outperformance is on a risk-adjusted basis.

For an individual, the easiest way to determine outperformance is by finding the Mean weekly return (in %) and its Standard Deviation (in %). Then find the leverage of S&P 500 that achieves exactly the same Standard Deviation. For example, if your portfolio standard deviation is 8% and S&P 500 (with no leverage) is 6%, then the equivalent leverage is 1.33x. Then compare the portfolio Mean return (in %) against 1.33x the Mean return of S&P 500.

To determine whether it is by chance that there is outperformance, it requires alpha and beta as you suggested. Specifically, it is the p-value / confidence interval / statistical significance of the estimated alpha, which most statistics software or Python (Scipy Stack) can calculate it for you. Having a positive estimated alpha does not guarantee that it was not by chance.

Without your data, there is not much that we can discuss.

Improve this answer
10
  • 1
    +1 for bringing in the Standard Deviation and for comparing your investments with leveraged ones.
    – Weirdo
    Commented Jun 12, 2020 at 11:44
  • Any p-value you calculate is meaningless, because the sample is pre-selected as a high outlier and you cannot know the size of the population of stock pickers with which to compare. A traditional 5% cutoff only means that if a 100 people are picking stocks, then 5 of them will be in that cutoff. If you pick people from the population on the basis that they are high enough in the population to ask a question on SE about it, then you've already biased the sample.
    – Jack Aidley
    Commented Jun 12, 2020 at 12:32
  • 1
    @JackAidley I don't see anything suggesting that the OP knows they are a top performer ("pre-selected as a high outlier"). They provide two scenarios where they outperform and underperform the market, so it's not like we have the self-selected case of a pool of lottery winners telling us about their number picking strategy. And what multiple hypotheses? You'd need to correct if you're testing multiple individuals' returns, but we only have one sample.
    – Nuclear Hoagie
    Commented Jun 12, 2020 at 13:32
  • 1
    @JackAidley We are not finding out whether OP is luckier than other people. We are finding out whether OP has consitently "beat" the strategy of buy and holding S&P 500. If we cherry pick 10 people (with 10 portfolio) who claim to have beaten the S&P 500 and they all have statistically significant alpha in their respective Portfolio Return = alpha + beta x (S&P Return), so be it, they have all beaten the market consistently.
    – base64
    Commented Jun 12, 2020 at 13:53
  • 1
    I think @JackAidley is referring to family-wise error rates. The classic fault in these is, of course, how to define the family. You could say the family is really just Flux, since we're not comparing to anyone else, and no adjustment needed. Or we say that we're only even talking about Flux's returns because they aren't equal to the market, so how many other people COULD have been in this place instead? If its say 100, then we need a higher p-value with Flux's return to be actually convinced.
    – Cain
    Commented Jun 12, 2020 at 15:31
7

The simple answer is that it's luck, because no-one seems to be able to pick stocks and always outperform the market with it (considering their leverage). That's not how any fortunes are made in finance that I know of.

For instance take Warren Buffett. He's outperformed the market a lot, I believe, and for decades on end. And yet, he likes a certain kind of company to invest in. It's probable that he's just lived in a period of time when--in retrospect--that kind of company was the right kind of company to be in. Put Buffett in previous or future decades, where other kinds of companies were the fastest growing, and would he recognize that and invest in those companies instead? Or would he still be interested in what he's interested in today, and only be a modest success or even failure?

Improve this answer
12
  • 5
    I agree 100% with SwissFrank. The "Buffett Story" is a totally non-interesting selection bias example. The guy bet hugely on "insurance" during just the right three decades. If that bet had been "wrong", the name Buffett would be as unknown as 20,000 other guessers.
    – Fattie
    Commented Jun 12, 2020 at 13:04
  • 2
    If it were simple luck then how do stock analysts and market fund managers ever keep their jobs? If no one has any skill to this stock market thing, then I find it hard to believe anyone could actually make a 30 or 40 year long career of it. It doesn't add up. Perhaps if Warren Buffett were in a different decade he would have bought other companies because he does actually know what he's doing. Then you'd probably say the same thing - what if it was in a different decade? This answer doesn't offer any substantive proof of either luck or skill.
    – Tracy Cramer
    Commented Jun 12, 2020 at 21:11
  • 6
    The ones who don't get lucky don't keep their jobs because they go out of business. Survivor bias is a real thing, and explains much of any issues around "if it's just luck, how can all these people be so successful?" - because you never hear about the unlucky ones.
    – Nij
    Commented Jun 12, 2020 at 21:24
  • 2
    Those of you disparaging Buffet's investing performance - please reassure us that you've actually studied his biographies enough to actually know what he did when, and what his rationale actually was - as opposed to your own guesses and suppositions?
    – Spike0xff
    Commented Jun 13, 2020 at 3:28
  • 2
    Another basic thing is that even if investing were as random as a coin flip, with enough people you'll have some that nearly always flip heads. Get 1024 people together and you'll have someone who got 10 heads in a row, and ten or so who got 9/10. And when the talk is of an era, such as "the era that value investing really paid off in", then Buffett's career has really been more like ONE coin flip. In order to say "Buffett is clearly a very skilled investor" you'd have to prove that in eras where value investing is a bad idea, he wouldn't have been a value investor. What's your proof?
    – Swiss Frank
    Commented Jun 14, 2020 at 6:05
6

From a qualitative point of view, it helps to keep notes on why you buy a certain stock/bond (or derivative). It keeps you honest as you'll see when you got the call right and whether it was for the right reason.

For a more quantitative assessment, you want to look at what's driving your gains/losses: is it really alpha or is it beta (correlated with the overall market, for example the S&P500)?

The rising tide lifts all boats, as the moniker goes. Conversely, when it goes out, you'll see who was swimming naked, to paraphrase Buffett.

Also, it could be due to good/bad luck with timing the market.

Investing is about making informed guesses. Skill vs luck is a continuum and having a robust process you can stick to is as important, if not more important, than the outcome.

To answer your specific questions: professionals either look at absolute performance or relative performance compared to a benchmark and/or risk (measured by standard deviation of returns).

To compute the return you can look at time-weighted returns, which accounts for the addition/withdrawal of funds, or money-weighted returns.

Once you know your rate of return, you can compute the Sharpe ratio or risk-adjusted excess return compared to the risk-free rate or other benchmark.

Improve this answer
4

Here is the question you want to ask: Is the mean return of your portfolio (excluding fees) relative to the index significantly different from zero?

First, determine a time-weighted return series P(t) for your portfolio, such that P(t+1)/P(t) - 1 is your return in each period excluding contributions, withdrawals, and fees. (Exclude those things because they aren't relevant to your stock-picking skill.)

Then, determine the total return series I(t) for the index, and define R(t) = P(t)/I(t).

Consider the series s(t) = R(t+1)/R(t) - 1. You want to test the null hypothesis that the underlying mean of s(t) is zero (i.e., your returns are consistent with luck in an efficient market). You can reject this (and find evidence of skill) if the empirical mean of s(t) has a magnitude more than a few times its standard error (the empirical standard deviation of s(t) divided by the square root of the number of samples).

Why is s(t) the right metric? Consider a professional investor who is holding the index by default and is trying to decide whether to reallocate any capital to track your portfolio (and they can do so with negligible fees). If they find your portfolio useful in improving their return, that's equivalent to your having skill.

If at time t, out of a notional 1 dollar of assets, they allocate a fraction f to your portfolio (leaving 1-f in the index), then their wealth at time t+1 will be W = (1-f) I(t+1)/I(t) + f P(t+1)/P(t). Using the standard log-wealth utility and expanding to first order in f, we get the utility ln(W) = ln(I(t+1)/I(t)) + f s(t). The dependence on f is only in the last term.

Thus, befitting intuition, if the mean of s(t) is positive, then expected utility is increased by taking f > 0 (how big f should be depends on higher-order terms) -- i.e., going long your portfolio is desirable. Somewhat less intuitively, if the mean of s(t) is negative, then your portfolio is still useful because expected utility is increased by taking f < 0 -- i.e., going short your portfolio is desirable.

It may be surprising that creating a portfolio with negative mean s(t) is as hard as positive mean s(t). You can't just buy high-fee funds or throw money away on trading costs, because we've excluded fees. You can't just buy out-of-the-money options that almost always expire worthless, because even if you lose everything, s(t) can't be less than -1, so its mean will be counterbalanced by the small chance of a big gain.

Improve this answer
2
  • Here is the question you want to ask: Is the mean return of your portfolio (excluding fees) relative to the index significantly different from zero? No, because given 100 million investors in the nation, a huge huge huge number will most likely have performance 3, 4, or 5 sigma unlikely, even if they were all flipping coins.
    – Swiss Frank
    Commented Jun 17, 2020 at 13:49
  • 2
    @SwissFrank If you look at a single, pre-specified investor (say yourself), then 5 sigma performance is significant. If you look at 100 million investors and find one with 5 sigma performance, then you're right, it isn't.
    – nanoman
    Commented Jun 17, 2020 at 20:02
2

Good answers above, the unfortunate truth is: you can't be certain. Given the relative efficiency of equity markets, it's more likely that you were just lucky (after risk adjustment) by standard measures.

A general rule of thumb is: the more obscure your area, the more likely your success is due to skill.

Say you mainly trade on mid-cap commodity stocks traded on the Zimbabwean Stock Exchange. This is not the most efficient of markets so you make take advantage of these inefficiencies. This could be more skill than luck: it took "skill" to identify the market and skill to exploit the inefficiencies. But on US large-cap equities the excess performance is almost certainly due to luck.

Under @DStanley's answer there is a comment from Philipp: "[...] Luck can run out any moment, while skill (hopefully) does not. [...]" Your skill might not run out, but its effectiveness might: I'm certain some traders in the 50s were successful "simply" for being very quick in their calculations and order placements; when computers came to the party, they still had the skill, but the skill did not give them an edge. So in some way if you are skilled, one could say you were lucky your skill is still relevant !

Improve this answer
1
  • 1
    This is a very apt observation. For supporting evidence, look at Buffett's record. With a portfolio size of $1M to $50M (The Buffett Partnerships), he averaged nearly 40% annualized returns for a decade and a half because he could find inefficiencies anywhere, and found a lot in penny stocks. Running Berkshire Hathaway the first decade and a half his portfolio size grew from a hundred million to a couple billion while averaging around 25% returns, because penny stocks had become too small for his portfolio. As his portfolio continued to grow every decade, his returns slowly dropped..
    – SafeFastExpressive
    Commented Jun 13, 2020 at 23:20
0

In both of these scenarios, what method can I use to determine whether the performance was due to good luck, bad luck, good skill, or bad skill? What established methods do financial professionals use to separate luck from skill?

Statistics.

Scenario A: I have outperformed the S&P 500 nearly every year for the last 20 years. My CAGR is about 12%. I turned every $1 in 2000 into about $10 by 2020.

If probability of outperforming in a year is 50%, in 20 years it is 0.00000095367 or about one in a million.

The world has 3.3 million stock market indexes: https://www.businesswire.com/news/home/20180122005183/en/Index-Industry-Association-Surveys-Index-Universe

If you have prior reasons to believe your strategy is superior, chose it because of it and found it is a winning strategy afterwards, then you probably have a market-outperforming strategy.

However, if you search through the 3.3 million stock market indexes, you are likely to find approximately three that outperform the most commonly used market benchmark index every year in 20 years, so most likely the outperformance was caused by selection bias if selecting the index suitably.

(Edit: Oops, it had the world "nearly". If you outperform market 19 times in 20 years, probability is 20 in a million; if you outperform market 18 times in 20 years, probability is 380 in a million; if you outperform market 17 times in 20 years, probability is 6840 in a million or in other words 0.7% -- so high that e.g. in medicine a drug that might have produced the observed results by sheer chance with 0.7% probability might be rejected.)

Scenario B: I have occasionally outperformed and occasionally under-performed the S&P 500 for the last 20 years. My CAGR is about 5%. I turned every $1 in 2000 into about $2.70 by 2020.

In the long run, stocks yield about 8%. Thus, you are consistently underperforming the market (however, in 2000 there was a stock market bubble and in 2020 there was a low point due to coronavirus crisis, so pick the dates suitably and 5% CAGR would be about average for the 2000-2020 period).

Are these methods easily adopted by individual investors?

Yes. Learn about statistical tests. Calculate the probability that the outcome would happen due to blind chance instead of superior strategy. If the probability is let's say less than 0.1%, then you have with a high probability a good strategy.

Improve this answer
-1

The only investors with stellar long term records use fundamental analysis (value investing) in their practice. To use FA means you can make accurate estimates of a companies value, and understand when estimates can't be made accurately.

If you don't understand how to value companies, you don't have enough skill to beat the market. If you do understand, you still need to be able to have the other soft skills necessary to beat the market (patience, resistance to outside bias, work ethic, etc).

Improve this answer
8
  • I don't think so, but say you're right. Not even insiders at a company know enough about their own company, vis a vis the competition, to know how to invest. Say you worked at UBS, knew they had a new brokerage ad blitz prepared, got a new job at archrival Credit Suisse, and saw their brokerage business ad budget was being cut back. Do you know who will outperform? Maybe UBS spends too much on ads, letting CS win. Maybe UBS buries CS. Or a brokerage price war drives commissions to 0! Or prop trading makes billions at CS. If the insider can't know, you won't know based on public info.
    – Swiss Frank
    Commented Jun 17, 2020 at 13:55
  • 2
    @SwissFrank Illegal insider trading has a pretty good track record of producing excess returns, which is why it's so tempting. There's a difference between being certain of an outcome and having a better estimate than the public does of the probability of the outcome -- the latter is enough to reliably make lots of money over time with appropriate bets. It's not true that just because insiders have uncertainty, they don't have an advantage. Likewise for legal trading by someone with unusual talent for analyzing the probability implications of public information.
    – nanoman
    Commented Jun 17, 2020 at 20:15
  • @SwissFrank Buffett's massive outperformance for the first 40 years shows how the market can be beat by using fundamental analysis, by someone with the right psychological temperament. That's clearly a rare combo, which is what I tried to allude to in my answer. It's not just that Buffett is never spooked by market crashes, he does a lot to limit psychological biases in his analysis. For example, when researching a company, he won't look at it's stock price until his value estimation is done, so if he falls in love with a business he can't use it's price as a target to justify buying it.
    – SafeFastExpressive
    Commented Jun 17, 2020 at 22:49
  • massive outperformance for the first 40 years shows how the market can be beat by using fundamental analysis Sigh, no it doesn't. The first question to ask is how many trials there are. And if someone likes only a certain type of stock, and there's a long era in which that certain type of stock is the right stock to be in, then separate years or separate positions are not separate trials. Maybe the 40 years might all be one_ trial. Second question is how leveraged he was, such as by the firms' beta. Third question is how many investors we have in our pool: what are odds one does well?
    – Swiss Frank
    Commented Jun 18, 2020 at 10:20
  • @SwissFrank Again, you've got a lot to learn about Buffett. Beating the market 39 of 40 years by luck is one in 27.5 Billion attempts. Beating the market by 15% a year for 40 years by luck is one in trillions. That 40 years included dozens of bull markets & bear markets, bubbles, inflation & deflation. He invested in massively diverse sets of companies including penny stocks, liquidations, growth companies, financial companies, insurance, transportation, energy etc, etc. Leverage? None. Pool Size? Not billions or trillions. If you can't accept Buffett as proof, you can't accept anything.
    – SafeFastExpressive
    Commented Jun 18, 2020 at 21:34
-2

Here is a calculation of luck and skill:

Determine the percentage of profitable years. Then luck is a 50% result but set luck at 0% skill and set maximum positive-skill at 50%. So luck is ((50 - 50) / 100) or 0% skill while maximum positive-skill is ((100 - 50) / 100) or 50% skill.

For example, a 50% result is 0% skill, a 53% result is 3% skill, a 99% result is 49% skill, and a 100% result is 50% skill.

Then a 47% result is -3% skill, a 1% result is -49% skill, and a 0% result is -50% skill.

Then luck, as a 50% result, plus skill equals to the underlying result. Also, a 100% result only claims 50% skill and thus 50% luck while a 0% result only suggests -50% skill and thus 50% luck. There is always a significant percentage of luck and therefor no 100% skill.

There is no 100% skill while 0% skill is 100% luck. Wow, a 50% result is 100% luck which is 0% skill !

Now suppose that a five-year investment result is positive due only to the fifth year result. An investor could claim all five years as profitable based on an average return. There are choices of accounting methods.

In fact I set investment gain against a year-beginning balance plus an average deposit/withdrawal balance as projected to year-end. That's a version of a modified-Dietz.

Now a money-market investor might wonder why they don't have 100% skill but are they setting their investment result against the inflation rate ? An investment is not really successful unless the result is greater-than-or-equal-to inflation. A comparison of investment return to inflation is not particularly required but the point is that 100% skill is not likely.

Improve this answer
2
  • 1
    There are two different 1D coordinate-system number-lines. There is a number-line for skill and a different number-line for luck. For instance on the luck number-line, 0% skill sets a value of 100% luck while 50% skill sets a value of 50% luck. The number-line for skill is developed in the post. There is a relationship between the two number-lines. For instance, a 53% result is 3% skill and 97% luck.
    – S Spring
    Commented Jun 12, 2020 at 2:00
  • This is not a situation of luck being most important but a situation of engagement in endeavor being most important. Skill is important because, for instance, there is a big difference between a 47% result and a 53% result.
    – S Spring
    Commented Jun 12, 2020 at 2:04
-6

The answer is extremely simple, "have you outperformed the market?" (Over and over.)

The definition of not-luck is very simply "outperformed the market consistently many times."

Improve this answer
8
  • 1
    -1. This answer is wrong. It is entirely possible to outperform the market by rolling dice to determine your stock picks. Your mathematical expectation with such a strategy is to equal the market return, but a large proportion of people (~50%) who follow this strategy will end up with an average return which outperforms the market, just as a large proportion (~50%) will underperform. Both outcomes would be entirely based on luck.
    – JBentley
    Commented Jun 12, 2020 at 14:14
  • 1
    Hi @JBentley , you missed the many times. If you roll lucky dice many times in a row, you're on the extreme edge of a statistical bell curve.
    – Fattie
    Commented Jun 12, 2020 at 14:38
  • 1
    Hence, your comment is wrong: It is entirely possible to outperform the market by rolling dice to determine your stock picks. It's completely possible to do that once, but it's incredibly unlikely to do that many many times in a row. This is, simply, the definition of "was it luck". If you toss heads 3 times in a row - just luck. If you toss heads 40 times in a row, it's incredibly likely that it is caused by a coin problem, not luck.
    – Fattie
    Commented Jun 12, 2020 at 14:40
  • 7
    The problem is that there are many, many people tossing many, many coins. The point of the bell curve is that if you sample enough data, you fill in the whole thing...including those bits way out on the end. If you put a large number of ball bearings through a bean machine/Galton board ( en.wikipedia.org/wiki/Bean_machine ), are the ones that end up on the far sides "skilled" or "lucky"?
    – user3067860
    Commented Jun 12, 2020 at 16:28
  • 1
    Even you yourself posted a comment on another answer claiming that Warren Buffet's success doesn't count because of selection bias. Yet if we believed your answer here, then by definition his success is "not-luck".
    – JBentley
    Commented Jun 13, 2020 at 19:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .