0

I noticed that some traders use backtesting to evaluate their trading strategies. What I don't understand is: how do they separate the predictive power of their trading strategy from the general market trend when interpreting the backtest results?

For example, long-only trading strategies will likely show good performance during rising markets, even if some of those strategies have no predictive power at all. Similarly for short-only trading strategies during declining markets. There must somehow be a way to remove the effect of market trends if the predictive power of a strategy is to be measured. I imagine that the real life situation is more complex than what I have just illustrated, because some strategies involve both long and short positions (at different times).

Improve this question
5
  • 1
    Using a long enough amount of time should alleviate that - test 50 years, not only 5.
    – Aganju
    Commented May 22, 2021 at 15:51
  • Does this answer your question? How can I determine if my stock picking performance was due to luck or due to skill?
    – nanoman
    Commented May 22, 2021 at 17:17
  • @nanoman No, this question is considerably more focused on the process of backtesting. Considering the nature of Bob Baerker's answer to this question, I would say that this question is not a duplicate of the one you are suggesting.
    – Flux
    Commented May 22, 2021 at 17:24
  • @nanoman By the way, I was the OP of that question too. I have carefully read all the answers there. The answers there are not specific to trading and backtesting.
    – Flux
    Commented May 22, 2021 at 17:26
  • I think my answer to that question applies here. True, that question assumed a real trading track record. In the case of backtested performance, the usual caveats apply (evaluation should be done on out-of-sample data that were not used to train the strategy). I am suggesting that you could use the backtested results in this question in place of the actual results assumed in the other question. As long as the backtest is reliable, the analysis is the same.
    – nanoman
    Commented May 22, 2021 at 20:40

3 Answers 3

Reset to default
1

I'm not sure that I agree that a strategy has 'predictive power'. That's probably a separate debate which I'm going to sidestep.

Long strategies perform well in up markets and short strategies perform well in down markets. So either you have to have something that suggests what type of market you're in (easier said than done), or you have to have a mix of long and short strategies, or you have to have something that is agnostic to direction (trading volatility).

FWIW, 25+ years ago I went through a technical analysis phase with a group of clever people. I'll spare you the details/conclusions but one thing we all had in common we had was that we all used a software program that could backtest any strategy that we could formularize.

The backtesting provided 3 sets of stats: long only, short only, and both. Now while those results didn't quantify the amount that the general market trend contributed, it surely indicated its affect.

Improve this answer
9
  • "we all used a software program ..." — MetaStock? Did you have a successful technical analysis phase?
    – Flux
    Commented May 22, 2021 at 16:20
  • LOL. BUSTED! I owned the first release of Metastock and a good number of the subsequent upgrades. Another good program was Technifilter which enabled you to run your trading strategy against a database, providing a list of stocks each day that met your screening criteria. In terms of P&L, I did not have a successful technical analysis phase. Nor was it unsuccessful. However, it did provide a lot of market awareness and was the foundation of my evolution into other strategies.
    – Bob Baerker
    Commented May 22, 2021 at 16:35
  • Are these backtesting software useful for those who are not doing technical analysis?
    – Flux
    Commented May 22, 2021 at 16:54
  • "In terms of P&L, I did not have a successful technical analysis phase ..." — Do you happen to know the reason(s) for your lack of success in technical analysis? What made you abandon technical analysis?
    – Flux
    Commented May 22, 2021 at 16:59
  • I can't imagine how backtesting software could be used for anything other than backtesting TA. I still use Metastock but not for its intended purposes. I've subscribed to Thomson Reuters EOD data for 20+ years and it is stored in Metastock files. I use a macro driven routine that exports the data and updates 75+ spreadsheets in a few minutes. I'd bet that there are more expedient ways to do this nowadays but I have no need to fix what ain't broke. It wouldn't be worth the time involved.
    – Bob Baerker
    Commented May 22, 2021 at 17:26
1

Yours is a question that is rampant in the industry, especially in quantitative finance. Let me put it this way: no company will ever offer a product with poor back tested results.

Any predicative power of a backtest, in my opinion, must be tested in different market environments so as to separate the power of the strategy from the prevailing market trends at the time. For example, can you get a picture of how the strategy did in periods of both rising and falling rates? How about periods where certain factors (growth/value/dividend, size, etc.) or certain sectors or subindustries (finance/consumer discretionary/technology, etc.) did well?

Improve this answer
1

There must somehow be a way to remove the effect of market trends if the predictive power of a strategy is to be measured.

This is the meaning of Alpha. Your returns minus the market return equal Alpha.

As Larry Page wrote in announcing Google's new holding company, Alphabet:

For Sergey and me this is a very exciting new chapter in the life of Google—the birth of Alphabet. ... We also like that it means alpha‑bet (Alpha is investment return above benchmark), which we strive for!

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .