Is backtesting a reliable concept?

Question

I've been using technical analysis approach for several years right now and currently explore the algorithmic/automated trading techniques.

It seems that one of the most important aspects of creating strategies for automated trading - is the backtesting. However I'm not sure how reliable backtesting really is? I don't mean to raise the "Whether backtesting is a good way to predict future price moves?" issue, but just the technical side of it. What I am trying to say is that when a real time trading takes place, there are real time factors as well.. For example:

• Buyers/Sellers presence - in real world, there's always a possibility that your sell\buy order will be executed only partially (for example you were able to sell only half of the shares for a certain price).
• Response times - I'm not talking about HFT(High Frequency Trading), but while backtesting, the sell/buy order will always be executed, and executed immediately. However in real life it might not be the case.

So my question is basically as follows:

1. I guess there are even more practical considerations I do not take into account due to my lack in experience and it could be great if someone named those..
2. Is there any way (or need) to incorporate these factors into the backtesting?

Answer 1

Backtesting is a good way to check the performance of a trading strategy or idea over various market conditions. By paper trading in current market conditions and backtesting in various past market condition, you should be able to design a robust trading system that is able to perform well under most market conditions.

Of course as you said backtesting (as well as paper trading) can be quite different from trading in the real world. There are things like the market gapping, different order types, liquidity in the market, and probably the biggest of all - your psychology - which can affect your trading in the real world and distort your results from the testing phase. That is why it is so important to have a plan, be disciplined and follow your plan all the time.

The best thing you can be is consistent with your decisions and actions, a bit like having an automated system. But even then you still won't get the exact results as the backtesting, but you should get close. And I think that is what backtesting is. It is not a method to get a perfect trading system, because in the real world there isn't one. Backtesting is a tool you can use to check and test that your trading system is in fact robust and should work well in various market conditions.

Answer 2

Backtesting via software always has a variety of limitations and assumptions that the software makes, whether it is in placing the trades, or how realistically you would get filled on the trades, or how the software calculates profits and losses.

Another limitation is that in a real world market, you real trade will affect the outcome of that trading day. Even if you are a small market participant, other market participants especially some HFT algorithms react to orders and odd lots based on their size. So if your entry point is based on some technical analysis concoction in the past, in the real world as soon as your order placed on the tape, there will be an immediate distortion of what you would expect. This really only matters in short time frames though, like something a daytrader or daytrading algorithm would have to worry about.

Backtesting on the other hand is useful in helping illuminate a bad strategy very quickly as there are many people that swear by a pretty random set of disciplines to help them make a trade.

Answer 3

Add 5 basis points to account for slippage on every order (one for buy and one for sell). This changes the sharpe ratio significantly on strategies with many orders.