Incorporating real-world parameters into simulated(paper) trading

Question

A bit expanding the question asked here, I want to try and test a strategy using a simulated(paper) trading. As a first step, I have tried to find any brokers who provide a convenient simulated trading platform. However, after some thought, I have decided to use simple manual "excel" approach. My strategy is based on the below assumptions:

1. The decision whether to enter/leave a position will be made by EOD, when the closing price for the same day is known.
2. The buy/sell action will be performed the day later, once trading starts.

So basically I'm going to wait for next trading day, see the actual prices on that day and write down the price I would have paid/received. Needless to say, that my main concern here is to create a simulation as much close to the real thing as possible. So I thought of what factors must be taken into account to make this simulation process a bit more life-like. I came up with the below list:

1. Brokerage fees. Can be easily deducted after each performed transaction.
2. Tax. Can be easily calculated.
3. Availability of buyers/sellers???

The last(third) issue seem problematic since I can't figure out how to take that into an account. I'm perfectly fine to just use the close price for decision making stage, but when I execute the order the day after, I'd want to be sure that I actually can buy/sell the desired papers and I don't want to assume the day's open price to be equal to the last day's close price. So, according to my understanding, I will need to have an access to the order book to see the actual bid/ask requests, including the number of shares(or anything else). It seems to me that utilizing the order book records in my simulations, will make the results more real, thus it will incorporate all unsuccessful/partially executed orders into the calculations.

Do you agree with that? Can I actually access this info or maybe it can be calculated indirectly in some way? What other factors could I incorporate into my calculations to make it more robust?

Answer 1

I think you're on the wrong track. Getting more and more samples from the real world does not make your backtest more accurate, it just confirms that your strategy can withstand one particular sample path of a stochastic process.

The reason why you find it simple to incorporate fees, commissions, taxes, etc. is because they're a static and constant process -- well they might change over time but most definitely uncorrelated to the markets.

Modelling overnight returns or the top levels of the order book the next day is serious work. First you have to select a suitable model (that's mostly theoretical work but experience can help a lot). Then, in order to do it data-driven, you'd have to plough through thousands of days of sample data on a set of thousands of instruments to get a "feeling" (aka significant model parameters).

Apropos data mining, I think Excel might be the wrong tool for the job. Level-2 data (even just the first 10 levels) is a massive blob. For example, the NYSE OpenBook historical data weighs in at a massive 15 TB compressed (uncompressed 74 TB) for the last 10 years, and costs USD 200k.

Anyway, as for other factors to take into account:

• Slippage! Even when the book's first 3 levels easily support your, say, 20k order, someone else might be there just before you and you'd fall through to levels 3, 4 and 5. (can be modelled with a Poisson process).
• Position limits! You didn't say what instruments you're trading, but particularly in the commodity futures section there are absolute position limits (across all maturities) you must obey.
• Margin calls! They are broker-initiated sudden position limits, for instance as a result of changing margin requirements (can be modelled with a Poisson process)
• Corporate Actions. In case of bond or stock equity instruments you should know that the next day open price can be ex-rights, ex-dividend, ex-interest, ex-spin-off, etc. Such actions typically affect the quotes but don't affect your investment (as you're compensated with cash or more instruments).
• Special corporate actions: squeeze-out, liquidation, bankruptcy, worthlessness declaration; they are so called tail events and typically modelled with a heavy-tail distribution. In your real-world backtest you will never find these because you suffer from the so called survivorship bias

So how to account for all this in a backtest? Personally, I would put in some penalty terms (as % on a return basis) for every factor you want to consider, don't hardcode them. You can then run a stress test by exploring these parameters (i.e. assign some values in the range of 0 to whatever fits). Explore them individually (only set one penalty term at a time) to get a feeling how the strategy might react to stress from that factor. Then you can run the backtest with typical (or observed) combinations of penalty factors and slowly stress them altogether.

Edit
Just to avoid confusion about terminology. A backtest in the strict sense (had I implemented this strategy X years ago, what would have happened?) won't benefit from any modelling simply because the real-world "does the sampling" for us.

However, to evaluate a strategy's robustness you should account for the additional factors and run some stress tests. If the strategy performs well in the real-world or no-stress scenario but produces losses once a tiny slippage occurs every now and again, you could conclude that the strategy is very fragile. The key is to explore the maximum stress the strategy can handle (by whatever measure); if a lot you can call the strategy robust.

The latter is what I personally call a backtest; the first procedure would go by the name "extension towards the past" or so.

Some lightweight literature:

• The Slippage paradox
• Au.Tra.Sy blog entry about slippage there's quite a few interesting views in that blog, also for example about data pitfalls
• Wikipedia's entry about Poisson processes

Answer 2

You said the decision will be made by EOD. If you've made the decision prior to the market close, I'd execute on the closing price. If you are trading stocks with any decent volume, I'd not worry about the liquidity.

If your strategy's profits are so small that your gains are significantly impacted by say, the bid/ask spread (a penny or less for liquid stocks) I'd rethink the approach. You'll find the difference between the market open and prior night close is far greater than the normal bid/ask.