So I've been trading for a while now, but after my internship, I learned about the different types of option pricing models like JumpDiffusion and CubicSpline. So I have begun trading both relative value and vol arb strategies. The forecasts have been spot on on a time frame t<5 min, and when I backtested it (made sure not to overfit and to avoid look-back bias) the actual - predicted value is usually within a 2 basis point range. So the model values are indeed statistically significant.

It's been 2 months of daily trades so far, but I have seen people spout strategies that just get lucky. This does not feel like one of them, but I am sure that some of you have already tried this, so I'm curious about what you have to say.

Thank you

  • 1
    It may depend on your strategy - maybe your strategy wins 99% of the time but that 1 out of 100 loss breaks you?
    – D Stanley
    Apr 8, 2022 at 0:36
  • Well, I am running defined-risk strategies, and most of the time my delta is net-zero or close to it. So my main concern isn't the drawdown volatility. The strategies are basically just capturing the spread of what the model estimates and what actual prices are, so it's a tight-margin trade but is very replicable (options-quant platform so I don't have to pay for all the data and models xD).
    – CharmVanna
    Apr 8, 2022 at 0:50
  • It's a reasonably safe assumption that every strategy that seems to work is lucky. Perhaps your strategy only works during certain market conditions, such as when it's going sideways, and you lose money if it goes up or down very much.
    – user253751
    Apr 8, 2022 at 10:59

1 Answer 1


Unfortunately, the question cannot be answered by anybody. Model validation is more than backtesting, it is also "does the model make sense?" To answer that, people would have to know specifics and have a lot of data.

I worked with a mathematician that made a small fortune using a very systematic and logical trading system. He made his trades on his breaks and at lunch. He tried to have management invest in it. He really was making quite a bit of money.

So a large scale validation was performed.

It was determined that the gains were unique to two things. First, his start date mattered. When the strategy was run beginning on other dates, it lost money and did so consistently. In fact, the strategy did poorly overall. Second, his breaks seem to roughly coincide with persistently illiquid times in the market.

However, while the market was illiquid in terms of covering smaller trades, large block orders would have shifted prices enough to liquify the market and then some. When volume estimates were added, so it was performed on larger amounts of money, the strategy lost money.

When the same strategy was performed at other times of the day, it lost money.

His model made sense, it made money, but on careful validation, it was discovered that it didn't scale and it was sensitive to the starting and likely ending points. Understand that he was a formally trained mathematician and he understood how to do things, but he was personally involved.

It is difficult to validate a model when you are personally involved. There are many reasons something can work other than the reasons you have told yourself. We would need a lot of information.

There is something that you can do, however. You can stress test it.

Let us assume that your model isn't just invalid, it is orthogonal to nature. You are not just wrong, you are perfectly wrong. You had just gotten lucky up to this point.

What would happen?

Secondarily, if you toss out all the statistical mumbo jumbo, does it make objective sense? To understand what I mean by statistical mumbo jumbo, consider a model that takes advantage of mean reversion.

What mean reversion would imply is that there is some basin of attraction. Supply and demand theory would support that idea. However, if it is true, it is not true because of mean reversion, it is true because supply and demand theory make sense.

If there is no mean reversion, that does not invalidate supply and demand theory, but it does imply a complex version of supply and demand theory. It would be complex enough that a simple statistical model would not work. Supply and demand are still valid, but the statistical model becomes magical words by a witch doctor.

Not the answer you're looking for? Browse other questions tagged .