Your Backtests Will Haunt You

Early on in my career as a Derivatives trader, I had to bring something to the trading desk that was different to all the other traders.

Whilst they were focussed on long term positioning and news based trading, I would focus on trading market patterns and dynamics that had a statistical edge.

My engineering and tech background meant that I was perfect to start a systematic, data-driven trading business which would capture intraday short term opportunities.

As I was a new trader, I did not have a track record to rely on, so with all of my ideas I had to prove profitability with historical performance, and show how the strategies would behave during major market moves.

This was a double edge sword; I could skip the years of watching charts and creating a strategy through trial and error, which is great. But it was all too easy to fall into the numerous traps of building strategies by backtesting.

The Dangers of Backtesting

There is no guarantee that the market will behave similar in any way to what it has done recently, and even less likely, what it has done 10 years ago.

Even if it behaved similarly, the order in which the opportunities present themselves could be wildly different.

So whatever your backtest showed for performance, drawdowns, metrics or trades might be the absolutely best case, or the worst case.

Making less profits than backtested is not a big issue but losing more than you thought is.

That single equity curve is never enough to forecast how your strategy will perform in the future, and certainly not enough to determine if it has any edge at all.

I learnt this the hard way, when I would spend weeks building an algorithm by endlessly tweaking parameters, backtesting, adding irrational filters then backtesting, changing the timeframe then backtesting.

My historical performance was epic, but my performance trading live was very different.

The Benefits of Backtesting

After getting burnt a few times with strategies that were far too overfit and fragile I learnt that backtesting, when used correctly and as intended, is a very powerful tool.

The only questions I would try to answer with backtesting is:

▶ Trades open when expected?

▶ Stops get placed accurately?

▶ Regime filter detects the right periods?

▶ Trade exit logic triggers correctly?

▶ Trailing Stops moves accurately?

Backtesting is perfect for checking that the logic in your algorithm works as expected.

That’s it.

There are better techniques specifically designed to measure edge, expected performance and drawdowns.

Which we will explore in the next post.