Lesson
The best backtest
By Thomas Jamais (Backtestd) · Published · Updated
In short
We took the best result out of thousands of backtests, 21 times, and retested each on a year it had never seen.
1 of 21 passed. And the one that passed was too small to trade.
The best of many backtests is mostly luck, until new data says otherwise.
The numbers
| Group | Value |
|---|---|
| Each tile: one retest on unseen data | 1 / 21 |
How we measured it
Every number comes from the lab's database: 19905 strategy versions with at least 100 trades each, tested on crypto perpetual futures from 2022 to 2025 with fees, slippage and funding. One version = one strategy with one set of settings. Results are in R (the amount risked per trade).
What to do instead
- Count how many settings you tried before choosing the best one.
- Keep data you never look at, and test only once on it.
- Prefer rules that work in most settings, not one perfect setting.
Verify it yourself
- Write the rule exactly (below: our specification and every setting we tried).
- Get free data: Binance's public archive (data.binance.vision) has years of candles; OKX and most exchanges offer an API.
- Include costs: fees on both sides, slippage, and funding for perpetual futures.
- Count results in R (risk per trade), not in %, and look at the typical version, not the best one.
- Split your data in two and keep a part you never look at until the end.
- Compare with simply holding the market over the same days.
Disclaimer
Educational content based on historical backtests. Not financial advice. Past results do not predict future results. Trading leveraged products can lose more than you invest.