Public simulation · real Binance futures prices · no orders, no real money

Does an AI reviewer make a trading system better?

BreakoutBot is a rule-based crypto breakout system with fixed-dollar risk and hard drawdown limits. Two copies of it run side by side on the same prices. One follows its rules alone. The other is built to ask an AI model to review every full entry first and skip the ones it vetoes — but the AI only goes live after passing a test written down in advance, and so far no model has. Everything both of them do is published here, losses included.

the rules alone Without AI Every signal the strategy produces is taken. This is the system as designed — the baseline the AI has to beat. loading… Open the track record → same bot + an AI veto With AI Same code, same settings, same account at the split. Before each full entry the AI reads the candles, the indicators and the bot's own stop and targets, and may veto it. It can remove trades, never add one. AI not live: no model has passed the test yet. The numbers below are frozen as of 29 Sep. Details below. loading… Open the track record →

With AI: no model has passed the test yet (13 Oct)

The AI is not reviewing trades right now. We will only switch it on after it passes a test that was written down before any result existed. So far no model has.

The test (offline, 13 Oct). We replayed the bot over two periods that come after the models' training data ends: 1 Jul – 24 Aug and 25 Aug – 24 Sep. At each of 98 full entries we asked the AI to allow or veto it. To pass, an AI has to leave the account better off, after its own API cost, in both periods.

Claude Opus 5.5 — failed. In the second period its vetoes skipped mostly losers (+$8.80). In the first period they skipped mostly winners (−$34.15). One good period is not enough.
Claude Sonnet 5.5 — no verdict. It vetoed only 4 of 98 entries, too few to measure anything.
Claude Haiku 5.5 — failed. It vetoed winners more than losers and lost money in both periods (−$27.16 and −$105.28).

The whole test cost $6.84 in API fees. Next month we will test one new reviewer design the same way. Until one passes, there is no live AI veto.

Meanwhile, a control run. Since 1 Oct the With-AI bot has been running with the AI switched off: it builds every review but always allows the entry. After 11 days and 82 entry signals it matched the bot without AI trade for trade, to the cent. So the setup itself does not create a gap. Any future gap will be down to the AI.

The first run, 25–29 Sep, is void. We sent the AI a wrong input: at every review it was told that five positions were open, while the bot held none (once, one). The bug was in our code, not in the model. It vetoed 3 of 9 entries, each time citing that phantom exposure. The With-AI numbers on this page are frozen as they stood when we stopped it. Of the $23.10 gap, −$9.45 came from the three vetoes, −$13.23 from two trades the AI was never asked about (most likely slightly different price data; we have since fixed how candles are read), and −$0.42 from fees and fills.

Since the split

Loading…

How the experiment is run

One intended difference. The two bots were split from the same account at the same moment — same balance, same open positions, same risk state. The only designed difference is the AI veto. But each bot fetches its own prices, and a candle read two seconds after it closes is often not final yet, so the paths can also drift apart without the AI being involved. We count that drift separately and do not credit it to the AI or blame it on the AI.
Exit change on 12 Oct 2026, 21:21 UTC — both bots. Once a trade reaches its first target, its trailing stop no longer drops below the entry price. Before, a trade could touch the first target and still close at a loss. In backtests the change helped slightly in both test windows (about +0.02R per trade, well inside the noise), so we adopted it as a small, low-risk fix, not a proven gain. Results before and after this moment come from slightly different exits.
The AI can only say no. It is asked once per confirmed entry. A veto skips that entry; anything else — an allow, a timeout, an outage, a spent budget — lets it through. The With-AI bot can therefore trade less than the one without it, never more.
Decided in advance. The test that decides whether the AI helps was written down before any result existed: a first check at 100 closed positions, a decision at 300, against a random veto of the same rate. Which page is ahead on a given day is not that test — one account path is not evidence.
No look-ahead. The AI only sees candles that had closed when the entry was made. It is not told how anything turned out.
Simulation. Prices are real Binance futures data; entries, fills and balances are simulated. No exchange orders, no real money, no return promises, not financial advice.