Skip to the content
TTHINRIDGE Audit your trades Test your strategy free

What luck alone would pass

The shelf tests 550 well-known setups and markets. How many of those results could luck alone have given? Worked out from the same coin flips the verdicts use.

Short answer: counted across all 535 tests, no result on the shelf is strong enough to rule out luck at the usual 5% bar, the passes included. For the passes, luck alone would account for about 18% to 26% of results that good. The three passes are one setup, the morning move, on three stock-index markets that move together.

The count

Of the shelf’s 550 tests, 535 made trades (the other 15 made none). If none of the setups had an edge, about 27 of them would beat their coin flips by 1 in 20 on the years before 2023 by luck alone, and about 5 by 1 in 100. In fact 59 beat them by 1 in 20, and 19 by 1 in 100.

That is more than luck would give. Three things say it is not enough to call any one of them an edge.

1. Both ends are crowded

Here is every test with trades, by the share of its 5,000 coin flips that did as well or better. If the setups had no edge, each tenth would hold about 54.

Share of the 5,000 coin flips that did as wellTestsBy luck alone, about
0% to 10% (the best tenth)8654
10% to 20%5354
20% to 30%4154
30% to 40%3454
40% to 50%3954
50% to 60%5854
60% to 70%4054
70% to 80%6054
80% to 90%5054
90% to 100% (the worst tenth)7454

If a few setups had real edges, only the best tenth would be crowded. The worst tenth is crowded too. That means a rule with no edge drifts further from the middle than our 5,000 coin flips allow for: trades bunch up in the same years and the same trends, and flipping the side of each trade doesn’t copy that. So on this shelf, beating the coin flips by 1 in 20 happens more often than 1 time in 20 by luck, and the 59 above overstate the case.

2. Test enough things and some look good

The standard allowance for many tests at once, the false-discovery rate (Benjamini–Hochberg), asks of each result: of results this good or better, what share would luck alone account for? At the usual 5%, none of the 535 qualifies. The strictest allowance (Bonferroni) can’t be met by any single test here: with 535 tests it asks for better than 1 in 10,700, and 5,000 coin flips can show 1 in 5,001 at best.

3. The years since 2023 don’t tell them apart

The years since 2023 are kept out of the coin flips, so nobody tuned on them. Of the 58 tests that beat their flips by 1 in 20 before 2023 and traded 30 times or more since, 21% made money since 2023. Of the other 469, 21%. If the strong results were edges, they should have done better in the years nobody tuned on.

The ones that passed

TestScoreCoin flips as good, before 2023Luck would account forSince 2023
The morning move on NQ9015 of 5,00021%$49,775 over 162 trades
The morning move on YM8937 of 5,00026%$2,552 over 77 trades
The morning move on RTY924 of 5,00018%$11,608 over 197 trades

The three passes are one setup, the morning move, on three stock-index markets that move together, so they count as about one result, not three. On ES the same setup scored 89 and is held at Close because its range of likely results before 2023 includes a loss. They passed a bar set before the runs, and the bar stays. A pass here means what it always meant: worth watching forward, not proven. They are filed on Filed before it happened, where the months after the filing decide.

What this page can’t fix

The years before 2023 are the years most of these setups were made popular in, so their authors may have seen them. Nothing here can undo that; the years since 2023 and the forward record are the clean tests. The coin flips test whether a setup picks the side well; they don’t test its timing, except for setups that only buy or only sell, which are compared with random times instead.

Check it yourself

Every test is in verdicts.csv with its coin flips (k of 5,000 as good or better) and the share luck would account for. For each test with trades, p = (k + 1) / 5,001; the shares are Benjamini–Hochberg across the 535. Worked out from the shelf’s records, run from November 1 to November 16, 2026. How a verdict is reached: the method, on one page.

Hypothetical backtests: one contract, after costs, not real trading. Simulated results have limits: they are not actual trading, they are made with the benefit of hindsight, and no account is likely to achieve results like these. The standard statement