Almost everyone does this, and almost nobody checks it: sweep a grid of parameters, keep the setting that looked best on history, then trade it. The question is: does that setting still work on data it has never seen?
The test
174 moving-average pairs on ^GSPC over twenty years, split in half. Score each pair’s Sharpe on the first half and on the second. If tuning means anything, the two numbers should travel together.
Each dot is one parameter pair. Across: Sharpe on the first half, where it was chosen. Up: Sharpe on the second, where the money would be.
The cloud has no tilt. Correlation: −0.073.
The number that stopped me
The best pair on the first half was MA 50/140 at Sharpe 0.67. On the second half it returned 0.56 — ranking 151st of 174.
Whereas doing no tuning at all, picking any pair blindly, the expected second-half Sharpe was 0.68.
So tuning made it worse than not tuning. Not neutral — actively harmful.
Nor is it a one-off: AAPL gives a correlation of −0.103 with the trained winner falling to 116th of 174. BTC-USD gives +0.049 and its winner happened to hold 13th — and “happened to” is the whole point.
Why
What you select on the first half is not the best parameter. It is the parameter that fits that period’s noise most closely. Noise does not repeat, so the outperformance evaporates and you are left holding a setting bent around a past that is over.
The more combinations you sweep, the more certainly you will find one that looks superb — and the more certainly it is a coincidence.
What this costs us
It is why every indicator here runs on conventional settings — RSI 14, MA 20/50 — rather than optimised ones. Nothing is fitted in advance, so nothing quietly expires.
That sounds like leaving capability on the table. It is declining something that has been measured and does not work.