Pitfall 1: Survivorship Bias in Asset Universes
Testing an equity strategy on the current constituent list of the S&P 500 or KSE-100 over a 10-year historical window introduces massive positive bias. Companies that went bankrupt, merged, or were delisted during market crashes are excluded from the test, creating an artificial world where you only trade the eventual winners.
The Solution: Always use point-in-time constituent datasets that reflect the exact basket of available stocks on each historical calendar date.
Pitfall 2: Overfitting via Parameter Grid Exhaustion
If you sweep through 2,000 combinations of moving average periods, RSI thresholds, and stop-loss widths on 3 years of historical data, you will invariably find a parameter set that produced stellar returns. You have not discovered market alpha; you have memorized historical noise.
The Solution: Enforce strict Walk-Forward Optimization and Deflated Sharpe Ratio (DSR) adjustments (Bailey & López de Prado, 2014) to penalize strategies based on the number of trials tested.