Lesson 12 Backtesting and Reviewing Evidence

Keep a reproducible test log, avoid common biases, and calculate basic performance statistics with appropriate limits.

Specify the test before viewing results

Backtesting applies predefined rules to historical information. It can reveal arithmetic errors, unclear rules, trading costs, and behaviour across different periods. It cannot guarantee how a future account will perform. Save the strategy version, data source, price type, time zone, date range, spread and commission model, gap rules, and handling of ambiguous bars. Freeze the rules for the first test. If you change them, label a new version and avoid treating the same data as independent confirmation.

Separate development from evaluation

For a classroom exercise, log the first 30 eligible trades in chronological order, preserving all rejected signals separately. Use the first 20 for development review and the next 10 as a holdout you do not inspect until the version is fixed. Reserve the holdout dates in advance. These small counts teach the process; they are too limited to establish a robust trading advantage. If you inspect the holdout and then change the rules, it becomes development information. A later test needs new unseen data. Do not delete an unfavourable trade because its setup “did not look right” after the fact.

Avoid common distortions

Look ahead bias: Using data that was not available at the decision time, including an unclosed candle or an unconfirmed swing.
Selection bias: Choosing favourable dates or keeping only winning examples.
Overfitting: Adjusting many parameters until they match historical noise.
Execution optimism: Using midpoint entries, ignoring spread, assuming every stop fills exactly, or ignoring minimum sizes and margin.

Use a bar replay or reveal one bar at a time. If you lack the required historical calendar or reliable bid and ask information, state the limitation and do not claim to have tested the full rules. You can still complete an explicitly simplified mechanics exercise.

Summarise net outcomes

Record trade count, net cash result, win rate, mean net R, largest peak to trough equity drawdown, and rule adherence. Count a net zero trade separately as breakeven and include it in the denominator for trade count. Compute average net R as the sum of all net R outcomes divided by all trades. Report deposits or resets separately so they do not distort the equity curve. Hypothetical sample: Ten trades produce +20 USD total net result with fixed 10 USD initial risk per trade. Average net R is 20 ÷ 10 ÷ 10 = +0.20R. This is a description of those ten trades, not a forecast. Run a sensitivity check by applying one additional pip of adverse execution at entry and one at exit to every model trade. For EUR/USD, deduct the resulting two pip cash cost from each original outcome and report how the statistics change. This is a cost sensitivity exercise, not a full simulation of altered stop and target paths. A more realistic retest recalculates paths and fills under the changed assumptions.

Practice

Complete the 30 trade log, or retain an unfinished log and continue until enough eligible observations exist. Report development and holdout results separately. Identify one limitation and one next test. A negative result is valid evidence and does not justify hiding the sample.

Check your understanding

After the four lessons in this module, complete Module 3 knowledge check from the course overview. It contains the three questions from each lesson with feedback after submission. Keep written calculations in your workbook.

Glossary

Ask: Price at which the customer can buy the pair
Bid: Price at which the customer can sell the pair
Commission: A separate trading charge under the provider’s terms
Drawdown: Decline from an equity peak to a subsequent lower value
Equity: Account value including open profit or loss and applicable adjustments
Gap: A jump between available prices that skips intermediate prices
Holdout: Data reserved from strategy development for later evaluation
Margin: Collateral required to support a position
Pip: A conventional exchange rate movement unit
R: A result measured relative to the initial planned cash risk
Spread: Ask minus bid