The question
A back-test shows my mix would have done well. How much should I trust it?
The idea
- A back-test applies a rule (these funds, weights and rebalancing) to past prices to see what would have happened. It is useful, and easy to fool yourself with. Ask three questions:Which window? Start and end dates can change the answer more than the rule does.Which funds? A back-test can only use funds that still have prices. Closed funds are missing, so the past looks better than it was. This is survivorship bias.When was the rule chosen? A rule picked today, then run on old prices, uses hindsight. ETFIQ labels those results hypothetical. Only results earned after the rule was fixed are tracked.
Worked example
The 5 steps beside the drawing at the top work this through on ETFIQ data. Point at a step to light the part of the drawing it describes.
ETFIQ's Strategic 60 benchmark is 60% VT (world stocks) and 40% US bonds (BND and AGG), reset to those weights each month end (Intermediate lesson 7). Every window below ends Oct 30, 2026.
| Window | Starts | Return a year (compounded) |
|---|---|---|
| 1 year | Oct 31, 2025 | 7.33% |
| 3 years | Oct 31, 2023 | 15.21% |
| 5 years | Oct 29, 2021 | 5.91% |
| 10 years | Oct 31, 2016 | 8.09% |
| Since Jan 2010 | Jan 4, 2010 | 7.52% |
Pick 3 years and the mix looks brilliant; pick 5 and it looks dull.
Only the start date changed.
The common mistake
"The back-test made 15% a year, so that's what the mix earns." That is the 3-year window, chosen after a fall had ended. Report every window to one end date, with the worst fall, labelled hypothetical.
A second mistake: trusting a back-test that "beat everything." Try enough rules on the same past and one looks great by chance. Choose the rule first, then run every window.
Check your understanding
Pick an answer to see why. Two of three right passes the lesson.
1. One mix shows 1.72% a year over 2020 to 2022 and 14.33% a year over 2019 to 2021. What changed?Answer: D. The window: one year dropped, another added. swapping 2019 for 2022 swapped a gain for a loss. Not B: the mix did not move.2. A back-test of today's small-company funds runs from 2005. Some small-company funds closed in 2009. What does that do?Answer: C. It likely makes the past look better than it was. failed funds are left out, so only survivors count. Not A: an investor in 2005 could have held a fund that later closed.3. A rule was fixed on Mar 1, 2026. Its 5-year result to Oct 30, 2026 is shown. How should it be labelled?Answer: B. Hypothetical. a window starting before the rule was fixed is hypothetical as a whole. Not C: ETFIQ never splits a window.
Takeaways
- Show every window to one end date; one window is a choice, not a result.
- Ask which funds are missing: survivors flatter the past.
- A rule chosen today and run on old prices is hypothetical.