Audit principle: reconstruct the information and order timeline before examining the equity curve. If the timeline is impossible, the performance statistics are irrelevant.
1. Look-ahead bias
Look-ahead bias occurs when a decision uses information that was not available at the simulated decision time. Obvious examples include tomorrow’s return or a later-revised fundamental value. Subtle examples include using the current composition of an index in a historical test, using a full-sample normalization, or calculating an end-of-day indicator and earning the return into that same close.
TickRun calculates signals from daily closing information and applies the previous session’s position to the current close-to-close return. In code, the position is shifted by one row before multiplying by asset return. That blocks same-interval use of the closing signal. It does not prove a real order could fill at the exact marker price; that remains a modeling simplification.
2. Same-bar execution
A daily rule often says “buy when the close crosses above the average.” The final close establishes that the cross occurred. Assuming a fill at that same final close requires an order to be submitted after observing the close but executed before the close—an impossible sequence.
Valid alternatives include applying exposure to the next close-to-close interval, simulating the next open with open data, or using intraday information that genuinely precedes the fill. The chosen convention should be explicit. A one-session delay stress test can reveal whether a result depends on an unrealistically precise boundary.
3. Survivorship and present-day selection
Survivorship bias appears when failed or delisted securities disappear from the tested population. Present-day selection is broader: choosing today’s famous winners and then studying their history also uses knowledge from the end of the sample.
TickRun currently offers individually maintained snapshots for AAPL, KO, and NKE. These are useful for demonstrating strategy mechanics, not for estimating how a rule would have performed across the historical investable universe. A universe-level claim needs point-in-time membership, delisting returns, identifier changes, and securities that ceased to exist.
4. Inconsistent corporate-action data
Splits and dividends can create artificial jumps if open, high, low, close, and volume are not adjusted consistently. Mixing an adjusted close with raw highs and lows can corrupt ATR, channels, stochastic oscillators, and any rule comparing fields within a row.
TickRun expects a self-consistent adjusted OHLCV file. Updating only one column or concatenating data from incompatible adjustment methods can produce valid-looking numbers with invalid economic meaning. Data maintenance therefore requires schema validation plus spot checks around known splits and distributions.
5. Treating turnover as free
Transaction costs compound against a strategy every time its position changes. High-frequency crossover rules may appear attractive before costs precisely because they react to small fluctuations. Those fluctuations also create the most turnover.
A zero-commission broker does not imply zero execution cost. The bid–ask spread and adverse price movement can remain. TickRun’s basis-point field charges once on entry and once on exit. It is a useful sensitivity control but does not estimate spreads or impact from the actual historical market. See transaction costs and slippage for a detailed model audit.
6. Data snooping and unreported trials
If 10,000 variations are tested, the best observed result is the maximum of 10,000 noisy outcomes. It should not be interpreted like the result of one predeclared test. Manual trial and error counts too: changing the threshold after every disappointing chart is an optimization process without a log.
White’s Reality Check and later probability-of-backtest-overfitting work address the fact that model selection changes the statistical question. TickRun deliberately labels optimization as in-sample, reports evaluated setup counts, and caps combinations and time. Those controls bound computation; they do not correct the winner’s selection bias.
7. Reusing the holdout
A test period is out-of-sample only until its result influences development. If a failed holdout causes parameter changes and the same holdout is tested again, information has leaked back into the strategy. Renaming the reused period “validation” does not restore independence.
Maintain a development record and use chronological partitions. For iterative research, walk-forward evaluation creates multiple later segments, but window design and selection rules must themselves be fixed without optimizing on the reported aggregate.
8. Choosing a weak or mismatched benchmark
Comparing an equity strategy only with cash can make ordinary market exposure appear to be skill. Comparing a partially invested strategy directly with fully invested Buy & Hold also needs interpretation because their market exposure differs.
TickRun displays Buy & Hold over the identical ticker history. It currently gives idle cash a zero return and does not report beta, average exposure, or a cash-yield benchmark. A lower-return strategy may still have shallower drawdown; whether that tradeoff is valuable cannot be decided from rank by total return alone.
9. Misreading trade count and win rate
TickRun counts completed entry–exit pairs. An open position at the final observation affects daily equity but does not enter trade count or win rate. Completed-trade return is the raw entry-to-exit price ratio; portfolio equity separately includes configured costs.
This means a displayed win rate is not the percentage of profitable days, and trade returns will not necessarily reconcile to net equity without applying costs and compounding. A 90% win rate can lose money when the losing 10% are much larger.
10. Mixing arithmetic and compounded returns
Daily returns add only as an approximation for small changes. Wealth compounds multiplicatively: equity after two sessions is E₀ × (1+r₁) × (1+r₂). A 50% loss followed by a 50% gain leaves 75% of starting wealth, not 100%.
TickRun compounds each net daily strategy return into equity and derives total return from final equity divided by starting capital. Any external spreadsheet comparison must use the same convention and align the cost dates.
11. Ignoring path and regime concentration
A ten-year return can conceal that all gains occurred during one crisis rebound or one uninterrupted trend. Summary statistics compress sequence. Maximum drawdown records only the deepest observed fall; it omits how long recovery took and whether several slightly smaller drawdowns occurred.
Inspect the equity curve, trade dates, time in market, and performance across distinct regimes. TickRun currently shows the full equity path and completed trades but does not calculate rolling returns, exposure, drawdown duration, or regime attribution.
12. Ignoring liquidity and capacity
A percentage return is not independent of capital. Larger orders can cross more of the spread, consume available depth, and move price. A strategy that appears viable in a liquid large-cap stock may not transfer to a thinly traded security at the same cost assumption.
TickRun uses daily bars and a fixed proportional cost. It does not compare order size with daily volume, cap participation rate, or model nonlinear impact. It should therefore be used to study signal behavior, not to establish deployable capacity.
A practical audit sequence
- Freeze the exact data file and code version.
- Write the information timestamp and simulated fill timestamp for every signal.
- Recalculate several rows manually.
- Reconcile position changes with charged costs.
- Verify compounding from daily returns to final equity.
- List every strategy and parameter set previously examined.
- Compare with a relevant benchmark and multiple cost assumptions.
- Identify which observations, trades, assets, and regimes drive the result.
- Evaluate on untouched later data before making a stronger claim.