Skip to content

backtesting.md

Latest commit

 

History

History
122 lines (103 loc) · 6.82 KB

backtesting.md

File metadata and controls

122 lines (103 loc) · 6.82 KB
Raw

Backtesting — alphaforge.backtest

The backtest engine is deliberately a pure accounting loop: it owns no alpha logic. All portfolio decisions are delegated to a weight_fn (by default a PortfolioConstructor). Its job is to turn target weights into a return series using configured execution lags, modeled costs and an explicit stale-price policy.

from alphaforge.backtest import BacktestEngine, BacktestConfig, run_backtest

bt = run_backtest(panel, signals=signal_panel, constructor=cons, ic=rank_ic, cfg=cfg)
metrics = bt.metrics            # net (after-cost)
gross = bt.gross_equity         # pre-cost curve (cost-drag overlay)

Design contract

  1. Signals are generated on the rebalance date using only data available on that date. The constructor receives the panel up to and including t.
  2. Orders are executed execution_lag_days sessions later (default 1), at that session's close. This is the structural guard against look-ahead: a signal can never be traded on the same close that produced it. Each signal retains its own execution date when another signal arrives before its fill. At the sample boundary, a signal lacking the full lag is recorded as unexecuted rather than filled early.
  3. The day's return is earned on the pre-trade book, and trading costs are deducted from NAV at the end of the session — so costs reduce the compounding base in the simulation. Fill prices and market impact remain model assumptions.
  4. Untradeable names are not silently dropped. A name with no price on the execution date keeps its position (marked at the last valid close); a name that has been dark for delist_grace_days sessions is force-liquidated at the last valid price. Dropping them instead is how backtests acquire an undeclared survivorship bias.

Configuration — BacktestConfig

Field Default Meaning
rebalance monthly rebalance cadence
initial_capital 10,000,000 NAV base
execution_lag_days 1 positive number of trading sessions between signal and fill
min_history_days 252 risk-model warm-up; first rebalances skipped until the window fills
adv_window 20 ADV for impact
max_stale_days 5 mark-to-market ffill limit; must cover the delist grace window
delist_grace_days 5 positive number of dark sessions before force-liquidation
max_gross_leverage 1.0 finite, positive gross exposure cap

Session-count settings accept integers or text integers; fractional values and booleans are rejected before they can change execution timing or warm-up. min_history_days is nonnegative and adv_window must cover the five-session minimum used for ADV. Rebalance names must be one of the five documented cadences. initial_capital must be finite and positive. allow_short accepts a boolean or the strings "true" and "false"; other values are rejected. An expired mark cannot silently remove a held position before its liquidation window.

Costs — gross vs net, made explicit

Transaction costs are modelled by CostModel (commission + slippage + square-root market impact) inside BrokerSimulator, and deducted per trade. The engine then reconstructs the pre-cost (gross) return series by adding the per-day cost drag back into the net series — exact, no re-run. Gross and net Sharpe / CAGR / volatility / Sortino / Calmar / MaxDD are reported side by side, and cost_drag_cagr = gross_cagr - net_cagr states the simulated transaction-cost burden in one number. The gross_equity curve is shipped for the cost-drag overlay in the report.

Commission, slippage, impact, minimum commission and borrowing assumptions must be finite and nonnegative; participation caps are within (0, 1]. Invalid cost assumptions fail before a run rather than crediting NAV. The broker's allow_fractional_shares accepts booleans or explicit "true"/"false" strings; "false" enforces integer fills. Minimum trade value is finite and nonnegative.

Large finite notionals and basis-point rates are scaled before multiplying so a representable fee does not overflow only because of the calculation order. Nonfinite trade values, unrepresentable cost components or totals, and fees greater than the available NAV raise ValueError. The engine applies the same fee boundary to custom brokers and forced liquidation. Fees equal to NAV are allowed: they produce a zero-capital bankruptcy, with zero subsequent returns. That boundary is absorbing; a later price recovery cannot recreate capital. Negative or nonfinite NAV and nonfinite reconstructed gross NAV are rejected. Finite positive NAV that rounds to a -100% daily return is also rejected, as the exported float return cannot represent that remaining wealth. Execution rejects share quantities outside the finite float range.

The benchmark is stored as a daily return series (the ETL layer converts provider price levels exactly once). ensure_returns makes that contract explicit: a return series is never double-differenced (an earlier bug turned a ~15%-vol benchmark into a 58,000%-vol one and silently destroyed beta/alpha), while a level series handed in directly still works.

Metrics — performance_stats

Every statistic is computed from the realised return series, so it already includes whatever the strategy paid. Definitions are stated because implementations differ:

  • Sharpe — (mean - rf/period) / std * sqrt(periods), excess over the period's risk-free accrual, not over zero.
  • Sortino — same numerator, denominator = downside deviation vs a zero target with all observations in the denominator (Sortino-Satchell), so a flat series does not get an infinite ratio by accident.
  • Max drawdown — worst peak-to-trough of the compounded curve.
  • Alpha / beta / IR / tracking error — Jensen's alpha from an OLS of excess strategy returns on excess benchmark returns, annualised from the intercept.
  • Annualisation is inferred from the index, never assumed, and reported alongside the numbers that depend on it.

Compounded wealth rejects infinite returns, returns below -100%, and a curve outside the finite float range instead of treating them as a successful run or a fabricated wipeout. A return of exactly -100% legally reduces wealth to zero. Missing returns retain the existing flat-period convention.

Output — BacktestResult

equity, returns, end-of-day weights, rebalance-date target_weights, trades (with signal_date and execution date), turnover, costs, metrics, gross_equity, benchmark, plus a diagnostics dict (n_rebalances, n_skipped_rebalances, n_unexecuted_signals, total_costs, cost_drag_ann, avg_turnover, avg_holdings, avg_gross_exposure). The reporting layer and API consume this object. Compare outputs from the same run when checking their agreement.