A pre-registered, 330-test research programme on six markets (NASDAQ-100, S&P 500, Dow, DAX, gold, EUR/USD) asking whether "volatility compression" carries information, and whether that information is about the size of the next move or its direction.
5-minute bars from January 2022 to July 2026, roughly 320,000 bars per instrument, with every result corrected for multiple testing.
- Volatility scale is strongly forecastable 1 to 3 hours ahead. A quiet hour going into a session handover (Asia → London, London → New York, the cash open, NY morning → afternoon) cuts the probability of a volatility spike in the next two hours by 13 to 55 percentage points. The effect is significant in 24 of 24 instrument × boundary cells (day-block bootstrap 95% CI entirely below zero).
- Compression is a persistent regime, not a memoryless state. The probability of still being compressed one hour later exceeds what a first-order Markov chain predicts by +0.16 to +0.18 on every instrument.
- No directional effect survives. 132 pre-specified directional tests produced 9 raw p < 0.05, against 6.6 expected by chance, and 0 survive Benjamini–Hochberg at q = 0.10. In the volatility family, 67 of 102 survive.
- The structural reason. When a session is going to be quiet, everything shrinks together: the opening range, the excursions and the reversal distances all scale down in the same proportion. A forecast that is excellent for scale therefore says almost nothing about path shape, and directional edge lives in shape.
A raw ATR percentile on index CFDs is mostly a clock: 93 to 96% of "compressed" bars on NQ/ES/YM fell in the Asian session. Each bar is therefore scaled by the volatility normally seen at that time of day:
where
Every feature at bar
where
One observation per boundary per day, so time of day is controlled exactly:
-
pre-window: the 12 bars (1 h) ending at the boundary. "Quiet" means bottom quintile of pre-window
abnfor that boundary. -
outcome: a clock-referenced spike,
$\max_{j \le 24} \text{range}_{t+j} / S_\tau(t+j) \ge 2$ , within the next 2 hours. -
inference: a day-block bootstrap of
$\hat p_{\text{quiet}} - \hat p_{\text{rest}}$ , resampling whole sessions (3,000 draws) so that intraday dependence cannot shrink the interval.
Elsewhere, effects are compared with a matched baseline: direct standardisation on (5-minute slot × calendar month), with numerator and denominator bootstrapped together over days.
All 330 tests are registered in results/research_log.json, each with its hypothesis, definition, control, sample size, p-value and an explicit prespecified flag (282 pre-specified, 48 exploratory). Pre-specified p-values are controlled with Benjamini–Hochberg:
applied separately to the volatility and directional families.
Event: an impulse bar emerging from compression (
Every confidence interval straddles zero, and every point estimate sits inside the spread band (mean spread at entry = 0.065 R).
Three traps were caught and handled before any statistics were run (details in scripts/r00_prep.py):
- The "M5" history is hourly before 2022 for US100, US500 and GER40, because MT5 silently serves coarser bars in deep history. Everything before 2022-01-01 is dropped.
- Broker timestamps are wall-clock EET/EEST, not UTC. A fixed-offset conversion is wrong for about 4 weeks a year, so timestamps are converted through the tz database.
- The broker's contract-spec CSV is column-shifted, so point sizes are asserted from observed price precision instead of being read from it.
- CFD data from one retail broker, with tick volume only and no order-flow information.
- The 2025 to 2026 period is inside the sample. The forecasts are stated with explicit falsification criteria in
FINDINGS.mdfor testing on data after 2026-07-29. - The forecastable quantity is volatility, which a CFD account cannot trade directly. Its value is in risk: position sizing, stop distance, and switching a strategy off on days it will not cover its spread.
scripts/
r00_prep.py raw MT5 export -> clean, clock-correct M5 panel
r0_features.py causal feature panel: ATR, seasonal baseline, abn, states, episodes
r1_persistence.py state persistence vs Markov and surrogate nulls
r2_transitions.py what happens when a compression episode ends
r3_conditional.py depth / duration conditioning, matched baselines
r4_sessions.py session-handover forecasts (headline result)
r5_directional.py every directional hypothesis (parts 7-10)
r6_consequence.py year-by-year stability, causal (trailing) flag, monetisability
r7_orfade.py exploratory lead (flagged as such): opening-range fade
r8_summary.py multiple-testing accounting over the whole tree
vol_core.py, stats_util.py, research_log.py shared primitives
results/ JSON outputs, research_log.json, logs/ (console output of every stage)
figures/make_figures.py rebuilds all figures from results/ (no broker data needed)
notebooks/walkthrough.ipynb executed walkthrough: BH control, persistence, session tables
FINDINGS.md the full 17-part research write-up
pip install -r requirements.txt
python figures/make_figures.py
jupyter notebook notebooks/walkthrough.ipynbFull pipeline from raw bars (format in data/README.md):
export BROKER_DATA_DIR=/path/to/mt5/export
cd scripts
for s in r00_prep r0_features r1_persistence r2_transitions r3_conditional \
r4_sessions r5_directional r6_consequence r7_orfade r8_summary; do python -u $s.py; doneSeeds are fixed. The full pipeline was re-run end to end from a clean copy, and the outputs in results/ are from that run.
Author: Muhammad Talal Malik



