From 7a32735d8974994ebd256f0bf0fe7c5d7e40de18 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Wed, 2 Sep 2026 18:21:33 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20E6=20verdict=20=E2=80=94=20gate=20FAILE?= =?UTF-8?q?D=20at=205.8057;=20the=20register-swap=20causal=20test=20is=20n?= =?UTF-8?q?egative?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Constant-budget diversification (8k news units swapped for classical Tashkeela, 30k total, identical schedule) scored 5.8057 vs the 4.8218 control and gate 4.5218 — worse than E5. Teacher reproduced at 2.289 (n=1200). PUBLICATION-NOTES: E5+E6 recorded as the data-vs- architecture negative pair; Paper-B paragraph updated (residual is not fixed by register mix at constant budget; G2b add is the live test). --- docs/EXPERIMENTS.md | 14 ++++++++++++++ docs/PUBLICATION-NOTES.md | 21 ++++++++++++++++++--- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/docs/EXPERIMENTS.md b/docs/EXPERIMENTS.md index 918de0f..19bc162 100644 --- a/docs/EXPERIMENTS.md +++ b/docs/EXPERIMENTS.md @@ -271,6 +271,20 @@ All rows passed the CER parity gate at release. Readings: - **Prediction (registered):** 4.45-4.75 — a register-diversification effect at constant budget should be real but second-order; if the domain attribution is right, this should beat E5's MTP-aux. +- **Status:** COMPLETE 2026-09-02 — **GATE FAILED: 5.8057** full-set + windowed DER (gate <=4.5218; control 4.8218; E5's failed 5.0853). + NOT ADOPTED. Teacher reproduces at 2.289 (n=1200, protocol + consistent). Paired bootstrap student-teacher: +3.2388pp, CI + [2.939, 3.575]. The prediction (4.45-4.75) missed badly. Read: + swapping news-domain units OUT for classical-register units at + constant budget HURT (−0.98pp vs control) — the news-domain + teacher labels match the benchmark surface better than classical + units do, and 30k total was already enough budget that register + mix was not the binding constraint. Together with G2a (4.5701 at + 6 epochs) the ladder now reads: longer training and MORE total + units move the rung; register swaps at constant budget do not. + Clean negative; direction shifts to G2b (48k add). Labels sha256 + 39b4213b5ab4c9f7273da90a9a1954f7701320b62870a954e6e2c02e810617a4. ## Parked diff --git a/docs/PUBLICATION-NOTES.md b/docs/PUBLICATION-NOTES.md index b413d83..80bdcb8 100644 --- a/docs/PUBLICATION-NOTES.md +++ b/docs/PUBLICATION-NOTES.md @@ -70,7 +70,19 @@ Paper: frontier section (fourth question). RESULTS.md run-003-pkm. RL teacher polishing flat/negative ×3; microkimi bridges improve structure but not accuracy; teacher beam-search unnecessary for Arabic; per-channel int8 rejected on measurement; the 30 MiB tier -closed as infeasible without pretraining. +closed as infeasible without pretraining; **E5 MTP-aux (2026-09-01): +5.0853 vs the 4.8218 control — multi-token-prediction as a training +auxiliary HURT at this scale (+0.26pp), with a disclosed preemption +confound (fresh aux head for the final 23% of steps); E6 +constant-budget register swap (2026-09-02): 5.8057 — replacing news +units with classical Tashkeela at constant total HURT (−0.98pp vs +control), the domain-shaped-residual hypothesis's causal test +failing in the swap direction; the add direction (G2b) remains +open.** The E5/E6 pair is the paper's data-vs-architecture exhibit: +two levers from the frontier-LLM literature (MTP, register +diversification at constant budget) both regressed on byte-level +student distillation — the levers that moved the rung were optimizer +(E3), fresher teacher labels (E4), and epochs (G2a). ### 9. Muon optimizer A/B on the memory student (E3) — LANDED 2026-08-28 **4.8287 vs 7.5553 full-set (−2.727pp from the optimizer alone); adopt @@ -217,8 +229,11 @@ The decomposition for paper B, every line full-set with brackets: | depth halved (lite, 6ep) | 5.784 | delta 3.25 [3.03, 3.49] | Paper-B framing: levers compose roughly additively (optimizer >> -teacher > epochs), the remaining ~2.0-2.2pp is domain-shaped (the -label-scale rung is the causal test), and capacity appears ONLY as +teacher > epochs), and the E6 causal test came back NEGATIVE — +swapping 8k news units for classical-register Tashkeela at constant +30k total scored 5.8057 (−0.98pp vs control), so the residual is +not fixed by register mix at constant budget; the add direction +(G2b, 48k total) is the live test, and capacity appears ONLY as depth — width is load-bearing (both stitch ratios collapsed) while depth trades 1.21pp for 37% of the artifact. The subset-overstatement phenomenon (five instances, up to 3.2x inflation) is the