Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/EXPERIMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,20 @@ All rows passed the CER parity gate at release. Readings:
- **Prediction (registered):** 4.45-4.75 — a register-diversification
effect at constant budget should be real but second-order; if the
domain attribution is right, this should beat E5's MTP-aux.
- **Status:** COMPLETE 2026-09-02 — **GATE FAILED: 5.8057** full-set
windowed DER (gate <=4.5218; control 4.8218; E5's failed 5.0853).
NOT ADOPTED. Teacher reproduces at 2.289 (n=1200, protocol
consistent). Paired bootstrap student-teacher: +3.2388pp, CI
[2.939, 3.575]. The prediction (4.45-4.75) missed badly. Read:
swapping news-domain units OUT for classical-register units at
constant budget HURT (−0.98pp vs control) — the news-domain
teacher labels match the benchmark surface better than classical
units do, and 30k total was already enough budget that register
mix was not the binding constraint. Together with G2a (4.5701 at
6 epochs) the ladder now reads: longer training and MORE total
units move the rung; register swaps at constant budget do not.
Clean negative; direction shifts to G2b (48k add). Labels sha256
39b4213b5ab4c9f7273da90a9a1954f7701320b62870a954e6e2c02e810617a4.

## Parked

Expand Down
21 changes: 18 additions & 3 deletions docs/PUBLICATION-NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,19 @@ Paper: frontier section (fourth question). RESULTS.md run-003-pkm.
RL teacher polishing flat/negative ×3; microkimi bridges improve
structure but not accuracy; teacher beam-search unnecessary for
Arabic; per-channel int8 rejected on measurement; the 30 MiB tier
closed as infeasible without pretraining.
closed as infeasible without pretraining; **E5 MTP-aux (2026-09-01):
5.0853 vs the 4.8218 control — multi-token-prediction as a training
auxiliary HURT at this scale (+0.26pp), with a disclosed preemption
confound (fresh aux head for the final 23% of steps); E6
constant-budget register swap (2026-09-02): 5.8057 — replacing news
units with classical Tashkeela at constant total HURT (−0.98pp vs
control), the domain-shaped-residual hypothesis's causal test
failing in the swap direction; the add direction (G2b) remains
open.** The E5/E6 pair is the paper's data-vs-architecture exhibit:
two levers from the frontier-LLM literature (MTP, register
diversification at constant budget) both regressed on byte-level
student distillation — the levers that moved the rung were optimizer
(E3), fresher teacher labels (E4), and epochs (G2a).

### 9. Muon optimizer A/B on the memory student (E3) — LANDED 2026-08-28
**4.8287 vs 7.5553 full-set (−2.727pp from the optimizer alone); adopt
Expand Down Expand Up @@ -217,8 +229,11 @@ The decomposition for paper B, every line full-set with brackets:
| depth halved (lite, 6ep) | 5.784 | delta 3.25 [3.03, 3.49] |

Paper-B framing: levers compose roughly additively (optimizer >>
teacher > epochs), the remaining ~2.0-2.2pp is domain-shaped (the
label-scale rung is the causal test), and capacity appears ONLY as
teacher > epochs), and the E6 causal test came back NEGATIVE —
swapping 8k news units for classical-register Tashkeela at constant
30k total scored 5.8057 (−0.98pp vs control), so the residual is
not fixed by register mix at constant budget; the add direction
(G2b, 48k total) is the live test, and capacity appears ONLY as
depth — width is load-bearing (both stitch ratios collapsed) while
depth trades 1.21pp for 37% of the artifact. The subset-overstatement
phenomenon (five instances, up to 3.2x inflation) is the
Expand Down
Loading