Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/EXPERIMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,28 @@ All rows passed the CER parity gate at release. Readings:
the tail, but the factorial attributes most of the remaining gap to
domain coverage, so the effect should be second-order.

## E6 — label-scale/domain-mix rung (run-008-tashkeela-mix)

- **Status:** REGISTERED 2026-09-01; data built (120,000 decontaminated
Tashkeela units on rababa-datasets:r5-units/tashkeela-scale.txt);
launches when a GPU slot frees (E5 + owner runs hold the budget).
- **Hypothesis:** the E2/E3 factorial attributes the client tier's
remaining ~2.0pp to domain coverage; the shipped mix is news-heavy.
Replacing 8,000 of the 24,000 news-domain units with classical-
register Tashkeela units AT CONSTANT TOTAL (30,000 units, identical
steps/schedule/optimizer/teacher) diversifies register coverage —
classical text is also the benchmark's own register.
- **Data hygiene:** lines <=1,450B with Arabic; decontaminated against
SadeedDiac-25 by stripped-normalized exact match and 80-char prefix
match (48 units dropped); deduped internally (17,036) and against
the existing domain/replay units; seeded shuffle; 120k emitted, the
chain samples 8,000.
- **Pre-agreed gate:** adopt if full-set windowed DER <= 4.5218
(>=0.3pp over 4.8218, the E3-style bar); honest report otherwise.
- **Prediction (registered):** 4.45-4.75 — a register-diversification
effect at constant budget should be real but second-order; if the
domain attribution is right, this should beat E5's MTP-aux.

## Parked

- **Speculative decoding** (LongCat converts sparsity→speed): revisit
Expand Down
23 changes: 23 additions & 0 deletions src/gpu/distill_specs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,29 @@ ara-diac-small-2-mtp:
beta: 0.15
mode: sequence

ara-diac-small-2-scale:
# E6 (EXPERIMENTS.md): constant-budget register diversification —
# 8,000 Tashkeela classical units replace 8,000 news units (30k total,
# identical steps/schedule to run-006/E5's control)
teacher: rababa_arabic_byt5/run-007-news/best
teacher_volume: rababa
student_init: google/byt5-small
optimizer: muon
muon_lr: '0.01'
train: r5-units/domain.txt
train_extra:
- r5-units/tashkeela-scale.txt
- r5-units/replay.txt
unit_limits:
- 16000
- 8000
- 6000
max_len: 1450
label_beams: '1'
out: rababa_arabic_distill_small/run-008-tashkeela-mix
labels_file: teacher_labels_r7.jsonl
mode: sequence

ara-diac-small-2:
teacher: rababa_arabic_byt5/run-007-news/best
teacher_volume: rababa
Expand Down
Loading