Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/PUBLICATION-NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,16 @@ stack. The frontier law (pretrained-or-collapse) and the controlled
capacity/aux/optimizer experiments give it empirical heft. Package as
one paper (current paper.adoc); the margin/head finding alone is also
a strong short workshop paper if a split is ever wanted.

## ara-diac-tiny: the "clean-label re-run" wasn't (2026-08-29)

Narrative caution for the paper: the Aug-24 retraction flagged that the
83.08 tiny-collapse number was measured on mojibake labels. The
attempted clean re-run (run-004) silently reused the same Aug-23
snapshot via a baked-in `labels_file` path — and reproduced 83.0797
exactly, which is itself the tell: identical-to-4-decimals means same
data, not same capacity law. Two lessons for the experiments section:
(1) label-provenance must be content-hashed into the run record, not
inferred from filenames; (2) a reproduced number is only evidence of
reproducibility when the input pipeline is versioned. run-005
(regenerating labels live from the r6 teacher) is the honest test.
18 changes: 18 additions & 0 deletions docs/RESULTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,24 @@ conclusion for Arabic is UNPROVEN pending a clean-label re-run. The
Thai tiny verdict is unaffected (umt5/sentencepiece labels were
byte-exact); the pretrained-backbone law rests on Thai evidence.

## ara-diac-tiny run-004 — the retracted verdict reproduced, on poisoned data (2026-08-29)

The spec intended to rerun the tiny tier on clean labels silently
consumed the Aug-23 label snapshot (pre `decode_joined` fix). Evidence
chain:

- run-004/best decodes `كتاب` as `ÙÙØ§ØªÙب` — UTF-8-as-Latin1 mojibake,
the exact label corruption the retraction describes
- the 300-paragraph windowed gate (n=300, teacher reproduces its
documented 1.3205) scores the student **83.0797** vs the retracted
**83.08** — identical to four decimals: the poisoned-data constant,
not a capacity result
- `final_eval.json` in the run dir is the durable provenance

Verdict: run-004 says nothing about from-scratch capacity; the
retraction stands. **run-005** (fresh teacher labels, no snapshot,
2026-08-29) is the actual clean-label falsification test — in flight.

## ara-diac-small-1.0 — Arabic client tier (2026-08-24)

Sequence-level KD from the r6 teacher (rababa_arabic_byt5/run-006-morph,
Expand Down
Loading