Skip to content

docs: ara-diac-tiny verdict (collapsed); eval DER x100 fix - #34

Merged
ronaldtse merged 1 commit into
mainfrom
docs/ara-tiny-verdict
Aug 23, 2026
Merged

docs: ara-diac-tiny verdict (collapsed); eval DER x100 fix#34
ronaldtse merged 1 commit into
mainfrom
docs/ara-tiny-verdict

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

Summary

  • ara-diac-tiny (33M from-scratch byte student) verdict: 83.08% DER-CE vs teacher 1.32% — collapsed, REJECTED; closes the 30MB tier as a negative result across both task families (Thai tiny collapsed at 75.8 PER)
  • the gate harness was validated by the teacher reproducing its documented tier on the same replication (1.32% over 300 Sadeed paragraphs, windowed 1400B, Misraj evaluator)
  • fix: evaluate_der multiplied the evaluator's already-percent DER by 100 again — every der_ce this entrypoint printed was 100x inflated
  • frontier conclusion updated with the int4 rung (202MB)

Test plan

  • CI green

… teacher 1.32%); eval ×100 fix

evaluate_der multiplied the Misraj evaluator's already-percent DER by
100 again; the harness was validated by the teacher reproducing its
documented tier (1.32% on the 300-paragraph replication). The tiny
tier is closed as a negative result across both task families.
@ronaldtse
ronaldtse merged commit 37bdf65 into main Aug 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant