Skip to content

docs(eval): GLM-5.3-Flash leaderboard row — zero-skip 8.8995 #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ronaldtse wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs(eval): GLM-5.3-Flash leaderboard row — zero-skip 8.8995 #64

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions results/sadeed-glm-5-3-flash/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# GLM-5.3-Flash on SadeedDiac-25 (2026-08-31)

First measurement of glm-5.3-flash on this benchmark. Protocol: neutral
completion prompt, temperature 0, max_tokens 8192, **reasoning_effort=low**
(explicitly pinned — absent/unrecognized values default to MAX; the 5.2-era
`thinking:{type:disabled}` knob is rejected by 5.3 with HTTP 400 code 1210
and must not be combined with reasoning_effort). Reasoning content is
present at effort=low (tripwire disclosed); checkpoint
/tmp/sadeed_glm53_clean.jsonl, 1,200/1,200 non-empty.

| Protocol | DER (CE) |
|---|---|
| raw (their default output) | 8.5911 |
| projected zero-skip | **8.8995** |

Context: GLM-5.2 zero-skip scored 2.6911 (our reproduction, thinking
disabled). The 5.3 flash tier regresses on this benchmark to below
dedicated Sadeed-1.5B (7.2915) — supporting the dedicated-vs-frontier
positioning: diacritization quality is not carried by general-frontier
scale at the flash tier. Effort=high/max remains unmeasured (protocol
prefers plain completion; a max-effort run would not be protocol-matched
to the other rows).