From 2d6af2210921f908c9ceb7d3841d92e67ad907cf Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Mon, 31 Aug 2026 06:05:27 +0200 Subject: [PATCH] =?UTF-8?q?docs(eval):=20GLM-5.3-Flash=20on=20SadeedDiac-2?= =?UTF-8?q?5=20=E2=80=94=20raw=208.5911=20/=20zero-skip=208.8995?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First 5.3 measurement on this benchmark; effort=low pinned and the 5.2 thinking-knob incompatibility (HTTP 400 code 1210) documented. Flash tier lands below Sadeed-1.5B. --- results/sadeed-glm-5-3-flash/README.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 results/sadeed-glm-5-3-flash/README.md diff --git a/results/sadeed-glm-5-3-flash/README.md b/results/sadeed-glm-5-3-flash/README.md new file mode 100644 index 0000000..6ef0b9e --- /dev/null +++ b/results/sadeed-glm-5-3-flash/README.md @@ -0,0 +1,22 @@ +# GLM-5.3-Flash on SadeedDiac-25 (2026-08-31) + +First measurement of glm-5.3-flash on this benchmark. Protocol: neutral +completion prompt, temperature 0, max_tokens 8192, **reasoning_effort=low** +(explicitly pinned — absent/unrecognized values default to MAX; the 5.2-era +`thinking:{type:disabled}` knob is rejected by 5.3 with HTTP 400 code 1210 +and must not be combined with reasoning_effort). Reasoning content is +present at effort=low (tripwire disclosed); checkpoint +/tmp/sadeed_glm53_clean.jsonl, 1,200/1,200 non-empty. + +| Protocol | DER (CE) | +|---|---| +| raw (their default output) | 8.5911 | +| projected zero-skip | **8.8995** | + +Context: GLM-5.2 zero-skip scored 2.6911 (our reproduction, thinking +disabled). The 5.3 flash tier regresses on this benchmark to below +dedicated Sadeed-1.5B (7.2915) — supporting the dedicated-vs-frontier +positioning: diacritization quality is not carried by general-frontier +scale at the flash tier. Effort=high/max remains unmeasured (protocol +prefers plain completion; a max-effort run would not be protocol-matched +to the other rows).