Skip to content

docs: sync ledger/blog with corrected numbers and the Arabic releases #127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 1 commit into from
Aug 28, 2026
Merged

docs: sync ledger/blog with corrected numbers and the Arabic releases #127

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 34 additions & 4 deletions src/content/blog/2026-08-24-phonological-layer-shipped.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
= The phonological layer: ten models, three wrong turns, and one correction
= The phonological layer: eleven models, three wrong turns, and one correction
Ronald Tse
v1.0, 2026-08-24
:doctype: book
Expand All @@ -13,7 +13,7 @@ haraqat a scribe left out. Turning unwritten Thai into phonemes.
Reading Urdu aloud. Those are learned — and for two years they lived
in a research branch, promising and unshipped.

This August we shipped the phonological layer: ten neural models under
This August we shipped the phonological layer: eleven neural models under
the same discipline as the maps — one artifact format, checksums
verified on every load, identical output from Ruby, Python, and
TypeScript, and a measured number next to every claim. This post is
Expand All @@ -32,11 +32,13 @@ The catalog now spans four languages and two tiers:
|urd-g2p-1.0 |grapheme→phoneme |CER 14.77 |fp32, 1.3 GiB
|urd-diac-1.0 |diacritization |CER 3.74 |fp32, 1.3 GiB
|heb-diac-1.0 |diacritization |DER 29.0 greedy / 17.5 beam-4 |fp32, parts
|tha-g2p-base-1.0 |grapheme→phoneme |PER 9.19 (teacher 4.43) |fp32, parts
|fas-g2p-1.0 |grapheme→phoneme |CER ≈1.6 — above published SOTA on SentenceBench |fp32, parts
|tha-g2p-base-1.0 |grapheme→phoneme |PER 3.53 greedy (teacher 1.25) |fp32, parts
|ara-diac-1.0 |diacritization |DER 2.58 — best dedicated on SadeedDiac-25 |fp32, parts
|fas-g2p-1.0 |grapheme→phoneme |CER ≈1.6 — above the published best on the SentenceBench homograph benchmark |fp32, parts
|tha-g2p-small-1.0 |grapheme→phoneme |PER 2.85 greedy |int8, 246 MiB
|tha-g2p-small-1.0-int4 |same student, 4-bit |byte-identical decode |int4, 193 MiB
|heb-diac-small-1.0 |diacritization |DER 30.37 (teacher 24.79) |fp32, 1.3 GiB
|ara-diac-small-1.0 |diacritization |DER 8.26 full-set (see update below) |fp32, 1.3 GiB
|===

Every student was distilled from a frozen, independently evaluated
Expand Down Expand Up @@ -131,3 +133,31 @@ Try it: `pip install secryst` —
Python, the same in TypeScript. The catalog lives at
https://github.com/interscript/interscript-ml[interscript-ml]; the
measured story behind every number lives in its results log.

== Update, 2026-08-26: two Arabic releases and one subset lesson

The catalog grew to eleven: `ara-diac-1.0` (server) and
`ara-diac-small-1.0` (client) shipped for Arabic haraqat restoration.

*ara-diac-1.0* is the r6 teacher — and on the full SadeedDiac-25
benchmark, under the benchmark's own Misraj evaluator protocol, it is
the best dedicated model measured: 2.5793% DER, ahead of GLM-5.2
(2.6911), Gemini Flash (3.1926), GPT-4 (3.8645), and Sadeed's own
1.5B model (7.2915); only Claude 3.7 Sonnet's published 1.3941 sits
above it. At 580M parameters, runnable locally from a checksummed zip.

*ara-diac-small-1.0* taught us a measurement lesson the hard way. Its
first published number, 3.66% DER, came from the benchmark's first 300
paragraphs; the full 1,200-paragraph run scores 8.26%. The teacher
reproduces its documented value on the same run (2.5815 vs 2.5793), so
the harness was sound — the subset simply sat in the student's
training-domain neighborhood and hid a domain-generalization gap. The
subset figure is withdrawn, the full-set number ships in the metadata,
and two standing rules follow: student-tier numbers publish from full
benchmark sets only, and a subset figure is labeled as such at first
publication.

Also corrected above: the Thai server-tier row now carries the greedy
numbers (3.53, teacher 1.25) measured under the runtime protocol — the
decode section earlier in this post explains why the beam-4 figures
overstated the error.
87 changes: 50 additions & 37 deletions src/pages/ml.astro
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,15 @@ const serverModels = [
id: "tha-g2p-base-1.0",
task: "g2p",
pair: "Thai → IPA",
metric: "PER 9.19 (teacher 4.43)",
metric: "PER 3.53 greedy (teacher 1.25)",
artifact: "fp32 · 2.6 GiB · parts",
status: "released",
},
{
id: "ara-diac-1.0",
task: "diacritization",
pair: "Arabic → haraqat",
metric: "DER 2.58 — best dedicated model on SadeedDiac-25",
artifact: "fp32 · 2.6 GiB · parts",
status: "released",
},
Expand Down Expand Up @@ -73,10 +81,18 @@ const clientModels = [
id: "fas-g2p-1.0",
task: "g2p",
pair: "Persian → IPA",
metric: "CER ≈1.6 · homograph 77.34%",
metric: "CER ≈1.6 · SentenceBench homograph 77.34 (published best 76.89)",
artifact: "fp32 · 2.6 GiB · parts",
status: "released",
},
{
id: "ara-diac-small-1.0",
task: "diacritization",
pair: "Arabic → haraqat",
metric: "DER 8.26 full-set (300-para subset had read 3.66)",
artifact: "fp32 · 1.3 GiB",
status: "released",
},
]
---

Expand All @@ -94,12 +110,11 @@ const clientModels = [
Where maps end,<br /><em>models</em> begin.
</h1>
<p class="hero-lead">
The 289 authority-backed maps cover romanization systems that a
committee has published. Some conversions have no committee: restoring
the haraqat a scribe left out, turning unwritten Thai into phonemes,
reading Urdu aloud. Those are learned — and they ship under the same
discipline as the maps: one artifact format, checksums verified on
every load, byte-identical output from every runtime.
The 289 authority-backed maps cover romanization systems that a committee has published.
Some conversions have no committee: restoring the haraqat a scribe left out, turning
unwritten Thai into phonemes, reading Urdu aloud. Those are learned — and they ship under
the same discipline as the maps: one artifact format, checksums verified on every load,
byte-identical output from every runtime.
</p>
</div>
</section>
Expand All @@ -113,12 +128,11 @@ const clientModels = [
<p class="eyebrow">The catalogue</p>
<h2>Every model, with its number.</h2>
<p class="section-deck">
No model is published without a measured metric next to its
teacher's, on the same harness, in the open. Students are distilled
from frozen teachers and gated at a pre-agreed error budget. Each
entry resolves from the models.yaml index; artifacts above GitHub's
2&nbsp;GiB cap ship as sha256-verified parts that the runtimes
reassemble transparently.
No model is published without a measured metric next to its teacher's, on the same
harness, in the open. Students are distilled from frozen teachers and gated at a
pre-agreed error budget. Each entry resolves from the models.yaml index; artifacts above
GitHub's 2&nbsp;GiB cap ship as sha256-verified parts that the runtimes reassemble
transparently.
</p>
</header>

Expand Down Expand Up @@ -162,8 +176,11 @@ const clientModels = [
}
</ul>
<p class="ledger-note">
“Releasing” = passing its parity gate now, entering the release
pipeline. Distillation budgets and per-model provenance: <a href="https://github.com/interscript/interscript-ml/blob/main/docs/RESULTS.md">interscript-ml/docs/RESULTS.md</a>.
“Releasing” = passing its parity gate now, entering the release pipeline. Distillation
budgets and per-model provenance: <a
href="https://github.com/interscript/interscript-ml/blob/main/docs/RESULTS.md"
>interscript-ml/docs/RESULTS.md</a
>.
</p>
</div>
</section>
Expand All @@ -177,12 +194,11 @@ const clientModels = [
<p class="eyebrow">The contract</p>
<h2>One artifact. Any runtime.</h2>
<p class="section-deck">
A model is a zip — the Interscript Model Format, IMF v1. Anything
that can read a zip, hash a file, and run two ONNX sessions can
serve it; no Interscript training code required. Every member is
sha256-verified against metadata.yaml on load; tampering raises
loudly. The tokenizer is raw UTF-8 bytes — no vocabulary to
download, no sentencepiece to drift.
A model is a zip — the Interscript Model Format, IMF v1. Anything that can read a zip,
hash a file, and run two ONNX sessions can serve it; no Interscript training code
required. Every member is sha256-verified against metadata.yaml on load; tampering raises
loudly. The tokenizer is raw UTF-8 bytes — no vocabulary to download, no sentencepiece to
drift.
</p>
<ul class="contract-facts">
<li>Byte tokenizer — token id = byte + 3, trailing EOS</li>
Expand Down Expand Up @@ -211,11 +227,10 @@ const clientModels = [
<p class="eyebrow">The crystals</p>
<h2>Three runtimes, same bytes out.</h2>
<p class="section-deck">
The neural layer is served by <em>secryst</em> — the same
cross-runtime contract as the map layer. Ruby, Python, and
TypeScript resolve a model id against the index, fetch (or reuse
the cache), verify every checksum, and decode with the KV-cache
graph. Golden sets pin the three implementations to each other.
The neural layer is served by <em>secryst</em> — the same cross-runtime contract as the map
layer. Ruby, Python, and TypeScript resolve a model id against the index, fetch (or reuse the
cache), verify every checksum, and decode with the KV-cache graph. Golden sets pin the three
implementations to each other.
</p>
</header>
<div class="crystal-code">
Expand All @@ -230,12 +245,12 @@ npm install secryst</code></pre>
<pre><code>require "secryst"
model = Secryst::Model.load("tha-g2p-small-1.0")
model.translate("สวัสดี")
# => "sa˨˩.wat̚˨˩.diː˧"</code></pre>
# =&gt; "sa˨˩.wat̚˨˩.diː˧"</code></pre>
</div>
<p class="env-note">
Point deployments at a mirror with <code>SECRYST_INDEX</code>; pin the
cache with <code>SECRYST_CACHE</code>. Both are read at call time in
all three runtimes.
Point deployments at a mirror with <code>SECRYST_INDEX</code>; pin the cache with <code
>SECRYST_CACHE</code
>. Both are read at call time in all three runtimes.
</p>
</div>
</section>
Expand All @@ -249,10 +264,9 @@ model.translate("สวัสดี")
<p class="eyebrow eyebrow-accent">Provenance</p>
<h2>Measured, or it doesn't ship.</h2>
<p class="cta-deck">
Teachers are frozen before distillation and never LLM-generated —
language-model teachers hallucinate diacritics. Every number in the
ledger links to a protocol in the open results log. BSD-3-Clause,
code and weights.
Teachers are frozen before distillation and never LLM-generated — language-model teachers
hallucinate diacritics. Every number in the ledger links to a protocol in the open results
log. BSD-3-Clause, code and weights.
</p>
</div>
<div class="cta-code">
Expand All @@ -277,7 +291,6 @@ github.com/secryst # runtimes + training</code></pre>
.page-headline em {
font-style: italic;
color: var(--color-brand);
font-variation-settings: "SOFT" 100, "WONK" 1;
}
.hero-lead {
font-size: var(--text-lead);
Expand Down Expand Up @@ -318,7 +331,7 @@ github.com/secryst # runtimes + training</code></pre>
color: var(--color-stone);
text-transform: none;
letter-spacing: 0;
font-family: var(--font-serif);
font-family: var(--font-sans);
font-style: italic;
}
.ledger {
Expand Down
Loading