Skip to content

fix(vendor): keep ML model registrations in the bundle (sideEffects) #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 3 commits into from
Aug 26, 2026
Merged

fix(vendor): keep ML model registrations in the bundle (sideEffects) #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2118de8
fix(vendor): mark model registration modules side-effectful (upstream…
ronaldtse Aug 26, 2026
42e0725
fix(package): keep npm artifact code-only; maps are deploy assets
ronaldtse Aug 26, 2026
e242ab9
fix(package): publish api-worker as code-only package
ronaldtse Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 21 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ Any organization can deploy this package privately on their own domain.

Field-for-field identical to the long-running Ruby API:

| field | behavior |
|---|---|
| `info` | version JSON |
| `systemCodes` | all 287 transliteration systems |
| `transliterate(systemCode!, input!)` | transliterate (1MB input cap) |
| `detect(input!, output!)` | ranked `[{mapName, distance}]` |
| field | behavior |
| ------------------------------------ | ------------------------------- |
| `info` | version JSON |
| `systemCodes` | all 287 transliteration systems |
| `transliterate(systemCode!, input!)` | transliterate (1MB input cap) |
| `detect(input!, output!)` | ranked `[{mapName, distance}]` |

Error strings match byte-for-byte (`Couldn't locate <code>`).

Expand All @@ -28,16 +28,19 @@ Error strings match byte-for-byte (`Couldn't locate <code>`).
[`interscript-ts`](https://github.com/interscript/interscript-ts) —
no regex compilation.
- **Maps**: the full corpus (287 systems + 4 dependency libraries,
compiled to schema-versioned JSON IR from interscript-maps 2.4.3) is
**bundled with the deploy as static assets** — minified ~260MB,
~21MB gzipped, every file well under the 20MiB asset limit. Immutable
per release, edge-cached, zero runtime third-party fetches.
compiled to schema-versioned JSON IR from interscript-maps) is a
deploy-time asset, not part of this npm package. Deployment repos fetch
a pinned map artifact from `interscript/maps`, verify it, and expose it
through the Workers Assets binding. Immutable per release, edge-cached,
zero runtime third-party fetches.
- **Server**: Hono + GraphQL Yoga.

## Deploy your own

```bash
npm install @interscript/api-worker
# Then fetch/extract a pinned interscript/maps JSON-IR artifact into ./maps.
# Do not copy maps from this npm package; the package is code-only.
```

Minimal `wrangler.jsonc`:
Expand All @@ -48,10 +51,10 @@ Minimal `wrangler.jsonc`:
"main": "src/index.ts",
"compatibility_date": "2026-08-01",
"assets": {
"directory": "./node_modules/@interscript/api-worker/maps",
"binding": "ASSETS"
"directory": "./maps",
"binding": "ASSETS",
},
"routes": [{ "pattern": "api.example.org/*" }]
"routes": [{ "pattern": "api.example.org/*" }],
}
```

Expand All @@ -72,16 +75,19 @@ npm run lint
Regenerating the map corpus (requires interscript-ruby + interscript-maps):

```bash
ruby -e '... see scripts/build-maps.rb ...'
ruby scripts/build-maps.rb /path/to/interscript-ruby /path/to/maps ./maps
```

For production deploys, prefer a pinned release artifact from
`interscript/maps` over local regeneration; verify the artifact checksum
before `wrangler deploy`.

## Parity

`test/fixtures/parity.json` holds queries recorded from the production
Ruby API; CI replays them against this implementation. Zero-diff is the
release gate.


## REST v1 (OpenAPI)

REST endpoints serve alongside GraphQL (which remains at `POST /graphql`):
Expand Down
50 changes: 10 additions & 40 deletions dist/src/generated/models.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,7 @@
{
"id": "khm-latn-1.0",
"task": "translit",
"scripts": [
"Khmr",
"Latn"
],
"scripts": ["Khmr", "Latn"],
"precision": "fp32",
"filename": "khm-latn-1.0-fp32.zip",
"url": "https://github.com/interscript/interscript-ml/releases/download/khm-latn-1.0/khm-latn-1.0-fp32.zip",
Expand Down Expand Up @@ -35,10 +32,7 @@
{
"id": "urd-g2p-1.0",
"task": "g2p",
"scripts": [
"Arab",
"IPA"
],
"scripts": ["Arab", "IPA"],
"precision": "fp32",
"filename": "urd-g2p-1.0-fp32.zip",
"url": "https://github.com/interscript/interscript-ml/releases/download/urd-g2p-1.0/urd-g2p-1.0-fp32.zip",
Expand Down Expand Up @@ -66,10 +60,7 @@
{
"id": "urd-diac-1.0",
"task": "diacritization",
"scripts": [
"Arab",
"Arab"
],
"scripts": ["Arab", "Arab"],
"precision": "fp32",
"filename": "urd-diac-1.0-fp32.zip",
"url": "https://github.com/interscript/interscript-ml/releases/download/urd-diac-1.0/urd-diac-1.0-fp32.zip",
Expand All @@ -92,10 +83,7 @@
{
"id": "heb-diac-1.0",
"task": "diacritization",
"scripts": [
"Hebr",
"Hebr"
],
"scripts": ["Hebr", "Hebr"],
"precision": "fp32",
"filename": "heb-diac-1.0-fp32.zip",
"url": null,
Expand Down Expand Up @@ -134,10 +122,7 @@
{
"id": "tha-g2p-base-1.0",
"task": "g2p",
"scripts": [
"Thai",
"IPA"
],
"scripts": ["Thai", "IPA"],
"precision": "fp32",
"filename": "tha-g2p-base-1.0-fp32.zip",
"url": null,
Expand Down Expand Up @@ -176,10 +161,7 @@
{
"id": "tha-g2p-small-1.0",
"task": "g2p",
"scripts": [
"Thai",
"IPA"
],
"scripts": ["Thai", "IPA"],
"precision": "int8",
"filename": "tha-g2p-small-1.0-int8.zip",
"url": "https://github.com/interscript/interscript-ml/releases/download/tha-g2p-small-1.0/tha-g2p-small-1.0-int8.zip",
Expand Down Expand Up @@ -207,10 +189,7 @@
{
"id": "tha-g2p-small-1.0-int4",
"task": "g2p",
"scripts": [
"Thai",
"IPA"
],
"scripts": ["Thai", "IPA"],
"precision": "int4",
"filename": "tha-g2p-small-1.0-int4.zip",
"url": "https://github.com/interscript/interscript-ml/releases/download/tha-g2p-small-1.0/tha-g2p-small-1.0-int4.zip",
Expand Down Expand Up @@ -238,10 +217,7 @@
{
"id": "heb-diac-small-1.0",
"task": "diacritization",
"scripts": [
"Hebr",
"Hebr"
],
"scripts": ["Hebr", "Hebr"],
"precision": "fp32",
"filename": "heb-diac-small-1.0-fp32.zip",
"url": "https://github.com/interscript/interscript-ml/releases/download/heb-diac-small-1.0/heb-diac-small-1.0-fp32.zip",
Expand Down Expand Up @@ -269,10 +245,7 @@
{
"id": "fas-g2p-1.0",
"task": "g2p",
"scripts": [
"Arab",
"Latn"
],
"scripts": ["Arab", "Latn"],
"precision": "fp32",
"filename": "fas-g2p-1.0-fp32.zip",
"url": null,
Expand Down Expand Up @@ -311,10 +284,7 @@
{
"id": "heb-diac-1.1",
"task": "diacritization",
"scripts": [
"Hebr",
"Hebr"
],
"scripts": ["Hebr", "Hebr"],
"precision": "fp32",
"filename": "heb.zip",
"url": null,
Expand Down
Loading