feat(inference): add GLiNER2.5 inference/training and speed up GLiNER2

[timkaye11]

Implement native Zig CPU and Metal support for GLiNER2.5 small, base, and multilingual checkpoints, including extraction APIs/SDKs, training, checkpoint/resume, and export. Shared execution changes also improve legacy GLiNER2 performance and batched contextual-label handling. Native inference and training do not require Python; pinned Fastino/PyTorch provides reference results.

Public GLiNER2.5 serving remains gated by the empty production qualification table. Larger numerical fixtures are omitted pending separate object-storage integration, reducing default CI coverage.

Inference and API

• Add strict boundary-architecture detection, DeBERTa encoding, boundary proposals, shared candidates, explicit-span scoring, and learned task heads.
• Support entities/attributes, relations, ordinary/constrained/ordinal classification, legacy structures, natural/latent/anchorless records, enums, and JointIE. Bound graph selection and distinguish infeasible, exhausted, and valid best-effort results.
• Extend POST /ai/v1/extract, inference-service POST /extract, and embedded extraction with schema_version: 2; retain version 1 when omitted. Add Go ExtractV2, Python extract_v2, TypeScript extractV2, and generated OpenAPI clients.
• Support mixed tasks, per-input schema/options replacement, explicit field-presence semantics, and original-text offsets in UTF-8 bytes, Unicode codepoints, or UTF-16 units. Add bounded regex/enum validation and opt-in long-document windowing/global merging; oversized inputs reject by default.

Artifacts and training

• Validate complete 334-tensor bundles and four configuration/tokenizer sidecars. Convert FP32, FP16 encoder, and Q8_0 for all variants; Q4_0 for small and Q4_K for base/multilingual. Heads, biases, normalization, and relative-position tables retain FP32. Publish verified artifacts atomically.
• Implement full, head-only, LoRA, and DoRA training on CPU/resident Metal, with explicit-schema JSONL preflight, task losses/gradients, proposal selection, matching, optimizer groups, accumulation, and absent-versus-zero gradient semantics.
• Bind checkpoint/resume to source, dataset/schema, parameters, schedules, RNG, optimizer state, and counters. Bound memory and replay work; add tiled attention, layer recomputation, dropout-mask replay, FP32/adapter export, and adapter materialization. A pinned PEFT export profile handles upstream inside_weight compatibility.

Quantized training and automatic quality evaluation/calibration/model selection are unsupported.

Shared runtime

• Optimize CPU attention/GELU/QKV handling and resident Metal weights, embedding uploads, and head operations. Correct legacy GLiNER2 per-sample contextual labels and graph-selected batch execution.
• Fix WebGPU activation freshness, caching, and transfer cleanup. Harden admission, cancellation/deadlines, atomic batch errors, shader readiness, model teardown, and allocation-failure recovery.

Performance against Fastino Python (CPU and MPS)

These are historical measurements from frozen builds; the complete timing campaigns have not been rerun on the final PR tree.

GLiNER2.5: original FP32 corpus

Representative mixed-task extraction, with latency medians across three fresh-process repetitions:

GLiNER2.5 model	Antfly vs Fastino backend	Fastino Python	Antfly Zig	Paired speedup
Small	CPU vs CPU	18.85 ms	14.42 ms	1.31×
Small	Metal vs MPS	69.46 ms	19.06 ms	3.58×
Base	CPU vs CPU	45.30 ms	35.51 ms	1.28×
Base	Metal vs MPS	72.15 ms	21.27 ms	3.26×
Multilingual	CPU vs CPU	49.02 ms	40.43 ms	1.22×
Multilingual	Metal vs MPS	73.42 ms	23.09 ms	3.08×

Paired speedups aggregate individual timing ratios and can differ from the ratio of displayed latency medians. CPU and GPU rows come from separate matched campaigns.

Base-model mixed extraction measured 20.61–21.68 ms on Metal versus 71.56–72.49 ms on MPS. Timing includes schema compilation, preprocessing, encoder, heads, decoding, and temporary cleanup; loading and protocol I/O are excluded. MPS synchronizes both ends. Intervals are per repetition; an independent audit recomputed 2,700 Metal timing pairs.

Legacy GLiNER2: warm entity extraction

Workload	Fastino Python CPU	Antfly Zig CPU	Speedup vs Fastino
36 tokens / B1	42.85 ms	30.48 ms	1.41×
256 tokens / B1	179.27 ms	140.36 ms	1.28×
256 tokens / B8	1453.24 ms	1022.27 ms	1.42×

Workload	Fastino Python MPS FP32	Antfly Metal default	Speedup vs Fastino	Antfly Metal FP32	Speedup vs Fastino
36 tokens / B1	29.23 ms	30.14 ms	0.97×	33.88 ms	0.86×
256 tokens / B1	78.89 ms	72.50 ms	1.09×	79.42 ms	0.99×
256 tokens / B8, supplemental GPU-only run	577.17 ms	529.99 ms	1.09×	534.67 ms	1.08×

Speedup above 1 favors Antfly; below 1 favors Fastino. Default Metal uses F16 encoder mirrors with FP32 head/activations; explicit FP32 keeps FP32 weights without F16 or quantized execution. Default Metal is approximately 3% slower on the short case. In the supplemental B8 run, default/FP32 Metal have 8.2%/7.4% lower latency than MPS.

Values are medians of two fresh-process warm medians, with reversed arm order, three warmups, and nine timed requests. The B8 GPU row comes from a separate GPU-only repeat with ten seconds idle before each process: primary MPS B8 medians varied from 573.38 to 874.47 ms, so their aggregate is not used as a stable speedup estimate. Timing includes preparation, inference, and extraction; loading, startup, and result destruction are excluded. MPS synchronizes both ends. These local comparisons do not establish serving-tail latency or optimal thread counts.

Validation

Latest fixture-policy validation used the repository Zig build graph with -Dmetal=false -Dcuda=false -j1, testing both omitted and temporarily restored payloads.

Check	Normal checkout: large fixtures omitted	Original pinned payloads temporarily restored
Python CI contract entrypoint	285 tests run; 12 reported skips; no failures	285 passed; no skips
Focused native CPU tests	33 selected: 14 passed, 19 skipped	33 selected: 31 passed, 2 model-dependent skips
Fixture integrity	Required local files verified; 27 external files reported missing	Every external size and SHA-256 verified; strict completeness check passed

Python skip totals include subtests. Negative probes rejected same-size external corruption and missing required fixtures without skipping. fixture_support.py --require-all rejects incomplete restoration. Runtime functionality, assertions, tolerances, and surviving capture values are unchanged; the final branch omits all 27 payloads.

[ajroetker]

Where did we use this?

[timkaye11]

we use it for Gliner2.5 finetuning purposes when solving the Assignment Problem. Astra implemented Scipy's Rectangular Linear Sum Assignment Problem (LSAP) (which is used by the python versions) in zig, hence the license.

[ajroetker]

Should these be more general than "extraction" seems like they're InclusiveProbability/ExclusionProbabability or something?
Also we should visit this as an API and make sure all these options make the most sense? Maybe they should be model options instead of extraction options?

[ajroetker]

[codex] Extraction API and UX review

Reviewed commit f570289943c25d744df6a3da0e4b6962b89fa0a8, focusing on extraction APIs, SDKs, model selection, and playground behavior.

Recommendation: address the playground issues before presenting this as a usable GLiNER2.5 feature. The versioned API is promising, but public GLiNER2.5 serving remains deliberately gated off. The UI should reflect that distinction.

Issues

1. [P2] Searching can overwrite—and then clear—the selected model. Both playgrounds now enable allowCustomValue. The shared Combobox writes every search keystroke into selectedModel; clicking an option whose value exactly matches the typed text then clears the selection. Reproduced using the actual handlers. Keep search text separate from the committed model value, and commit only on explicit selection or custom-value confirmation.

   • Extraction playground integration
   • Combobox handlers

2. [P2] The playground offers a GLiNER2.5 path that cannot serve requests. Users can manually enter a GLiNER2.5 model ID and run extraction, but the production qualification table is empty. For an installed boundary model, the serving gate rejects execution with a generic unsupported-feature error. The gate is intentional; presenting this path as usable is the UX issue. Show an explicit “Not yet available” state and disable execution until the backend advertises support. Do not bypass the qualification gate to fix the UI.

   • Current UI message
   • Serving gate

Model/provider UX recommendations

Pioneer is a provider; GLiNER2.5 is a model family. Keep these choices separate:

Choice	Suggested UX
Connection	Local Antfly or Pioneer
Model	Available base or fine-tuned models
Task	Entities, classification, records, relations
Advanced settings	Constraints, attributes, windowing, mixed tasks

• Drive protocol selection and task availability from capabilities, not model-name substrings. isGliner25Model only recognizes IDs containing gliner2.5. A custom name such as invoice-extractor takes the v1 path regardless of its architecture. Preserve the model's capability metadata in the picker. Current helper
• Expose the useful v2 features progressively. The extraction playground still offers entities and basic string/list structures. Add task presets or an advanced schema editor for classification, attributes, enums, richer records, and mixed tasks. Include copyable request examples.
• Use user-facing explanations. Replace “strict schema v2 contract” with supported tasks and limitations; keep protocol details in exported API examples. Explain unavailable models and errors with an actionable next step.
• Make relation endpoint types editable. The new graph request permits every selected entity type at both endpoints. Let users express person → works_at → organization rather than implicitly allowing every combination.

Pioneer integration gap — predates this PR

Pioneer's documented native API uses model_id, text, and X-API-Key. The existing Pioneer adapter sends an Antfly-shaped request and Bearer authentication. This mismatch predates this PR and is not a newly introduced regression, but provider interchangeability should not be claimed without a provider-specific adapter and contract tests covering authentication, request translation, and response normalization.

• Pioneer quickstart
• Existing adapter

Validation and review limits

• Passed 37 focused Zig tests and 45 Python extraction-v2 SDK tests.
• Go extraction-v2 tests passed.
• Checked-in configuration and reference fixture verification passed; those checks explicitly do not establish real-model/runtime qualification.
• Listed CI checks were green at review time.
• UX assessment used source inspection and a handler probe; the browser connection failed, so there was no live visual pass.
• This was not an exhaustive audit of the 135k added lines, training implementation, or numerical/performance claims.
• No tracked files were changed by this review.

[timkaye11]

/ci run 22a46e7

[ajroetker]

/ci run 22a46e7

[timkaye11]

/ci run 58de571