Core AI Models — community fork (-zoo)

This is an unofficial community fork of apple/coreai-models. It is not affiliated with, endorsed by, or supported by Apple Inc. "Core AI" and "Apple" are trademarks of Apple Inc.; this fork uses them only to identify the upstream project, not to endorse or promote this fork (BSD 3-Clause §3). For the official, supported project go to apple/coreai-models.

What this branch is. Upstream main through #207 (2026-08-28) plus the zoo patches, on branch zoo-0.4 / tag 0.2.8-zoo. Everything not listed below is byte-for-byte upstream.

• Community model ports under python/src/coreai_models/models/ — files marked Community port — NOT an Apple model (ternary/1-bit, speech, OCR, MoE, MLA and hybrid SSM decoders). Bundles built from them are published at huggingface.co/mlboydaisuke.
• Pipelined engine: loads decode-only S=1 bundles (logitsSeqIsStatic guard, reported upstream as apple#212), discovers a static-chunk prefill function in multifunction bundles, supports per-token inputs, and caps iOS dynamic-KV capacity at 1024 (guard for apple#124).
• Sequential engine: checkpoint / restore of a hybrid model's recurrent state (InferenceEngine.checkpoint()). CoreAIKit's typed decisions checkpoint after a state's prompt prefix, and each later question on that state restores it instead of replaying the prompt from the first token: 4.4–5.9× per decision on three hybrid decision models on an M4 Max, every answer bit-identical to a full replay. Only the fixed-shape recurrent states are copied (10 MB, under 1 ms), not the KV cache.
• Export: build_macos_export_spec / export_core() shims for the hybrid ports on top of upstream's export contract, plus multifunction export.

Tags. Use 0.2.5-zoo or later; in a macOS app, or if your package declares a floor below 27, 0.2.8-zoo or later. 0.2.8-zoo builds for x86_64 macOS: since 0.2.6-zoo lowered the floor to macOS 26, a Release (universal) build of a macOS app compiles this package for x86_64 too, where Float16 is unavailable, and three upstream files had no x86_64 guard (RepetitionPenaltyGPUState, CoreAISequentialVLMEngine, StateHandler+NDArray), so the build failed. They now take the guard upstream's other Float16 code uses, #if !((os(macOS) || targetEnvironment(macCatalyst)) && arch(x86_64)); the arm64 code is unchanged (CoreAIKit's decider-0.8b fixture reads bit-identical to 0.2.7-zoo, 44 of 44 rows). 0.2.7-zoo adds the sequential engine's checkpoint above (checkpoint(), discardCheckpoint() and supportsCheckpoint on InferenceEngine, no-ops by default): a reset(to:) at or past the checkpoint, or a prompt that keeps its prefix, restores the recurrent state instead of a full reset and replay. Per decision, eight questions on one state, checkpointed vs every prompt from scratch: 186 vs 825 ms (decider-0.8b), 120 vs 696 ms (openthai-systemone), 220 vs 1,297 ms (qwen3.5-2b-decision) on an M4 Max, 2026-09-24; iOS not yet measured. The pipelined engine is unchanged. 0.2.6-zoo declares a macOS 26 / iOS 26 floor with @available(macOS 27, iOS 27, *) on everything that touches Core AI, so packages with a lower floor can depend on it (see Requirements); nothing runs below 27. 0.2.5-zoo stops the engine at a stop sequence instead of draining it to maxTokens (VanillaDecodingStrategy): before it, every chat turn on the pipelined engine kept decoding for the whole remaining response budget after EOS — a 14-token LFM2.5 1.2B answer with a 2048 cap took 8.3 s to complete on an M4 Max (0.4 s with a 64 cap), 0.2 s after the fix at any cap; on a phone that was tens of seconds of "generating" after the answer until the user cancelled. 0.2.4-zoo drops the fork's sampler drain, which upstream #121 made redundant — decode +43–52% on macOS (qwen3-0.6b, measured drain vs. none; iOS not yet measured). 0.2.2-zoo and earlier predate upstream #121 (the pipelined sampling fix) and produce corrupted text at temperature > 0.

---

Core AI Models

Model export recipes, Python primitives, and Swift runtime utilities for building on-device AI with Core AI.

The main components include:

• Model export — Recipes to export popular open source models from Hugging Face and other sources to Core AI format.
• Reusable primitives — Python building blocks for authoring custom Core AI models in PyTorch.
• Runtime utilities — Swift package built on top of Core AI framework to run models on macOS and iOS.
• Skills — Plugins to help coding agents leverage Core AI effectively.

Directory	What's inside
models/	Model catalog with README and export recipes.
python/	Python primitives for authoring and utilities for exporting models.
swift/	Swift package (coreai-models): runtime utilities to integrate Core AI models in your app.
skills/	Pluggable skills that enable coding agents to leverage Core AI more effectively.

Requirements

If you haven't installed uv, install it by

brew install uv

or

curl -LsSf https://astral.sh/uv/install.sh | sh

Once installed successfully, refer to the README.md for each model or family of models, in models folder for their exporting recipe.

Requirements (running and app integration)

• macOS and iOS 27.0+

• Xcode 27.0+

The Swift package declares a macOS 26 / iOS 26 floor so that packages with a lower floor can depend on it. Every API that touches Core AI is marked @available(macOS 27, iOS 27, *); a caller with a lower floor guards it with #available, and the Core AI framework is weak-linked automatically. Nothing runs below 27.

Core AI models are exported as standalone .aimodel files for integration into apps via the Core AI framework.

Some models require additional resources. Language models require a tokenizer, for instance, and diffusion models run multiple models in sequence as part of a single pipeline. For these cases, export recipes in this repo produce a resource folder containing one or more .aimodel files alongside any required resources. The Swift package in this repo provides runtime utilities for integrating these into an app.

Command line interface (CLI) tools are also included for running exported models directly on a Mac (requires Xcode 27.0+). See each model's README for available tools and example invocations.

Explore supported models

Find supported models by

git clone https://github.com/apple/coreai-models.git && cd coreai-models
uv run coreai.model.registry --list-models

Run uv run coreai.model.registry --help for details.

Agent Skills

This repo includes a plugin with skills to enable coding agents to use Core AI like an expert.

Available skills

Skill	Description
working‑with‑coreai	End-to-end workflow for deploying PyTorch models on Apple silicon, covering export with coreai-torch and running with the Core AI runtime.
model‑authoring	Empirical rules for authoring PyTorch models for on-device execution on Apple platforms, covering BC1S layout, op compatibility, KV cache patterns, precision rules, MoE, and common issues.
model‑compression‑exploration	Systematically explore weight compression configurations (quantization and palettization) for a PyTorch model using coreai-opt.

Install

Installation differs depending on your coding agent of choice.

Claude Code

Register the marketplace:

/plugin marketplace add git@github.com:apple/coreai-models.git

Alternatively, register the marketplace from a local git checkout:

/plugin marketplace add /path/to/coreai-models

Install the plugin:

/plugin install coreai-skills@coreai-models

Codex CLI

Register the marketplace:

codex plugin marketplace add https://github.com/apple/coreai-models

Alternatively, register the marketplace from a local git checkout:

codex plugin marketplace add /path/to/coreai-models

Launch Codex in your workspace:

codex

Install the plugin through the interactive browser: once the Codex session is active in your terminal, open the plugin manager by typing /plugins, locate the coreai-models marketplace tab (use your arrow keys or the built-in search), select coreai-skills, and choose Install.

Gemini CLI

Install the extension from a local directory:

gemini extensions install /path/to/coreai-models/skills

Once installed, the skills activate automatically based on your task context, or you can invoke them explicitly.

Contributing

We are not accepting code contributions at this time

Core AI Models is focused on maintaining a curated, well-tested gallery of models and a reliable Swift package. We are not accepting pull requests at launch while we learn how the community uses this project.

If you open a pull request, it will be closed. This is not a reflection of the quality of your contribution but it is a deliberate scope decision for this release.

What we do welcome

We actively want your feedback! GitHub Issues are open for:

• Bug reports — if something in the Python scripts or Swift utilities does not work as expected
• Model requests — if you have ideas for models you would like to see, or improvements to the workflow or Swift utilities

Use the issue templates to get started.

Support

• GitHub Issues — Feedback, bug reports, and feature requests

License

This project is licensed under the BSD 3-Clause License.