Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,17 @@ jobs:
- run: uv sync --locked
- run: uv run pytest --cov --cov-report=term-missing

provider-compatibility:
name: test (provider SDKs)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: astral-sh/setup-uv@v6
with:
python-version: "3.10"
- run: uv sync --locked --extra openai --extra anthropic
- run: uv run pytest tests/test_providers.py tests/test_provider_compatibility.py

lowest-dependencies:
name: test (lowest dependencies)
runs-on: ubuntu-latest
Expand All @@ -77,7 +88,7 @@ jobs:
test:
name: test
if: always()
needs: [tests, lowest-dependencies]
needs: [tests, lowest-dependencies, provider-compatibility]
runs-on: ubuntu-latest
steps:
- name: Every test job passed
Expand Down
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,29 @@ public API, a minor version adds to it, and a patch version fixes it.

### Added

- Regenerated the AI credits contract, including write-only `ReportUsageValue`,
typed usage outcomes/gauges and model/conversion-rule events. Original AI
response bridge names now alias generated models.
- Model catalogue CRUD, license conversion-rule management, paged inference
history and open reservation reads on both clients.

- Optional OpenAI and Anthropic drop-in clients and existing-client wrappers, with
allow-listed reserve/call/commit accounting, sync/async streaming settlement,
pluggable estimates and provider timeout-based reservation lifetimes.
- Documented current agent SDK accounting interfaces and the missing supported
per-model-inference gating boundary; no misleading run-budget wrappers.

- `feature_flags.targeting_context()`, `feature_flags.lint_rule()` and `feature_flags.test_rule()`:
the facts a targeting rule can read, the check the API runs before accepting one, and a dry run
of a rule against a customer or an instance.
- `feature_flags.update(..., slug=...)` renames a flag.

### Changed

- Unreleased AI helpers, meters and provider wrappers select the AI credit entitlement
with `entitlement=` rather than `pool=`; the old keyword is not accepted.
Low-level instance methods still use `entitlement_slug`, and HTTP routes are unchanged.

### Fixed

- `feature_flags.update` sends the flag's slug. Without it, Kaiten blanked the slug and the flag
Expand Down
8 changes: 8 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,3 +86,11 @@ the async source, the contract or a generator, then run `task generate`.
Do not report unpatched vulnerabilities in public issues.

See [SECURITY.md](./SECURITY.md).

## Optional AI provider compatibility

Provider packages are optional extras, not core dependencies. Their supported
major ranges are declared in `pyproject.toml`; CI checks synthetic usage and
in-process mock transports, never real provider calls. See
[`internal-docs/AI_WRAPPERS.md`](internal-docs/AI_WRAPPERS.md) for the bounded
surface and verified agent SDK constraints.
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -835,3 +835,26 @@ By contributing, you agree to certify your contribution under the

The Kaiten name and logos are not licensed under Apache-2.0.
See the Kaiten trademark policy in the main Kaiten repository.

### AI credit provider wrappers

Use `from kaitencloud.openai import OpenAI` or
`from kaitencloud.anthropic import Anthropic`, then pass
`kaiten=client, instance="acme-prod", entitlement="chat-credits"` alongside your normal
provider options. Async drop-in classes and `wrap_openai` / `wrap_anthropic`
for existing clients are available. Install the corresponding `openai` or
`anthropic` extra. Every allow-listed call reserves before execution and settles
with normalized usage; pricing stays on the server.

See [AI wrapper behavior and limits](internal-docs/AI_WRAPPERS.md), especially
stream closure, missing audio usage and agent SDK gating constraints.

AI management uses `client.models` (catalogue list/get/create/update/delete) and
`client.licenses.list_conversion_rules`, `create_conversion_rule`,
`get_conversion_rule`, `update_conversion_rule`, `archive_conversion_rule`.
Pass `ConversionRates` and `VendorCosts` from `kaitencloud.types`; decimal rates
are strings, and no pricing calculation runs in the SDK. Inference history is
one page at a time via `instances.list_ai_inferences(..., cursor=...)`; its
`next_cursor` continues the history. `instances.list_ai_reservations` returns
open reservations. Usage gauges now expose generated `reserved`/`ai` fields.
The vendored contract's exact revision is recorded in `openapi/source.yaml`.
117 changes: 117 additions & 0 deletions internal-docs/AI_WRAPPERS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# AI credit wrappers

The server prices every call. The SDK sends only model, request type and usage;
no prompts, responses or vendor prices are sent to Kaiten.

AI pools are `NUMBER_AI_CREDIT` entitlements with numeric `SUM` usage. Grants
and usage reads still use the `number` value shape; `ai_usage` is only the
reserve/commit calculation input. Credit adjustments/refunds use numeric
`report_usage(..., behavior="append")`, never `set`.

Reserve and commit share `value.transactionId`; retries repeat the identical
phase/body. The server rejects `Idempotency-Key` on AI reports, so these phases
omit it case-insensitively, including SDK and supplied HTTP-client defaults;
numeric reports keep their configured headers. Decimal units accept integers, strings or `Decimal`
(12 integer digits and at most 6 decimal places), not binary floats. Conversion
rates and credit outcomes remain decimal strings. An omitted cache rate is
unknown policy, not an explicit zero/free rate; leave it omitted unless a rate
is known, and let the server determine pricing and warnings.

## Provider clients

Install `kaitencloud[openai]` (OpenAI 3.x, >=3.35.1) or
`kaitencloud[anthropic]` (Anthropic 1.x, >=1.15.0).

```python
from kaitencloud.openai import OpenAI

provider = OpenAI(
api_key="your-provider-key", kaiten=kaiten, instance="acme-prod", entitlement="chat-credits"
)
reply = provider.chat.completions.create(model="your-model", messages=messages)
```

`AsyncOpenAI`, `Anthropic` and `AsyncAnthropic` use the same keyword options.
Provider and Kaiten clients must both be sync or both be async. An existing
client can be delegated with `wrap_openai(client, kaiten=..., instance=..., entitlement=...)`
or `wrap_anthropic(...)`. The wrapper does not own a supplied client's lifetime;
`close()` and context-manager exit delegate to that client. Copies made with
`copy()` / `with_options()` retain the meter.

Every metered call requires an explicit model. `estimator=` receives an
`AiRequest` and returns an optional usage estimate; by default serialized prompt
bytes/4 and the declared output ceiling (otherwise 1024 for completions) are
reserved. Reservation TTL follows the request/client timeout when available.

### Bounded surface

OpenAI: `chat.completions.create`, `responses.create`, `embeddings.create`,
`images.generate`, `audio.transcriptions.create`, `audio.speech.create`.
Anthropic: `messages.create`, `messages.stream`.

Unlisted methods/resources stay provider-owned, including raw/streaming HTTP
response helpers, parsing helpers, batches, realtime and tool runners. They are
not metered: use generic `kaitencloud.ai.run` when you need explicit accounting.

A refused reserve raises its typed error before the provider is called. An
unsupported server is logged once per meter and subsequent calls pass through
unreported. Settlement is best effort under the original transaction ID; a
failed commit is logged, never substituted for the provider result.

### Streams

Consume streams fully or close them explicitly; context managers are strongly
recommended. A bare `break` on a retained stream does not close it. Garbage
collection cannot reliably perform asynchronous settlement.

Chat streaming merges `stream_options.include_usage=True`. Responses uses the
terminal response usage without adding chat-only parameters. Anthropic starts
with `message_start` usage and overwrites cumulative fields from `message_delta`
(never sums these snapshots). Its `messages.stream()` context-manager factory,
`text_stream`, `until_done` and final-message helpers are preserved.

Settlement occurs once on completion, iteration failure or close. An abandoned
stream with received events reports `CUSTOMER_ERROR` and known usage; a stream
closed before receiving anything reports `NOT_SENT`, following the V1 wrapper
policy. Network/protocol failures report `PROVIDER_ERROR` unless the failure is
proven to precede sending. Image partial frames are not additional images;
completed images and actual returned image count supply units. Speech binary
responses and transcription text formats may expose no usage; these settle
without usage and the server applies its minimum/warning policy. Never infer
billable duration from compressed audio bytes.

Compatibility tests use actual optional SDKs and in-process mock transports,
never provider endpoints or credentials.

## Agent SDK constraints (survey 2026-10-19)

Do not mistake tool hooks or user-facing runs for model-inference boundaries.
No fake agent wrapper is shipped until it can satisfy the per-model-turn gate.

* Claude Agent (`claude-agent-sdk` 0.2.168, Python >=3.10):
`AssistantMessage` exposes `model`, `usage`, `message_id` and subagent context;
partial `StreamEvent` exposes raw Anthropic events. `ResultMessage` usage is
aggregate. Public `HookEvent` includes tool/user/stop/subagent hooks but no
before-every-model-call hook. A `PreToolUse` callback runs after inference.
[Types](https://github.com/anthropics/claude-agent-sdk-python/blob/main/src/claude_agent_sdk/types.py).
* Codex (`openai-codex` 0.163.0, Python >=3.10): `Thread.run` / `turn` and sync/async
handles are available. `thread/tokenUsage/updated` carries `thread_id`,
`turn_id`, `token_usage.last` and `total`, but no model field. One user-facing
turn may include multiple model calls. No awaited per-model-call veto hook is
established by the public API.
[API](https://github.com/openai/codex/blob/main/sdk/python/docs/api-reference.md),
[usage types](https://github.com/openai/codex/blob/main/sdk/python/src/openai_codex/generated/v2_all.py).
* Copilot (`github-copilot-sdk` 1.0.22, Python >=3.11): `AssistantUsageData` exposes
model, input/output/cache/reasoning counts; turn events and `agent_id` support
attribution. `session.on(callback)` is synchronous; dispatcher exceptions are
logged and do not stop inference. Session hooks expose no awaited per-model
gate. Cache inclusion semantics need verification before normalizing usage.
Runtime startup may auto-download binaries, so compatibility tests must not
start real sessions.
[Events](https://github.com/github/copilot-sdk/blob/main/python/copilot/generated/session_events.py),
[callbacks/hooks](https://github.com/github/copilot-sdk/blob/main/python/copilot/session.py).

Post-call accounting adapters are possible, but cannot honestly promise the
required preflight for each internal model turn or subagent. Integration needs
a supported veto-capable model boundary or an explicitly approved scope change;
run-level budgets, private monkeypatches and hosted proxies are not substitutes.
20 changes: 19 additions & 1 deletion openapi/coverage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,19 @@

core:
operations:
listModels: models.list
createModel: models.create
getModel: models.get
updateModel: models.update
deleteModel: models.delete
listConversionRules: licenses.list_conversion_rules
createConversionRule: licenses.create_conversion_rule
getConversionRule: licenses.get_conversion_rule
updateConversionRule: licenses.update_conversion_rule
archiveConversionRule: licenses.archive_conversion_rule
listAiInferences: instances.list_ai_inferences
listAiReservations: instances.list_ai_reservations

# components
list-components: components.list
get-component: components.get
Expand Down Expand Up @@ -86,7 +99,12 @@ core:
getAuditTrails: instances.list_audit_trails
getEntitlementsUsageMetrics: instances.list_usage
getEntitlementUsageMetrics: instances.get_usage
reportEntitlementUsageMetric: instances.report_usage
# One operation, two report shapes: a number, or one phase of an AI call (kaitencloud.ai
# reserves, runs and commits through the last two).
reportEntitlementUsageMetric:
- instances.report_usage
- instances.reserve_ai_usage
- instances.commit_ai_usage
get-instance-integration: instances.get_integration
create-instance-integration: instances.create_integration
update-instance-integration: instances.update_integration
Expand Down
Loading
Loading