Repository navigation
Add Liquid AI backend with per-model prices; release 0.9.0 - #24
Merged
Merged
Conversation
Add a liquid backend for Liquid AI's decision models (default model d1:free), let a registry backend name a price per model, accept usdPerMTok 0, and record each request's cost at its own price so the daily spend and the usdPerDay cap stay right across models and backends.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
liquid, a registry backend for Liquid AI's decision models, which serve the same decisions protocol.createTypeSafe({ backend: "liquid" })sends judgments toapi.liquid.aiunder/decisions/v1/systemonewith the key fromLIQUID_API_KEY. The model list is read from/decisions/v1/modelsand checks the key, solistModels()verifies it.d1:free, which is free.model: "d1"selects the paid model.prices):d1at $0.040 andd1:freeat $0 per million input tokens. A client without a callerusdPerMTokprices each request at its backend's price for the model it sends, andusdPerMTok: 0is accepted.usdPerDaycap stay right when one ledger holds requests to models with different prices. Free requests add nothing. Ledger files written before this change are read and estimated as before.d1:freetook a median of about 14 seconds per request when requests were sent one at a time, so callers raisetimeoutMsor used1.The last commit bumps the version to 0.9.0 and moves the changelog notes under
## 0.9.0.Verification:
npm run check: 147 tests, all pass.d1andd1:free, and oneevaluateond1:freewith a noul, a choice, and a score question returned all three answers.