Open-source LLM migration and regression testing for AI agents. Compare models, detect tool-call regressions, and gate model changes in CI.
-
Updated
Nov 17, 2026 - Python
Open-source LLM migration and regression testing for AI agents. Compare models, detect tool-call regressions, and gate model changes in CI.
Source code for the paper "Accelerating Hierarchical Federated Learning under Mobility via Model Migration in Cloud–Edge–End Collaborative Networks".
Agent skill that upgrades LLM code to Claude Opus 5.5, GPT-6, Gemini 3.8 without breaking prod. Fixes breaking changes, then proves it with tests and evals.
Detect behavioural drift between LLM versions before you upgrade. Compare model responses, classify regressions, and generate migration reports with validated prompt patches.
Check LLM responses against the output rules your application depends on, before changing a model, prompt, schema or provider. Also flags retired model IDs in your code. Local CLI, GitHub Action and MCP server; no third-party runtime dependencies.
Early-stop canary testing for LLM model migrations
A portable memory skill and CLI for AI agents: curated context, user preferences, topic interruptions, handoff packets, and model/runtime migration without transcript dumps.
Snapshot tests for AI model upgrades: see which tool-calling behaviors consistently change before you switch LLMs. No expected answers to write.
Open-source capture SDK for AI agents — records model and tool calls to build golden suites for LLM migration and regression testing.
Why a passing benchmark isn't safe to ship: a free 2-stage (benchmark + replay) validation recipe for LLM model swaps & prompt changes, run on flat-rate coding-agent subagents — no eval API bill.
Plan, research, and evaluate LLM application migrations between models, providers, and platforms (Claude, GPT, Bedrock) before changing code
Pin the behavior, not the model. For teams shipping LLM apps: before you switch models, replay your app's real requests on the new one and see what actually changed. CLI + GitHub Action. Open-core, Apache-2.0.
Check whether your prompts, tool calls and JSON outputs still work on a replacement LLM before you switch. Go / no-go verdict, BYOK, no telemetry.
Prove a model swap is safe before shipping it: golden set, field-level parity, per-class floors and a cost model. Python, 82 tests.
Switch AI models without breaking your product: find every model your code calls and when it retires, turn real usage logs into tests, compare and pick replacement models. CLI, GitHub Action and MCP server.
Decide in hours whether to migrate/deprecate an LLM: golden set, old-vs-new comparison, go/no-go, rollback (synthetic data)
Is a cheaper model as good as the one you run now? Paired-bootstrap non-inferiority check for swapping models.
Ollama model store porter: re-hash every blob, export/import models offline as verified tarballs
Shadow deployment for moving a production LLM workflow onto a new model or agent harness. The new model shadows live incidents, and a sequential test (SPRT) on incidents held out of tuning says promote, continue or reject. Ops agreed on 52 of 62, so it keeps shadowing. Render was rejected. Python, Ollama, Wilson intervals, PII redaction.
A minimal eval harness for LLM model migrations: golden set with trap cases, LLM-as-judge, A/B on quality, latency and cost.
To associate your repository with the model-migration topic, visit your repo's landing page and select "manage topics."