Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 85 additions & 19 deletions website/src/app/(site)/action/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -102,19 +102,31 @@ export default function ActionPage() {
<div className="mt-8 flex flex-wrap items-center gap-6 text-xs text-(--muted)">
<span className="flex items-center gap-2">
<span className="w-1.5 h-1.5 rounded-full bg-emerald-400" />
<span className="text-(--fg) font-semibold">30-Second Setup</span>
<span className="text-(--fg) font-semibold">30-Second Setup with agentdiff init</span>
</span>
<span className="text-(--border) select-none">•</span>
<span className="flex items-center gap-2">
<span className="w-1.5 h-1.5 rounded-full bg-emerald-400" />
<span>Automated PR Comment Diagnostic</span>
<span>Interactive /agentdiff approve Bot</span>
</span>
<span className="text-(--border) select-none">•</span>
<span className="flex items-center gap-2">
<span className="w-1.5 h-1.5 rounded-full bg-emerald-400" />
<span>Goodhart Threshold Protection</span>
<span>Hosted Identity (agentdiff[bot])</span>
</span>
</div>

<div className="mt-8 flex flex-wrap items-center gap-4">
<a
href="https://github.com/apps/agentdiff-ci"
target="_blank"
rel="noopener noreferrer"
className="px-6 py-3 rounded-full bg-(--fg) text-(--bg) text-xs font-semibold hover:opacity-90 transition-opacity inline-flex items-center gap-2 shadow-sm"
>
<span>Install GitHub App</span>
<ArrowRight className="w-3.5 h-3.5" />
</a>
</div>
</div>
</Reveal>
</div>
Expand All @@ -134,17 +146,17 @@ export default function ActionPage() {
Drop into any repo in 30 seconds.
</h2>
<p className="text-base text-(--muted) leading-relaxed font-normal">
Execute your agent test script in CI, record the candidate trace, and invoke <code className="text-xs font-mono text-(--fg) bg-(--surface-2) px-1.5 py-0.5 rounded border border-(--border)">agentdiff-check</code>. If divergence, cost, or retry loop thresholds breach tolerance, the action fails with exit code 1.
Generate your gate in one command with <code className="text-xs font-mono text-(--fg) bg-(--surface-2) px-1.5 py-0.5 rounded border border-(--border)">agentdiff init --with-approve</code>. If divergence, cost, or retry loops breach tolerance, the action fails with exit code 1 and posts a human-first PR verdict.
</p>

<div className="pt-2">
<a
href="https://github.com/lostmartian/agentdiff-demo/pull/3"
href="https://github.com/lostmartian/agentdiff-demo/pull/4"
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-2 text-xs sm:text-sm font-semibold text-emerald-500 hover:underline"
>
<span>Inspect live PR comment demo on GitHub #3</span>
<span>Inspect live PR comment demo on GitHub #4</span>
<ExternalLink className="w-4 h-4" />
</a>
</div>
Expand Down Expand Up @@ -180,10 +192,10 @@ jobs:
--input '{"scenario": "customer_support"}' \\
--output traces/pr_candidate.json

- name: Gate Against Golden Baseline
uses: kerrshift/agentdiff/.github/actions/agentdiff-check@main
- name: Gate Against Statistical Baseline Envelope
uses: kerrshift/agentdiff/.github/actions/agentdiff-check@v0.5.0
with:
baseline: baselines/golden_run.json
baseline: baselines/customer_support.envelope.json
candidate: traces/pr_candidate.json
pr: \${{ github.event.number }}
github-token: \${{ secrets.GITHUB_TOKEN }}`}
Expand Down Expand Up @@ -257,29 +269,75 @@ jobs:
</div>
</div>

{/* Pillar 3: Automated Rotation */}
{/* Pillar 3: Interactive Approve Bot & Baseline Rotation */}
<div className="py-10 lg:py-12 lg:pl-10 space-y-4">
<div className="flex items-center justify-between">
<span className="text-xs uppercase tracking-wider text-blue-500 font-semibold">
<span className="text-xs uppercase tracking-wider text-emerald-500 font-semibold">
Deliverable 03
</span>
<RotateCcw className="w-4 h-4 text-blue-400" />
<RotateCcw className="w-4 h-4 text-emerald-400" />
</div>
<h3 className="text-xl font-bold text-(--fg) tracking-tight">
Automated Baseline Rotation
Interactive /agentdiff approve Bot
</h3>
<p className="text-sm sm:text-base text-(--muted) leading-relaxed">
Set <code className="text-xs font-mono text-(--fg) bg-(--surface-2) px-1 py-0.5 rounded border border-(--border)">update-baseline = true</code> to safely advance your golden baseline trace only when the candidate execution passes all assertion criteria.
Reviewers bless intentional improvements directly by commenting <code className="text-xs font-mono text-(--fg) bg-(--surface-2) px-1.5 py-0.5 rounded border border-(--border)">/agentdiff approve</code>. Candidate runs join the baseline envelope without local checkouts or manual JSON edits.
</p>
<div className="pt-2 flex items-center gap-2 text-xs text-(--muted)">
<span className="w-1.5 h-1.5 rounded-full bg-blue-400" />
<span>Zero manual JSON copy-pasting</span>
<span className="w-1.5 h-1.5 rounded-full bg-emerald-400" />
<span>Hosted identity as agentdiff[bot] via token.agentdiff.app</span>
</div>
</div>

</div>
</div>
</Reveal>

{/* Approve Bot Spotlight Box */}
<Reveal delay={120}>
<div className="mt-16 p-8 sm:p-10 rounded-3xl border border-(--border) bg-(--surface) shadow-2xs space-y-6">
<div className="flex flex-wrap items-center justify-between gap-4">
<div className="space-y-1">
<span className="text-xs uppercase tracking-wider text-emerald-500 font-semibold">
In-PR Workflow
</span>
<h3 className="text-2xl font-bold text-(--fg) tracking-tight">
Bless improvements without leaving GitHub
</h3>
</div>
<a
href="https://github.com/apps/agentdiff-ci"
target="_blank"
rel="noopener noreferrer"
className="px-5 py-2 rounded-full bg-(--fg) text-(--bg) text-xs font-semibold hover:opacity-90 transition-opacity inline-flex items-center gap-2"
>
<span>Install AgentDiff CI App</span>
<ArrowRight className="w-3.5 h-3.5" />
</a>
</div>

<div className="grid grid-cols-1 md:grid-cols-3 gap-6 text-xs text-(--muted) divide-y md:divide-y-0 md:divide-x divide-(--border)">
<div className="space-y-2 md:pr-6">
<div className="font-semibold text-(--fg) text-sm">1. Reviewer Comment</div>
<p className="leading-relaxed">
A team member with write access comments <code className="text-xs font-mono text-(--fg) bg-(--surface-2) px-1 rounded">/agentdiff approve</code> on a PR flagged for path drift.
</p>
</div>
<div className="space-y-2 md:px-6 pt-4 md:pt-0">
<div className="font-semibold text-(--fg) text-sm">2. D3 Invariant Check</div>
<p className="leading-relaxed">
Path drift and cost deltas are blessable; infinite loops are <strong>never blessable</strong>. The bot refuses to bless cyclical bugs.
</p>
</div>
<div className="space-y-2 md:pl-6 pt-4 md:pt-0">
<div className="font-semibold text-(--fg) text-sm">3. Green Checks-API Flip</div>
<p className="leading-relaxed">
The bot updates the committed baseline envelope, pushes the commit, and flips the GitHub Check result to green.
</p>
</div>
</div>
</div>
</Reveal>
</div>
</section>

Expand Down Expand Up @@ -321,7 +379,7 @@ jobs:
run: git show origin/main:agentdiff.toml > /tmp/baseline-agentdiff.toml

- name: Gate Against Baseline with Governance Guard
uses: kerrshift/agentdiff/.github/actions/agentdiff-check@main
uses: kerrshift/agentdiff/.github/actions/agentdiff-check@v0.5.0
with:
baseline: baselines/golden_run.json
candidate: traces/pr_candidate.json
Expand Down Expand Up @@ -415,11 +473,19 @@ jobs:
Add one GitHub Action step to automate trajectory regression testing across your engineering team in under 5 minutes.
</p>
<div className="flex flex-wrap items-center justify-center gap-4 pt-4 text-sm font-semibold">
<a
href="https://github.com/apps/agentdiff-ci"
target="_blank"
rel="noopener noreferrer"
className="px-8 py-3.5 rounded-full bg-(--fg) text-(--bg) hover:opacity-90 transition-opacity shadow-sm"
>
Install GitHub App →
</a>
<Link
href="/quickstart"
className="px-8 py-3.5 rounded-full bg-(--fg) text-(--bg) hover:opacity-90 transition-opacity shadow-sm"
className="px-8 py-3.5 rounded-full border border-(--border) text-(--fg) hover:bg-(--surface-2) transition-colors"
>
Get Started with Quickstart
Quickstart Guide
</Link>
<Link
href="/docs"
Expand Down
14 changes: 14 additions & 0 deletions website/src/app/(site)/compare/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ interface RowItem {
}

const COMPARISON_ROWS: RowItem[] = [
{
category: "CI/CD & Determinism",
capability: "Statistical baseline envelopes (N-run variance bands)",
agentdiff: true,
judge: false,
observability: "partial",
},
{
category: "CI/CD & Determinism",
capability: "Deterministic CI gate (100% reproducible verdict)",
Expand All @@ -61,6 +68,13 @@ const COMPARISON_ROWS: RowItem[] = [
judge: "partial",
observability: false,
},
{
category: "CI/CD & Determinism",
capability: "In-PR re-baselining bot (/agentdiff approve)",
agentdiff: true,
judge: false,
observability: false,
},
{
category: "CI/CD & Determinism",
capability: "Blocks PR merge in automated CI pipelines",
Expand Down
26 changes: 14 additions & 12 deletions website/src/app/(site)/features/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -1102,26 +1102,28 @@ export default function FeaturesPage() {
<CodeBlock
language="toml"
filename="agentdiff.toml"
code={`# agentdiff.toml — Committed repository configuration
code={`# agentdiff.toml — Committed v0.5.0 repository configuration

[compare]
detect_loops = true
strict_tool_signatures = false
[scenario.customer_support]
mode = "statistical" # "statistical" (N-run envelope) or "strict" (N=1)
sample_runs = 3 # Rolling window of recorded runs
max_cost_increase_pct = 5.0 # Max cost increase delta

[assertions]
max_divergence = 0.25 # Fails CI if TDI > 0.25
max_wasted_effort = 0.10 # Fails CI if > 10% compute is wasted
allow_loops = false # Fails CI on any tool repetition
max_cost_increase_pct = 5.0 # Fails CI if cost delta > +5%
max_recovery_step_ratio = 1.5 # Fails CI if error recovery degrades > 1.5x
[scenario.customer_support.hard_invariants]
fail_on_identical_loops = true # Hard block: infinite retry loops never pass
max_tool_repeats = 3 # Hard block: repeat limit per tool node

[scenario.customer_support.tolerances]
step_count_std_dev = 2.0 # Variance band: mean ± 2σ step count
divergence_ceiling = 0.35 # Max acceptable sequence drift

[masking]
ignore_keys = ["timestamp", "session_id", "trace_id", "auth_token"]
regex_patterns = ["^uuid_[0-9a-f-]+$", "^Bearer\\\\s+.*$"]

[governance]
warn_stale_baseline_days = 30 # Warns when golden baseline exceeds 30 days
flag_threshold_changes = true # Flags PRs that lower gate rigor`}
warn_stale_baseline_days = 30 # Warns when envelope exceeds 30 days
flag_threshold_changes = true # Flags PRs that lower gate rigor`}
/>
</div>
</div>
Expand Down
85 changes: 52 additions & 33 deletions website/src/app/(site)/quickstart/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,9 @@ const STEPS = [
{
num: "01",
tag: "INSTALLATION",
title: "Install the Engine",
title: "Install the CLI & Library",
description:
"Install AgentDiff into your project via pip or uv. Air-gapped, zero cloud accounts, and zero background services.",
"Install AgentDiff into your virtual environment via pip or uv. Pure Python, zero telemetry, and zero background daemons.",
badge: "< 3 seconds",
codeLanguage: "bash",
filename: "terminal",
Expand All @@ -54,45 +54,50 @@ uv add agent-trajectory-diff`,
},
{
num: "02",
tag: "BASELINE CAPTURE",
title: "Record Golden Baseline Trace",
tag: "INIT WIZARD",
title: "Initialize Your Project with agentdiff init",
description:
"Execute your known good agent once to capture its canonical execution DAG. Commit the resulting JSON file directly into git.",
badge: "1-Click Snapshot",
"Run the zero-config wizard. It auto-detects your agent framework (LangGraph, CrewAI, OpenAI Agents, OpenTelemetry), writes agentdiff.toml, and generates your GitHub Actions CI gate workflow.",
badge: "Auto-Detection",
codeLanguage: "bash",
filename: "terminal",
code: `agentdiff record my_agent_module:run \\
--input '{"query": "Generate Q3 sales analysis"}' \\
--output baselines/golden_run.json`,
code: `# Auto-detect framework and generate config + gate workflow
agentdiff init --scenario customer_support --runs 3

# Or include the /agentdiff approve bot workflow
agentdiff init --with-approve`,
},
{
num: "03",
tag: "TOPOLOGICAL EVALUATION",
title: "Diff Candidate Trajectories",
tag: "STATISTICAL BASELINE",
title: "Record an N-Run Baseline Envelope",
description:
"When upgrading prompts, tools, or model weights, run the comparator to verify graph invariance and catch loops in < 5ms.",
badge: "< 5ms Diff",
"Capture variance bands across multiple runs so non-deterministic agents don't flake the CI gate on harmless timing or token jitter.",
badge: "Variance Bands",
codeLanguage: "bash",
filename: "terminal",
code: `agentdiff baselines/golden_run.json candidate_run.json \\
--explain \\
--tree`,
code: `# Record a 3-run baseline envelope
agentdiff record my_agent_module:run \\
--input '{"query": "Generate Q3 sales analysis"}' \\
--runs 3 \\
--out baselines/customer_support.envelope.json`,
},
{
num: "04",
tag: "CI/CD REGRESSION GATE",
title: "Gate Pull Requests in CI",
tag: "HONEST MERGE GATE",
title: "Gate CI & Bless with /agentdiff approve",
description:
"Enforce strict trajectory thresholds in your CI pipeline. Automatically exits with code 1 if thresholds fail and posts a rich PR comment.",
badge: "Exit Code 0 / 1",
"Run the comparator in CI. Hard invariants (loops, error cascades) strictly block merging, while reviewers can bless intended improvements directly from PR comments.",
badge: "Exit 0 / 1 + Bot",
codeLanguage: "bash",
filename: "ci_pipeline.sh",
code: `agentdiff baselines/golden_run.json candidate_run.json \\
code: `# Diff candidate run against statistical baseline envelope
agentdiff diff baselines/customer_support.envelope.json traces/pr_candidate.json \\
--fail-on-regression \\
--max-divergence 0.25 \\
--max-cost-delta 10.0 \\
--format markdown \\
--output-file pr_verdict.md`,
--pr \${{ github.event.number }}

# Reviewers bless accepted improvements right on GitHub:
# Comment: /agentdiff approve`,
},
];

Expand Down Expand Up @@ -280,12 +285,18 @@ export default function QuickstartPage() {
<CodeBlock
language="toml"
filename="agentdiff.toml"
code={`[assertions]
max_divergence = 0.25
code={`[scenario.default]
mode = "statistical"
sample_runs = 3
max_cost_increase_pct = 5.0
allow_loops = false
max_wasted_effort = 0.10
max_recovery_step_ratio = 1.5`}

[scenario.default.hard_invariants]
fail_on_identical_loops = true
max_tool_repeats = 3

[scenario.default.tolerances]
step_count_std_dev = 2.0
divergence_ceiling = 0.35`}
/>
</div>
</div>
Expand All @@ -311,16 +322,24 @@ max_recovery_step_ratio = 1.5`}
Ready to automate on GitHub?
</h2>
<p className="text-base sm:text-lg text-(--muted) leading-relaxed font-normal">
Add the official AgentDiff GitHub Action to your repository and receive instant diagnostic PR comments on every commit.
Install the official GitHub App or drop the Action into your repository for instant diagnostic comments and in-PR approval.
</p>

<div className="flex flex-wrap items-center justify-center gap-4 pt-6">
<a
href="https://github.com/apps/agentdiff-ci"
target="_blank"
rel="noopener noreferrer"
className="px-8 py-3.5 rounded-full bg-(--fg) text-(--bg) text-sm font-semibold hover:opacity-90 transition-opacity inline-flex items-center gap-2 shadow-sm"
>
<span>Install GitHub App</span>
<ArrowRight className="w-4 h-4" />
</a>
<Link
href="/action"
className="px-8 py-3.5 rounded-full bg-(--fg) text-(--bg) text-sm font-semibold hover:opacity-90 transition-opacity inline-flex items-center gap-2 shadow-sm"
className="px-7 py-3.5 rounded-full border border-(--border) bg-(--surface) text-(--fg) text-sm font-semibold hover:bg-(--surface-2) transition-colors"
>
<span>Explore GitHub Action</span>
<ArrowRight className="w-4 h-4" />
</Link>
<Link
href="/docs"
Expand Down
2 changes: 1 addition & 1 deletion website/src/app/layout.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ const jsonLd = {
price: "0",
priceCurrency: "USD",
},
softwareVersion: "0.3.0",
softwareVersion: "0.5.0",
},
],
};
Expand Down
Loading
Loading