Skip to content

Onboarding, project map, Observe pages and console theme - #122

Open
garimamann wants to merge 63 commits into
mainfrom
onboarding-guardrails-map
Open

garimamann wants to merge 63 commits into
mainfrom
onboarding-guardrails-map

Conversation

@garimamann

Copy link
Copy Markdown
Collaborator

What this does

Onboarding

  • agentfox init --key maps a repo, registers its agents and tools under a project, and stays connected as the attack-test runner (onboarding_flow routes, OnboardingSession model + migration).
  • The mapper now reads the OpenAI Agents SDK (typed Agent[Ctx], function_tool(fn) / function_tool(...)(fn), helper-built tool lists), raw function-calling loops, multi-agent repos with hand-offs, and models from constants or config defaults. Tested on three real repos (LangChain marketplace agent, ITSM helpdesk, OpenAI SDK airline agents).
  • New agents get watch-only first guardrails on what users say, what goes to the LLM and what goes to tools; the tools found in their code are granted so default-deny doesn't block them.
  • The setup card lives inside the dashboard: create key → run init → live progress (agents identified → fetching → guardrails on → open map). A new signup and "+ Add a project" see the same view.
  • Disconnect a project (DELETE /api/projects/{project}): agents leave the dashboard, guardrails and grants are cleared; running init again brings it back as new. History is kept.

Map and navigation

  • Projects (agents.project), one at a time, chosen in the sidebar; "Agents" opens the project canvas.
  • Canvas: one user, agent, LLM and a tools box; our mark on guarded lines, wide click targets. The side panel is a setup guide, or for a selected line its deterministic checks and the LLM fallback chain.
  • Observe: Overview, Events, Sessions, Tools, Models, Errors, Security, Evaluations.

Theme

  • dashboard/app/af-theme.css: tokens from the style card, light and dark; Tailwind v4 + shadcn alongside the existing styles; new shield-fox mark.

Fixes

  • A judge outage now fails closed when the judgment posture says so (pipeline.judge_outage).
  • Tool risk labels for money / booking / assign / note tools.

Dev only: /dev/as/admin and /dev/as/fresh switch local accounts (next dev, tokens in .env.local).

Tests

  • Backend: 4,551 passed, 40 skipped (one Presidio test needs the pii extra).
  • Dashboard: 145 passed.
  • New: mapper tests, setup-key tests, disconnect-and-reconnect, judge outage, onboarding unit tests.

Still open

See PRODUCT_TODO.md: more frameworks (LangGraph, CrewAI, TS), all agents on one canvas, setting guardrails from the map, observability through init, agent audit.

🤖 Generated with Claude Code

Onboarding
- `agentfox init --key` maps the repo, registers agents/tools under a project,
  and stays connected as the attack-test runner (onboarding_flow routes,
  OnboardingSession model + migration).
- Mapper reads OpenAI Agents SDK (typed Agent[Ctx], function_tool(fn) and
  function_tool(...)(fn), helper-built tool lists), raw function-calling loops,
  multi-agent repos with hand-offs, and models from constants/config defaults.
- New agents get watch-only first guardrails on what users say, what goes to
  the LLM and what goes to tools; their found tools are granted.
- Setup card inside the dashboard: create key, run init, live progress.
  Same view for a new signup and "+ Add a project"; skip folds it away.
- Disconnect a project (DELETE /api/projects/{project}); init brings it back
  as new. History is kept.

Map and navigation
- Projects (agents.project), one at a time; sidebar switcher; "Agents" opens
  the project canvas.
- Canvas: one user, agent, LLM, a tools box; our mark on guarded lines;
  wide click targets; panel shows a setup guide, or for a line its
  deterministic checks and the LLM fallback chain.
- Observe: Overview, Events, Sessions, Tools, Models, Errors, Security,
  Evaluations.

Theme
- af-theme.css: tokens from the style card, light and dark; Tailwind v4 +
  shadcn alongside the existing styles; new shield-fox mark.

Fixes
- Judge outage now fails closed when the judgment posture says so.
- Tool risk labels for money/booking/assign/note tools.

Dev only: /dev/as/admin and /dev/as/fresh switch local accounts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
guardrails-api Ready Ready Preview Oct 23, 2026 6:41am UTC
guardrails-dashboard Ready Ready Preview Oct 23, 2026 6:41am UTC
guardrails-redteam-lang Ready Ready Preview Oct 23, 2026 6:41am UTC

- Move remove_project to capabilities/onboarding/projects.py: platform may not
  import capabilities (import-linter layers contract).
- Classify PRODUCT_TODO.md in the plugins' docs map (shared, claude-code, codex).
- agent_project migration: add the column and its index separately and only when
  missing, and drop the index only if it exists, so a database whose column was
  added by init_db (no index) upgrades and downgrades cleanly.
- Rebuild the vendored wheels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- `agentfox init --key` now prints "Recognised: N agents, M tools from K Python
  files", any files it couldn't read, and tools found but not given to any agent,
  then "Missing something?": the code shapes init reads and to run it again
  (re-running keeps existing guardrails).
- map_repository returns read/unreadable/unassigned_tools for that summary.
- Test and mock code (testing/, mocks/, fixtures/, test_*.py, *_test.py,
  conftest.py) is no longer read as the agent; a stand-in tool in a repo's
  testing/mocks.py was being reported.
- Rebuild the vendored wheels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
garimamann and others added 2 commits October 20, 2026 16:37
- The user is the only thing outside; inside one box, the project: every agent
  (shown as the loop that decides, not itself guarded) with its LLM, tools and
  documents & memory.
- Guardrails are rails: always-labelled pills on straight lines (Input, Output,
  LLM calls, Tool calls, Tool results, Retrieval & memory), our mark and a count
  where set, "+" where not. Input/Output sit outside the box.
- Several agents: a grid, four to a row; the entry agent (the one that hands off
  to the most others) is the one users talk to; hand-off lines for the active
  agent only; the canvas opens on the active agent and the panel switches agents.
- Panel: a rail says what it checks and between which two things; the guide
  explains how to read the map.
- "LLM calls" (completion) is a step on every agent's map.
- /api/projects lists each project's agent slugs, so the switcher follows an
  agent's page to its project.
- Rebuild the vendored wheels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s beside them

Fixed card sizes so line lengths are exact; documents & memory sit to the
agent's right; hand-off targets are outlined instead of drawn as lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hing sent to the LLM

- First guardrails: one set for what goes to the LLM (user message, documents,
  tool results), plus the LLM's answer (personal data, secrets, harmful content).
  An untouched layer from the earlier init is brought up to date on re-run.
- Map: the LLM line shows 'To the LLM' and 'From the LLM'. It was wired to the
  'completion' surface, which is the agent's done-claim, not an LLM call.
- Decisions count on every line their surface crosses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-test

- Connecting a project queues its 140-attack test with no guardrails.
- The panel shows progress, results per kind of attack, the worst one, and a
  before/after once the basic guardrails are on.
- Basic guardrails: prompt attacks in what users send blocked, personal data
  and secrets in replies hidden; nothing on tools. Enforcing, in a pack of
  their own per agent. init no longer adds watch-only rules.
- Removing a project also clears its basic guardrails and its tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… sets

- Each set is a row: name and what it tries on the left, before -> after on
  the right. Run and + sit in the header; + searches the library.
- Four new sets graded like the rest: hidden encodings, other languages,
  fake authority, bulk actions. A project's chosen sets are kept.
- Removed the panel's 'Set up guardrails' heading and the header's Protect
  button; Pause is now a quiet 'Pause agent' with what it does on hover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ve goes

- Policies opens on Applied rules: every rule in force now, what it does
  (and whether it only watches), every agent or which ones, and the library
  it comes from. Filter by every/some agents or one agent. New endpoint
  GET /api/policies/applied, worked out from each agent's effective policy.
- Libraries: sets of rules applied together (was 'packs'). Each says what it
  covers, how many rules, and whether it's applied and blocking, with one
  action: apply, start blocking, or back to watching.
- 'New rule' in the header. Checks, Performance, Changes, Code and Advanced
  are out of the tab bar but their links still work.
- Sidebar: Guardrails (Agents, Policies, Approvals, Issues, Testing, Traces)
  and Settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a line's rules by library

- Every rule in force knows its library: a shipped one by its title, 'Basic
  guardrails', 'Starter rules (watch only)' (what earlier inits added), or
  'Set for this agent'. Shown in Applied rules and on the map.
- Libraries: Basic guardrails first (the default set) with Apply to all
  agents; Starter rules with Remove (keeps anything else set for an agent).
  POST /api/policies/libraries/basic/apply and /starter/remove.
- A line's panel lists its rules under their library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Applying a library gives each of the project's agents its own copy
  (agent.<slug>.<library policy>), watching first; Start blocking / Back to
  watching switch just those copies. Disconnecting a project removes them.
  Routes: GET /api/policies/libraries/status, POST /libraries/<id>/apply and
  /libraries/<id>/mode with { agents }, basic/apply and starter/remove too.
- Libraries and Applied rules read the project from the sidebar switcher:
  'Apply to <project>', 'On 2 of 5 agents · watching', and rules counted on
  this project's agents only. Libraries applied workspace-wide before are
  labelled as such and not applied twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
architsharm and others added 10 commits October 23, 2026 10:53
…packs, and survive a disconnect

- /libraries/basic/apply goes through the onboarding card's path
  (service.add_basic_guardrails with may_enforce): watching unless the caller
  may change production policy and these exact rules were simulated. It
  crashed with a TypeError before. deps.may_enforce is the one place that
  says who may make a policy block; onboarding, restore and libraries use it.
- /libraries/{id}/mode is gated like POST /api/policies/{key}/mode: a
  policy_production role either way (403), and to block, a recorded
  simulation of each copy's live rules (409 "simulate first: ..."), checked
  for every copy before any switches; each switch audited. 'basic' is a
  library id here too (agent.<slug>.basic).
- POST /libraries/{id}/simulate {agents, project?} replays each agent's
  traffic against its copies with main's simulate + record_simulation, so the
  gate then passes; per agent: blocked / held / hidden, examples, simulation
  ids. SimulationDiff gains would_hide (redactions the candidate would add).
- Each version a library writes records its source in its notes
  ([from <library> <policy> <rules hash>]); status reports outdated /
  outdatedAgents, and applying again updates outdated copies in their mode
  (publish_in_current_mode). An emptied copy is filled again on apply.
  'blocking' reads the project's copies, not an observing workspace pack.
- Disconnect snapshots agent.<slug>.* library copies with the other layers
  (projects.own_keys), and restore-guardrails brings them back under the same
  gate.
- Drafts from agentfox init take libraries (they apply once added);
  /api/policies/applied counts them, marks rule agents draft, and adds
  agentList. Library routes accept project and refuse agents outside it
  (400), unknown projects/agents (404, tenant-scoped), and anything but a
  shipped pack id (404).
- Her four onboarding tests follow #123: drafts, root-slug project keys,
  watching until simulated, the setup key reused across inits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…locking

- Act asks in the page instead of window.confirm, which the in-app browser
  suppresses (the button did nothing). Every Act confirm benefits.
- Basic guardrails say they watch first and block only after a simulation by
  someone allowed; the card shows watching/blocking from the status and gets
  the same Simulate / Start blocking / Back to watching as the other
  libraries. Outdated copies show 'update available' and an Update button.
  Requests carry the project. Server refusals (403/409) show beside the button.
- 'Would block · watching' (Would ask / Would hide) in Applied rules and the
  map's line panel via vocab.wouldDo; ruleTitle keeps PII, API, URL, SQL, ID,
  LLM, MCP, AI, EU upper case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- 'Start blocking' refused without a simulation said to POST an API path; it now
  says to click Simulate.
- A successful simulation resets the Start blocking button, so the earlier
  refusal doesn't stay on screen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Onboarding on the product's own scanner, projects, tokens and enforcer (fixes for #122)

This branch was successfully deployed

3 active deployments
Preview – guardrails-api — 3f01de2f Deployed Oct 23, 2026 by vercel[bot]
Preview – guardrails-redteam-lang — 3f01de2f Deployed Oct 23, 2026 by vercel[bot]
Preview – guardrails-dashboard — 3f01de2f Deployed Oct 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants