Intent-Consistency Verification at the Action Gate: A Request-Derived Alternative to Formal Contract Gating for Tool-Using LLM Agents
LLM Security · Agentic AI · AI Safety — Thesis Project (2026) · github.com/Atik203/IntentGate
Status (2026-10-22): v1 implementation and Phases 1–7 evaluation are complete; the ICEEICT manuscript completed its final pass on 2026-10-20. Phase 8 is a planned journal-strengthening study. Its research questions and candidate contributions are provisional until the related-work and protocol gates pass. See
roadmap.mdand blueprint Section 19.
MCP-style agents execute real actions (email, payments, code, files). Request-derived policies and runtime action authorization now appear in several systems. Their policy representations, update lifecycles, threat models, and evaluations differ; the journal study will compare them under a shared protocol before making a novelty claim.
IntentGate v1 is a model-agnostic middleware gate that derives an intent contract from the user's original request and scores proposed tool calls before execution. Its completed evaluation compares an unprotected agent and a ToolGate reimplementation on pinned InjecAgent and MCPTox suites. The resulting evidence is bounded to those tasks, models, and implementations.
New to the project? Start with docs/team_guide.md — a beginner-friendly
explanation of the project, what has been done, and where it is going.
Install and verify in ~5 minutes: see docs/setup.md (venv + pip install -e ".[dev]" + .env + pytest -q + smoke-test scripts).
IntentGate\
├── README.md # this file
├── CONTRIBUTING.md # branch model, PR rules, definition of done
├── SECURITY.md # vulnerability reporting + scope
├── CODE_OF_CONDUCT.md # Contributor Covenant v2.1
├── CHANGELOG.md # Keep a Changelog; updated per PR
├── LICENSE # MIT
├── AGENTS.md # instructions for AI coding agents (+ LLM cache guidance)
├── roadmap.md # phased tracker (Phases 0–8, Gates 0–3, success criteria)
├── docs\
│ ├── README.md # docs hub (where everything lives + status snapshot)
│ ├── setup.md # install + verify instructions (venv, pip, .env, tests)
│ ├── blueprint.md # single source of truth (design, Sec 0–18, indexed)
│ ├── references.bib # BibTeX citation basis (10 verified papers)
│ └── experiments\ # experiment record: README hub + 01–05 reports (E1–E17)
├── pyproject.toml # intent-gate package (pip install -e .)
├── .github\ # PR/issue templates + CI (pytest on dev/main)
├── configs\ # intent_schema.json, parser_fewshots.json, thresholds.yaml, models.yaml
├── src\intent_gate\ # parser / scoring / gate / agent / baselines.toolgate / eval
├── harness\ # run_injecagent.py, run_mcptox.py, compare_b2.py, common.py
├── scripts\ # pilot_score_dist.py, check_parser.py, clone_benchmarks.ps1
├── tests\ # pytest suite (parser, veto, no-bypass, ordering, metrics)
├── data\ # (gitignored) benchmark clones + snapshots — scripts/clone_benchmarks.ps1
├── results\ # tracked on purpose: JSONL traces + reports (teammates read instead of re-running evals)
├── literature_review\
│ ├── index.md # Master Comparison Matrix + Gap Map + Verification Log (5 anchors + Closest)
│ └── papers\
│ ├── 01-agentdojo-debenedetti-2024.md # Anchor — dynamic 97-task/629-case injection harness
│ ├── 02-injecagent-zhan-2024.md # Anchor — 1,054 indirect-injection cases
│ ├── 03-mcptox-wang-2025.md # Anchor — 1,312 live-MCP poisoning cases
│ ├── 04-asb-zhang-2025.md # Anchor — 10-scenario/27-method superset
│ └── 05-toolgate-liu-2026.md # Closest — Hoare-contract gate (B2 baseline)
├── literature_review.md # (local, gitignored) original 10-paper Q1–Q9 synthesis
├── mds\ # (optional, gitignored) pasted paper markdowns for offline access
└── pdfs\ # (optional, gitignored) paper PDFs
feature branch (feat/, fix/, docs/, exp/, <member>/) ──PR──> dev ──PR (tested+reviewed)──> main
main— production. Only merges fromdevafter all tests pass. Never commit directly.dev— integration branch. All feature PRs targetdev.- Feature branches start from
dev, named after the feature or the member (e.g.feat/gate-veto,atik/toolgate-contracts).
Full rules: CONTRIBUTING.md. Security reports: SECURITY.md.
- Index:
literature_review/index.md— v1 review matrix plus newly identified closest systems; the Phase 8 feature-level audit remains open - Papers: 5 detailed reviews in
literature_review/papers/— each follows the same template (badges, Summary, Relevant to Our Idea, Gap, Q1–Q9, Citation, Method, Results, Limitations, Comparison, Positioning, Reproducibility, Cross-References, Relevance to Thesis) - Blueprint:
docs/blueprint.md— completed v1 design (Sections 0–18) plus provisional journal study (Section 19) - Roadmap:
roadmap.md— phased tracker (Phases 0–8, Gates 0–3, success criteria, current status) - Docs hub:
docs/README.md· Team guide (start here):docs/team_guide.md· Experiments:docs/experiments/README.md— status matrix, experiment index E1–E17, datasets, pinned models, artifact map, reproduce commands, supervisor Q&A
All 6 reviewed papers were checked against their primary full text. BibTeX: docs/references.bib.
- Title: Intent-Consistency Verification at the Action Gate: A Request-Derived Alternative to Formal Contract Gating
- Domain: LLM Security · Agentic AI · AI Safety
- Motivation: Tool-using agents can produce harmful side effects; current task-derived and runtime authorization systems need matched comparison under common threats and tasks
- Problem: Injection, tool poisoning, and drift all converge to one observable — an agent executes an action the user never intended
- V1 study question: whether a request-derived action gate reduces measured attack success on InjecAgent and MCPTox with acceptable utility and latency compared with the tested baselines; this does not establish uniqueness against newer systems
- Solution: Parse user request → structured intent contract (goals, tool categories, data scopes, side-effect limits) → middleware gate scores every proposed
tool_call(name,params)viaS = α·((cosine(embed(contract), embed(call))+1)/2) + (1-α)·rule_compliancewith veto; decisionallow / block / escalatevia thresholdτ(sweep) - Research status: v1 prototype and frozen evaluation are complete; request-derived policies and action-level gates overlap with newer systems, so no “first” claim is made. The journal study's candidate contribution is a controlled comparison of policy representation and update lifecycle, subject to the Phase 8 related-work gate.
- Deliverables: Pip-installable wrapper, ToolGate reimplementation (Appendix G), frozen evaluation report, and completed ICEEICT conference manuscript; journal extension is planned in
roadmap.mdPhase 8. - Frozen evaluation: B1 vs. B2 ToolGate reimplementation vs. v1 on InjecAgent (2,108) and MCPTox (1,348); results and limitations are frozen in the experiment reports. Planned studies use separate Phase 8 runs.
- Stack: Open LLM (Qwen/Llama via API), ReAct/MCP client, Python middleware,
all-MiniLM-L6-v2(CPU), no fine-tuning
- B1: Unprotected ReAct (no gate) — raw vulnerability floor
- B2: ToolGate reimplementation (Hoare
{P}T{Q}per Appendix G, symbolic world-state) — validated on ToolBench/MCP-Universe subset before adversarial runs - Metrics: ASR-valid, FPR, escalation rate, latency p50/p95, setup cost, utility retention on benign cases
- Ablations: A1 semantic-only, A2 rule-only, A3 raw-request vs structured contract, A4 threshold/escalate sweep
- Stats: Bootstrap 95% CI on ASR, paired McNemar per case, per-risk-category breakdown (MCPTox 11 categories, InjecAgent 17 tools)
# 1. Clone pinned benchmark snapshots
git clone https://github.com/uiuc-kang-lab/InjecAgent
git clone https://anonymous.4open.science/r/AAAI26-7C02 # MCPTox snapshot
git clone https://github.com/OceannTwT/ToolGate # B2 reference
git clone https://github.com/ethz-spylab/agentdojo # stretch
# 2. Install gate
pip install -e . # pip wrapper around agent tool executor
# 3. Run a harness evaluation (use the exact commands/configs in the experiment reports)
python harness/run_injecagent.py --cases data/raw/InjecAgent/data/test_cases_dh_base.json --gate ours --threshold 0.6 --report results/injecagent.json
python harness/run_mcptox.py --gate ours --snapshot v1 --report results/mcptox.json
python harness/compare_b2.py --cases data/raw/InjecAgent/data/test_cases_dh_base.json --report results/comparison.jsonFor reproduction commands and artifact integrity checks, follow
docs/setup.mdanddocs/experiments/README.md. Do not overwrite frozen results; new journal experiments use the Phase 8 protocol andresults/phase8/.
The ICEEICT conference manuscript covers the completed v1 study. A journal target has not been
selected: Phase 8 first requires a current related-work audit, a defensible extension, and verification
of journal scope, category-specific quartiles, fees, and conference-extension policy. See
roadmap.md Phase 8 and docs/blueprint.md.
| # | Paper | Venue | Role |
|---|---|---|---|
| 1 | AgentDojo (Debenedetti et al., 2024) | NeurIPS D&B | Anchor benchmark |
| 2 | InjecAgent (Zhan et al., 2024) | Findings of ACL 2024 | Anchor benchmark |
| 3 | MCPTox (Wang et al., 2025) | arXiv → AAAI 2026 | Anchor benchmark (live) |
| 4 | ASB (Zhang et al., 2025) | ICLR 2025 | Anchor superset |
| 5 | ToolGate (Liu et al., 2026) | arXiv 2601.04688v1 | Closest system (B2) |
Full reviews: literature_review/papers/
5-member team workflow: feature branch → dev → main. Read CONTRIBUTING.md before your first PR; CI runs pytest -q on dev and main. All contributors follow CODE_OF_CONDUCT.md. Report vulnerabilities privately per SECURITY.md (never in public issues).
MIT — Copyright (c) 2026 IntentGate contributors.
@misc{intentgate2026,
title={Intent-Consistency Verification at the Action Gate: A Request-Derived Alternative to Formal Contract Gating for Tool-Using LLM Agents},
year={2026},
note={Thesis project — IntentGate, https://github.com/Atik203/IntentGate}
}Last updated: 2026-10-13 — blueprint is the single source of truth, roadmap.md tracks status. Phase 6 (E18–E22) complete.