Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions .github/workflows/deploy-github-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,7 @@ jobs:
check-latest: true

- name: Install PostCSS dependencies
run: |
npm init --yes
npm install --save-dev autoprefixer@10.6.0 postcss-cli@12.0.0 postcss@8.5.28
run: npm ci --ignore-scripts --no-audit --no-fund
working-directory: ./site

- name: Build Hugo Site
Expand Down
2 changes: 1 addition & 1 deletion cmd/chaos-agent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This provides a realistic workload for the SAM mesh scale testing experiment. Ra

1. It connects to the `sam-node` MCP endpoint via SSE (or `sam-box` proxy in the microVM).
2. It dynamically converts all discovered MCP tools into LangChain `Tool` objects with JSON schemas.
3. It initializes a LangChain `AgentExecutor` using the SAM mesh's OpenAI-compatible inference endpoint (`/v1/chat/completions`).
3. It initializes a LangChain `create_agent` tool-calling agent using the SAM mesh's OpenAI-compatible inference endpoint (`/v1/chat/completions`).
4. It is given an adversarial prompt to explore all tools, fuzz them with extreme inputs, and find bugs.
5. The LangChain framework automatically handles the multi-turn reasoning and tool-calling loop until the agent achieves its goal or hits the max iteration limit.

Expand Down
52 changes: 36 additions & 16 deletions cmd/chaos-agent/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
from typing import Any, Dict

from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate
from langchain.agents import create_agent

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

There is no create_agent function in langchain.agents. Based on the usage of agent.astream(..., stream_mode="updates") and the arguments model=llm, tools=lc_tools, you are actually using the LangGraph prebuilt ReAct agent. This requires importing create_react_agent from langgraph.prebuilt.

Suggested change
from langchain.agents import create_agent
from langgraph.prebuilt import create_react_agent

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

langchain.agents.create_agent exists in LangChain 1.x (https://reference.langchain.com/python/langchain/agents/factory/create_agent) and is the replacement for AgentExecutor; it returns a compiled LangGraph graph, hence astream(..., stream_mode="updates"). langgraph.prebuilt.create_react_agent is the older, now-deprecated path. Verified against langchain 1.4.1.

from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from openai import AsyncOpenAI

Expand All @@ -22,6 +22,32 @@ def tool_schema(spec):
return getattr(spec, "input_schema", None) or getattr(spec, "inputSchema", {})


# AgentExecutor counted model+tool iterations; a LangGraph agent counts graph
# steps, and one iteration is two of those.
MAX_ITERATIONS = 500


async def run_round(agent, prompt):
"""Drive one agent run to completion, echoing each turn as it happens.

Streaming the updates is what verbose=True used to give us: the tool calls
and their results are the whole point of a chaos run, so they are printed
as they arrive rather than summarised at the end.
"""
final = ""
async for step in agent.astream(
{"messages": [HumanMessage(content=prompt)]},
config={"recursion_limit": 2 * MAX_ITERATIONS},
stream_mode="updates",
):
for update in step.values():
for msg in (update or {}).get("messages", []):
msg.pretty_print()
if msg.type == "ai" and not getattr(msg, "tool_calls", None):
final = msg.content
return final


async def pick_model(inference_url, requested):
"""Ask the mesh what it will serve this agent rather than guessing a name.

Expand Down Expand Up @@ -101,27 +127,21 @@ async def mcp_tool(arguments_json: str) -> str:
print(f"mesh offered model: {model}", file=sys.stderr)
llm = ChatOpenAI(
model=model,
openai_api_base=args.inference_url,
openai_api_key=args.auth if args.auth else "none",
base_url=args.inference_url,
api_key=args.auth if args.auth else "none",
default_headers=headers
)

# Define the agent prompt
# No system turn: Gemma and several other instruction-tuned models
# reject the system role outright with a 400, and the mesh may hand
# this agent any model at all. The persona goes in the human turn,
# which every model accepts.
prompt_template = ChatPromptTemplate.from_messages([
("human", "You are an autonomous adversarial AI agent. You have access to tools.\n\n{input}"),
("placeholder", "{agent_scratchpad}"),
])
prompt = f"You are an autonomous adversarial AI agent. You have access to tools.\n\n{args.prompt}"

# Create the LangChain Agent
# create_agent runs the model/tool loop until the model stops
# calling tools; it replaced AgentExecutor in LangChain 1.x.
print("Initializing LangChain Tool-Calling Agent...")
agent = create_tool_calling_agent(llm, lc_tools, prompt_template)

# AgentExecutor runs the ReAct/Tool loop automatically!
agent_executor = AgentExecutor(agent=agent, tools=lc_tools, verbose=True, max_iterations=500)
agent = create_agent(model=llm, tools=lc_tools)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Use create_react_agent from langgraph.prebuilt to correctly initialize the LangGraph ReAct agent, as create_agent does not exist in LangChain.

Suggested change
agent = create_agent(model=llm, tools=lc_tools)
agent = create_react_agent(model=llm, tools=lc_tools)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

langchain.agents.create_agent exists in LangChain 1.x (https://reference.langchain.com/python/langchain/agents/factory/create_agent) and is the replacement for AgentExecutor; it returns a compiled LangGraph graph, hence astream(..., stream_mode="updates"). langgraph.prebuilt.create_react_agent is the older, now-deprecated path. Verified against langchain 1.4.1.


print(f"\n--- Starting Chaos Monkey Agent Loop ---")
print(f"Instruction: {args.prompt}\n")
Expand All @@ -132,9 +152,9 @@ async def mcp_tool(arguments_json: str) -> str:
print(f"\n--- Round {round_num} ---")
started = time.monotonic()
try:
result = await agent_executor.ainvoke({"input": args.prompt})
output = await run_round(agent, prompt)
print("\n--- Final Agent Result ---")
print(result["output"])
print(output)
except Exception as e:
# A crash is a data point, not a reason to stop: an agent
# that dies on the first refusal stops testing anything.
Expand Down
13 changes: 6 additions & 7 deletions cmd/chaos-agent/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
openai>=1.40,<3
openai>=3.11.0,<4
# Pinned to a major version: 1.x spells the transport streamablehttp_client and
# 2.x spells it streamable_http_client, so an unpinned floor silently changes
# the API underneath a sandbox image.
mcp>=2.0,<3
# Pinned below 1.0 deliberately: LangChain 1.x removed AgentExecutor and
# create_tool_calling_agent from langchain.agents, so an unpinned install
# builds an image whose agent fails at import rather than at run time.
langchain>=0.3,<1
langchain-openai>=0.2,<1
mcp>=2.2.0,<3
# LangChain 1.x replaced AgentExecutor with create_agent, which is what the
# agent is written against; 0.x no longer receives security fixes.
langchain>=1.4.0,<2
langchain-openai>=1.6.2,<2
2 changes: 1 addition & 1 deletion cmd/nano-init/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/google/sam/cmd/nano-init

go 1.26.5
go 1.26.6

require (
github.com/aojea/agents.net/tun2connect v0.0.1
Expand Down
2 changes: 1 addition & 1 deletion cmd/sam-a2a-bridge/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/google/sam/cmd/sam-a2a-bridge

go 1.25.7
go 1.25.13

require (
github.com/a2aproject/a2a-go/v2 v2.5.0
Expand Down
4 changes: 2 additions & 2 deletions development/examples/agent-harness/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
# There is no proxy support here on purpose. The sandbox routes the agent's
# traffic to the boundary through tun0, so the agent neither configures a proxy
# nor knows there is one.
openai>=1.40,<3
openai>=3.11.0,<4
# Pinned to a major version: 1.x spells the transport streamablehttp_client and
# 2.x spells it streamable_http_client, so an unpinned floor silently changes
# the API underneath a sandbox image.
mcp>=2.0,<3
mcp>=2.2.0,<3
6 changes: 3 additions & 3 deletions development/examples/calc-mcp/calc_server.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""Calculator MCP backend exposed by node B in the local dev mesh."""
from mcp.server.fastmcp import FastMCP
from mcp.server.mcpserver import MCPServer

mcp = FastMCP("calculator", host="0.0.0.0", port=7777)
mcp = MCPServer("calculator")
Comment on lines +2 to +4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

There is no MCPServer class or mcp.server.mcpserver module in the MCP Python SDK. The high-level server API is still FastMCP from mcp.server.fastmcp.

Suggested change
from mcp.server.mcpserver import MCPServer
mcp = FastMCP("calculator", host="0.0.0.0", port=7777)
mcp = MCPServer("calculator")
from mcp.server.fastmcp import FastMCP
mcp = FastMCP("calculator")

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mcp.server.mcpserver.MCPServer is the mcp 2.x API: 2.0 renamed FastMCP and moved host/port to run() (https://github.com/modelcontextprotocol/python-sdk/blob/main/docs/migration.md#fastmcp-renamed-to-mcpserver). On mcp 2.2.0 import mcp.server.fastmcp raises ModuleNotFoundError: ... This is mcp 2.x, where FastMCP was renamed. The Bats e2e on this PR builds and runs this server.



@mcp.tool()
Expand All @@ -17,4 +17,4 @@ def multiply(a: float, b: float) -> float:


if __name__ == "__main__":
mcp.run(transport="streamable-http")
mcp.run(transport="streamable-http", host="0.0.0.0", port=7777)
2 changes: 1 addition & 1 deletion development/examples/calc-mcp/requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
mcp[cli]>=1.0.0,<2.0.0
mcp[cli]>=2.2.0,<3
12 changes: 6 additions & 6 deletions development/examples/chat-a2a/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
a2a-sdk>=1.0
google-genai>=1.0
httpx>=0.27
sse-starlette>=2.0
starlette>=0.40
uvicorn>=0.30
a2a-sdk>=1.1.2
google-genai>=2.22.0
httpx>=0.28.1
sse-starlette>=3.4.11
starlette>=1.6.0
uvicorn>=0.52.4
4 changes: 2 additions & 2 deletions development/examples/code-reviewer-pool/manager/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
FROM node:26-slim@sha256:14bf3eac4bf209d906d3c41256597d3ab1f926b2e93a79e9bdfe1efd32454239

WORKDIR /srv
COPY package.json server.mjs lease-token.mjs ./
RUN npm install
COPY package.json package-lock.json server.mjs lease-token.mjs ./
RUN npm ci --omit=dev --ignore-scripts --no-audit --no-fund

EXPOSE 7780
CMD ["node", "server.mjs"]
Loading
Loading