feat: add agentrig test for repository case files (AR-RFC-0006 B1) - #3
Merged
Merged
Conversation
用例写成仓库里的 YAML(agentrig/project.yaml + agentrig/cases/**),与 Agent 代码在同一个 PR 里 评审。agentrig test 按正式迁移建临时库、进程内执行: - 所有文件校验通过才开始,报错带文件与从 1 开始的位置;运行前真实启动一次被测 Agent; - 按目录、标签、名称挑选,并发与重复次数可配置; - 默认 Fixture → Sample → Curator,缺 Key 直接报错,--no-curator 为严格模式; - 控制台摘要指出违规调用与结果来源,另输出 JUnit 与 Markdown; - 退出码与 agentrig gate 一致:0 通过、1 配置错误、2 回归、3 无法判定; - --keep-db 保留的库可直接用 agentrig serve 查看。 python_agent 的 entry 同时支持 path/to/file.py:attribute,入口所在目录不必是 Python 包。 新增运行时依赖 PyYAML。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why / 动机
落地 AR-RFC-0006 的 B1:用例写成仓库文件,与 Agent 代码在同一个 PR 里评审;
agentrig test一条命令在本地和 CI 跑完,出现回归返回非零退出码。What changed / 改动
agentrig/casefiles/:project.yaml与用例文件的加载、校验、挑选、进程内执行与报告;agentrig test:--tag、-k挑选;--concurrency、--repeat;--no-curator为严格模式;--junit、--markdown输出;--keep-db保留可被agentrig serve打开的库;agentrig gate一致:0 通过、1 配置错误、2 回归、3 无法判定。python_agent的entry支持path/to/file.py:attribute,入口所在目录不必是 Python 包;uvicorn[standard]间接安装);Boundaries and risk / 边界与风险
agentrig init(B2);文档示例与 README 主路径改写(B3);GitHub Action。web/);指南中已写明。Verification / 验证
uv run ruff check src tests scripts examplesuv run mypy src/agentriguv run pytest:235 passed,5 skipped--keep-db库可被服务端校验、文件路径入口agentrig test:基线 0、跳过确认的候选版本 2agentrig test基线通过;提示词改坏的候选版本退出码 2,并指出违规调用web/未改动