Skip to content

feat: real agentic Propose via GitHub Copilot CLI, remove mock - #91

Open
ineedjet wants to merge 1 commit into
mainfrom
feat/real-propose-agent
Open

feat: real agentic Propose via GitHub Copilot CLI, remove mock#91
ineedjet wants to merge 1 commit into
mainfrom
feat/real-propose-agent

Conversation

@ineedjet

Copy link
Copy Markdown
Contributor

Summary

Splits Propose into a deciding half and a mechanical half, mirroring the split #11 already draws between Apply/Reject (deterministic) and Distill/Rework (agent work):

  • New propose-context mode: gathers regulations text, the candidate issue's title/body, and the field catalog into one self-contained prompt. Output only — agnostic to whichever inference mechanism reads it.
  • propose now requires --model-response (JSON: {"title", "body", "fields": {...}}) instead of deciding content itself. mock_value_for is deleted entirely — mock mode is no longer needed, per instruction.
  • proposal-shared.yml's propose job now runs propose-context → installs Copilot CLI → calls actions/ai-inference with that prompt → feeds the response into propose.
  • validate hard-errors on schedule/workflow_dispatch runs missing model_credentials, same as it already does for capacity_query/queue_query.

A real detour: GitHub Models is retired

Went looking for "GitHub's own built-in agent tooling" and found GitHub Models (the inference API this would have used) was fully retired July 30, 2026 — playground, catalog, inference API, BYOK, gone for everyone. actions/ai-inference pivoted to wrap GitHub Copilot CLI instead, which is what this PR actually uses. Requires a PAT with an active Copilot seat, not GITHUB_TOKEN.

A real bug found along the way

write_output wrote plain key=value\n lines to $GITHUB_OUTPUT — corrupts the file for any multiline value. prompt is the first output that's always multiline. Fixed by switching to GitHub's documented delimiter form unconditionally.

Dogfood tuning (this repo's own regulations_path: TRANSLATE.md)

  • Test Project retitled to Translate.
  • Replaced the five generic placeholder fields (Summary/Points/Due/Priority/Tags) with a single real Proposal Size/Size pair (SINGLE_SELECT, options matching this repo's own □ XS/S/M/L/XL label names — reused for a familiar scale, not read from those labels; the regulation estimates size from text density, not effort).
  • proposal.yml wired to a new COPILOT_PAT secret — not yet created, needs a PAT with an active Copilot seat before Propose can run for real here.

Verification

Live against real repository data, not assumed:

  • propose-context produces a correct, complete prompt — regulations text, a real non-English test issue's title/body, and the live field catalog.
  • propose --model-response correctly creates a real proposal from a hand-crafted JSON response — title, body, and the Size field all landed correctly. Confirms the mechanical half works end-to-end.
  • The real model call (actions/ai-inference → Copilot CLI) still needs COPILOT_PAT to exercise for real — not yet tested live.

Test plan

  • pre-commit run --all-files passes.
  • propose-context verified live (real regulations file, real non-English issue, real field catalog).
  • propose --model-response verified live end-to-end (hand-crafted JSON → real created proposal, all fields correct).
  • Full real model call through actions/ai-inference/Copilot CLI — blocked on COPILOT_PAT secret creation.

Related: #11, #9

🤖 Generated with Claude Code

Splits Propose into a deciding half and a mechanical half, per the
same split #11 already draws between Apply/Reject (deterministic) and
Distill/Rework (agent work) -- deciding proposal content is agent
judgment, out of scope for a deterministic script:

- New propose-context mode: gathers regulations text (fetched from
  regulations_repo/regulations_path's default branch via
  repository.object(expression: "HEAD:<path>")), the candidate issue's
  title/body, and the field catalog (reusing proposal_fields) into one
  self-contained prompt, output only -- agnostic to whichever inference
  mechanism a caller wires in.
- propose now takes a required --model-response JSON blob
  ({"title", "body", "fields": {...}}) instead of deciding content
  itself via mock_value_for, which is deleted entirely (mock mode is
  no longer needed, per instruction). An unknown field name in the
  response is a warning and a skip, not an error -- the model's output
  isn't trusted to match the catalog exactly.
- proposal-shared.yml's propose job now runs propose-context, installs
  the Copilot CLI, calls actions/ai-inference (GitHub's own built-in
  agent tooling -- GitHub Models itself was fully retired July 30,
  2026, actions/ai-inference now wraps Copilot CLI instead) with that
  prompt, and feeds the response into propose. validate hard-errors on
  schedule/workflow_dispatch runs missing the model_credentials secret,
  same as it already does for capacity_query/queue_query.
- Fixed a real latent bug found while building this: write_output used
  a plain key=value line, which corrupts $GITHUB_OUTPUT for any
  multiline value -- prompt is the first output that's always
  multiline. Switched to GitHub's documented delimiter form
  unconditionally.

Also, for this repo's own dogfood use (regulations_path: TRANSLATE.md):
retitled the test Project to "Translate", replaced the five generic
placeholder test fields (Summary/Points/Due/Priority/Tags) with a
single real Proposal Size/Size pair (SINGLE_SELECT, options matching
this repo's own □ XS/S/M/L/XL label names -- reused for a familiar
scale, not read from those labels), and wired
proposal.yml's model_credentials to a new COPILOT_PAT secret (not yet
created -- needs a PAT with an active Copilot seat before propose can
run for real in this repo).

Verified live against real repository data, not assumed:
propose-context produces a correct, complete prompt (regulations text,
a real non-English test issue's title/body, and the live field
catalog); propose --model-response correctly creates a real proposal
from a hand-crafted JSON response (title, body, and the Size field all
landed correctly) -- confirming the mechanical half works end-to-end
before wiring in the real model call, which still needs COPILOT_PAT to
exercise for real.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dac6c6eafe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

filled = []
for field_name, field in fields.items():
value, names = mock_value_for(field)
for field_name, raw_value in model_fields.items():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Validate model fields before creating the issue

When inference returns valid JSON whose fields value is an array or string, the proposal has already been created before this .items() call raises AttributeError; it is therefore left without field values, a control comment, or its transition, and a retry can create a duplicate proposal. Validate that the response is an object and fully normalize its fields before performing createIssue.

AGENTS.md reference: AGENTS.md:L138-L138

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant