Skip to content

fix(release-verify-rc): use placeholders in step-6 artefact example - #971

Open
Shaurya2k06 wants to merge 1 commit into
apache:mainfrom
Shaurya2k06:fix/release-verify-rc-step6-placeholders
Open

fix(release-verify-rc): use placeholders in step-6 artefact example#971
Shaurya2k06 wants to merge 1 commit into
apache:mainfrom
Shaurya2k06:fix/release-verify-rc-step6-placeholders

Conversation

@Shaurya2k06

Copy link
Copy Markdown
Contributor

Summary

  • Replace the concrete apache-airflow-2.11.0-source-release worked example in
    release-verify-rc step 6 with the skill's existing <artefact-source-release>
    placeholder so the extension-stripping lesson stays project-agnostic.
  • Keeps the taught behaviour: strip the archive extension (.tar.gz / .zip),
    do not drop the -source-release suffix.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • uv run --project tools/skill-and-tool-validator skill-and-tool-validate passes
  • tools/dev/check-placeholders.sh passes
  • prek run --files skills/release-verify-rc/SKILL.md passes (includes
    check-placeholders + skill-and-tool-validate)
  • Confirmed no remaining concrete artefact name in the step-6 example
  • For skill behaviour changes: n/a — prose-only placeholder example swap;
    eval fixtures intentionally keep concrete names under the allowlisted
    tools/skill-evals/evals/ path

RFC-AI-0004 compliance

  • HITL — any new mutation is gated on explicit user confirmation
  • Sandbox — no new unrestricted host access; network reach declared in the adapter
  • Vendor neutrality — placeholders used in all skill / tool prose
  • Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
  • Write-access discipline — no autonomous outbound messages; drafts only, sent on confirmation
  • Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed

Linked issues

Closes #945

Replace the concrete apache-airflow-2.11.0-source-release worked example
with the skill's existing <artefact-source-release> token so the
extension-stripping lesson stays project-agnostic.

Signed-off-by: shaurya2k06 <shaurya2k06@gmail.com>
@justinmclean

Copy link
Copy Markdown
Member

The placeholder swap is right in principle, but these lines are inside ## Step 6 — Binary exclusion check, and step-config.json extracts that step as the system prompt for step-6-binary-exclusion. Both of its cases expect paste_recipe to contain find apache-airflow-2.11.0-source-release …, and that field is judge-graded, so this is a prompt change with a fixture dependency. Could you run that suite before this merges? Note: I ran it and am seeing failures.

skills/release-verify-rc/SKILL.md:453

The lines are inside Step 6, and step-config.json for step-6-binary-exclusion extracts that step as the eval's system prompt. Both cases expect paste_recipe to contain find apache-airflow-2.11.0-source-release -type f ..., derived from the report's apache-airflow-2.11.0-source-release.tar.gz, and paste_recipe is prose-graded by a judge. The worked example you removed is what made that derivation obvious. It may well still pass, but the test plan lists the validator, the placeholder check and prek, not the suite that actually covers this:

uv run --directory tools/skill-evals skill-eval --cli "claude -p" \
  evals/release-verify-rc/step-6-binary-exclusion/fixtures/

Second, the example no longer demonstrates anything. <artefact-source-release>.tar.gz unpacks to <artefact-source-release> is a tautology, and the very next sentence warns "do not drop the -source-release suffix" — a suffix now visible only inside the placeholder's own name. Keeping a worked shape without the product identity would satisfy the issue and keep the lesson:

`apache-<project>-<version>-source-release.tar.gz` unpacks
to `apache-<project>-<version>-source-release`

Both <project> and <version> are already used as placeholders across the release skills.

This review was drafted by an AI-assisted tool and
confirmed by a Magpie maintainer. The findings
below are observations, not blockers; a Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Magpie handles maintainer review:
[CONTRIBUTING.md](https://github.com/apache/magpie/blob/main/CONTRIBUTING.md).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace the hardcoded product identity in release-verify-rc step 6

2 participants