You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: keep committed external skills through APM Materialize - #240
It's not line endings or a Windows-generated lock. The committed security-audit files are LF and their SHA-256 hashes match apm.lock.yaml exactly.
apm-cli 0.35.0 trusts a skill file's overwrite baseline only from per-user deployment receipts (~/.apm/deployment-receipts/ plus .apm-local/installation). It never trusts the committed lock (DeploymentMaterializationSession.baseline). A fresh runner has no receipts. For a root-level skill package, _integrate_native_skill has no "identical, adopt it" shortcut, unlike agents, which are "adopted". So every committed file is skipped with no trustworthy local deployment baseline. Nothing gets deployed for the package, so the cleanup phase deletes all 21 lock-tracked paths as stale.
APM Materialize: a new scripts/apm-external-skills.py prepare step removes the external skill folders recorded in apm.lock.yaml before apm install. APM then redeploys them from the pinned commit, and a missing target needs no receipt. Git still holds the committed copy, so the materialize PR diff still shows any upstream change. The review gate is unchanged, and apm-overrides.json is still re-applied after the install.
Guard: apm-external-skills.py check runs after the install and fails the run, rather than opening a PR, in any of these cases:
a locked dependency deploys nothing;
a deployed file is missing, or a skill folder has no SKILL.md;
a deployed path is git-ignored (a new or renamed skill folder without its .gitignore negation);
a dependency lost a path at the commit the baseline lock already pins (HEAD in Materialize; the PR base in PR CI via check --base);
a committed .agents/skills/ file isn't claimed by the lock.
tests/test_apm_external_skills.py runs the same check against the committed tree on PRs (Markdown Validation) and in lefthook. On chore: materialize APM primitives #233's head it fails with cloudflare/security-audit-skill/skills/security-audit deploys no files. prepare refuses to delete through a symlinked .agents or .agents/skills.
validate-coverage-ledger.cjs and validate-findings.cjs regain their upstream 100755 mode, which the Windows commit in feat: add cloudflare security-audit skill #226 dropped. Otherwise every Linux materialize run would show a mode change.
docs/apm.md explains the workaround and when to drop it.
Testing
Run in separate testing contexts on WSL Debian (aarch64):
Repro with real apm-cli 0.35.0 and an empty HOME, on main (7e4ff4b): apm install --update --only apm exits 0 but deletes all 20 skill files (Cleaned 21 stale deployed files).
Same run with this branch's prepare step, then the overrides and check: the skill is untouched and check exits 0. The only lock change is generated_at, plus the stale require-push-approval.sh hash from feat(hooks): let git-push-approval allow feature-branch pushes #230, which is legitimate drift. A second run is deterministic.
test_apm_external_skills.py (11 tests), test_apm_overrides.py, validate-prompts.sh and check-index.sh all pass. Manual Linux scenarios also fail as expected: a renamed skill, a partial file loss, a symlinked parent. Every signed commit was made with all lefthook hooks passing on Linux.
This PR doesn't touch apm.yml, so merging it won't trigger APM Materialize. Dispatch it manually, or wait for the next release bump. It should then open a PR with only that lock refresh.
apm-cli 0.35 trusts a skill file's overwrite baseline only from per-user
deployment receipts, so on a fresh runner it skips the committed
security-audit skill and then deletes it as stale (microsoft/apm#3372).
APM Materialize now removes the lock-recorded external skill folders before
`apm install`, so they are redeployed from the pinned commit, and fails the
run if a locked dependency loses its files. A PR-time test runs the same
check against the committed lock. The two upstream-executable validator
scripts regain their exec bit, which the Windows commit in #226 dropped.
Closes#237
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
No actionable comments were generated in the recent review.
Walkthrough
The change adds a script that prepares locked external skill folders for APM deployment and checks deployed files against the lockfile and tracked skill files. The materialization workflow runs these commands around installation. CI, pre-commit, and documentation also cover the script.
Changes: External Skill Materialization
Layer / File(s)
Summary
Lock parsing and preparation scripts/apm-external-skills.py, tests/test_apm_external_skills.py
The script parses locked dependencies and deployed files, then removes existing locked skill directories before deployment. Tests cover lock parsing and preparation behavior.
The script reports dependencies without deployed files, missing locked files, and tracked skill files not claimed by the lock. The CLI returns status codes for reported problems and errors. Tests cover check results and command execution.
Materialization and validation integration .github/workflows/apm-materialize.yml, .github/workflows/markdown-validation.yml, .lefthook.toml, docs/apm.md
The materialization workflow runs prepare before APM installation and check afterward. CI and pre-commit run the tests. The documentation describes the behavior and commands.
sequenceDiagram
participant APMMaterializeWorkflow
participant ExternalSkillsScript
participant APM
APMMaterializeWorkflow->>ExternalSkillsScript: Run prepare
APMMaterializeWorkflow->>APM: Run install
APMMaterializeWorkflow->>ExternalSkillsScript: Run check
Loading
Merge Risk: Minimal · up to 61033
This change keeps committed external skills from being deleted by APM Materialize on fresh runners, and it adds a check that catches dropped deployments. No concrete merge-blocking issue remains.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the fix and its subject: committed external skills being kept through APM Materialize. The changeset adds prepare and check commands in scripts/apm-external-skills.py, wires them into …
Linked Issues check
Issue #237 has one active direct link and no active parent context. Its coding requirements are:
1. Remove lock-recorded external skill folders before apm install, so APM redeploys them. Implemente…
Out of Scope Changes check
All changes map to #237 or to support for it:
- .github/workflows/apm-materialize.yml: prepare and check steps. In scope. - scripts/apm-external-skills.py and `tests/test_apm_external_skills.…
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check
Explanation
The title names the fix and its subject: committed external skills being kept through APM Materialize. The changeset adds prepare and check commands in scripts/apm-external-skills.py, wires them into the APM Materialize workflow, and documents the workaround. The title matches the main change and uses the 'fix:' convention. It is concise and specific enough for a teammate scanning history.
Full details: Linked Issues check
Explanation
Issue #237 has one active direct link and no active parent context. Its coding requirements are:
Remove lock-recorded external skill folders before apm install, so APM redeploys them. Implemented by prepare(root, deps) in scripts/apm-external-skills.py. The APM Materialize workflow runs prepare before apm install when apm.yml exists. The script refuses to remove symlinks or non-directories.
Keep the committed copy as the review baseline. The prepare step only removes working-tree folders. The committed files stay in Git, so the materialize PR diff still shows upstream changes.
Add a guard that fails materialize and PR CI when a locked dependency deploys nothing, a deployed file is missing, or a committed .agents/skills/ file is not claimed by the lock. Implemented by check(root, deps, tracked), run after install in the workflow. tests/test_apm_external_skills.py runs the same check against the committed tree in .github/workflows/markdown-validation.yml and in .lefthook.toml.
The author reports a real-CLI reproduction with apm-cli 0.35.0 and an empty HOME. The author reports that the prepare/install/overrides/check sequence preserves the skill and passes check. This is author-reported evidence, not verified in this review. The request to close #233 unmerged is administrative and does not apply.
Full details: Out of Scope Changes check
Explanation
All changes map to #237 or to support for it:
.github/workflows/apm-materialize.yml: prepare and check steps. In scope.
scripts/apm-external-skills.py and tests/test_apm_external_skills.py: implementation and tests of the prepare/check logic. In scope.
.github/workflows/markdown-validation.yml and .lefthook.toml: run the check on PRs and in pre-commit hooks. These run the guard required by #237. In scope.
docs/apm.md: documents the workaround and when to remove it. Supporting documentation. In scope.
validate-coverage-ledger.cjs and validate-findings.cjs mode restored to 100755: not a coding requirement of #237. The PR description says the Windows commit in #226 dropped these modes, and that without the restore every Linux materialize run shows a mode change. This is a small, stated reason tied to materialize runs. Not a demonstrated unrelated change.
The mode restore is not visible in the supplied summary. Its effect was checked only through the PR description. No out-of-scope change was demonstrated.
Finishing Touches
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Critical issues allow deletion outside the checkout through symlinked parent directories and let an empty deployed skill pass the preservation check without a claimed, regular SKILL.md.
Addresses #237 by working around APM’s fresh-checkout behavior that deletes committed external skills, preserving the toolkit’s reviewable dependency snapshots.
Changes:
Adds lock-driven preparation before installation and deployment checks afterward.
Integrates regression tests into CI and pre-commit hooks, and documents the workaround.
Restores upstream executable modes for the two vendored validators.
File
Description
tests/test_apm_external_skills.py
Adds parsing, preparation, deployment, and committed-tree checks.
scripts/apm-external-skills.py
Adds external-skill cleanup and post-install validation commands.
docs/apm.md
Explains the workaround, fresh-clone usage, and removal criteria.
.lefthook.toml
Runs external-skill tests before commits.
.github/workflows/markdown-validation.yml
Adds relevant triggers and external-skill tests.
.github/workflows/apm-materialize.yml
Prepares skills before installation and validates them before opening a PR.
SkillSpector 2.13.4 static scan (no LLM, no network) of 21 AI primitive targets at 7231f1b.
HIGH/CRITICAL findings block unless the owner accepted them in .github/skillspector-accepted.json. Findings are heuristic, not confirmed vulnerabilities.
Accepted: External cloudflare/security-audit-skill (vendored via apm.yml): audit guidance asking who may create, update and delete agent memory across tenants; it does no...
Accepted: External cloudflare/security-audit-skill (vendored via apm.yml): checklist item asking whether credentials.json or .env files are committed to the audited repo;...
Accepted: External cloudflare/security-audit-skill (vendored via apm.yml): names keychain ACLs as an OS control to account for when judging a trust boundary; it does not ...
Accepted: External cloudflare/security-audit-skill (vendored via apm.yml): derived from the accepted static_parse_limit on the bundled validator, which only reads its JSO...
Accepted: External cloudflare/security-audit-skill (vendored via apm.yml): derived from the accepted static_parse_limit on the bundled validator, which only reads its JSO...
Accepted: External cloudflare/security-audit-skill (vendored via apm.yml): test fixture asserting the ledger validator rejects /etc/passwd as an unsafe absolute path; the...
Accepted: External cloudflare/security-audit-skill (vendored via apm.yml): test fixture (unsafe source paths) asserting the findings validator rejects /etc/passwd; the fi...
`prepare` now refuses to remove a skill folder when `.agents`,
`.agents/skills` or the folder itself is a symlink, so it can't delete
outside the checkout. `check` now requires every deployed skill folder to
keep a claimed SKILL.md, so a skill emptied down to its directory fails.
Refs #237
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The check accepts Git-ignored replacement skill files, allowing publication to delete the old folder and update the lock without including replacement content.
A renamed or newly added external skill folder lacks its .gitignore
negation, so the materialize PR would carry the lock entry but not the
files. `check` now asks `git check-ignore` about every lock-deployed path
and names the ignored folder once.
Refs #237
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The check can miss a lost report-schema.json when its tracking entry is also removed, allowing publication of an incomplete skill and preventing the findings validator from loading its schema.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Detect partially missing files when a dependency commit is unchanged
scripts/apm-external-skills.py:126
If installation drops report-schema.json from both disk and deployed_files while retaining the folder and SKILL.md, this existence condition skips its tracked path and check succeeds. Materialize can then publish an incomplete skill even at an unchanged resolved commit, leaving the findings validator unable to load its schema. Retain and compare the pre-install inventory when a dependency's resolved commit is unchanged, and add a regression case for this partial-loss scenario.
If `apm install` drops one file from both disk and `deployed_files`
while the dependency's commit stays the same, the earlier checks passed.
`check` now compares the lock with the one at HEAD and fails on any
deployed path lost at an unchanged resolved commit.
Refs #237
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The committed-tree check can miss deletion of a companion file and its lock entry because PR CI compares against HEAD rather than the PR base revision, leaving a validation gap that needs a baseline fix and regression test.
In PR CI, HEAD already holds the PR's lock, so a PR could drop a locked
companion file and its lock entry together. `check --base REV` compares
with REV's lock instead, and Markdown Validation runs it against the PR
base. An unreadable explicit base fails rather than passing.
Refs #237
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #237
Why #233 deletes the skill
It's not line endings or a Windows-generated lock. The committed
security-auditfiles are LF and their SHA-256 hashes matchapm.lock.yamlexactly.apm-cli 0.35.0 trusts a skill file's overwrite baseline only from per-user deployment receipts (
~/.apm/deployment-receipts/plus.apm-local/installation). It never trusts the committed lock (DeploymentMaterializationSession.baseline). A fresh runner has no receipts. For a root-level skill package,_integrate_native_skillhas no "identical, adopt it" shortcut, unlike agents, which are "adopted". So every committed file is skipped withno trustworthy local deployment baseline. Nothing gets deployed for the package, so the cleanup phase deletes all 21 lock-tracked paths as stale.Upstream tracks this as microsoft/apm#3372. v0.35.1 fails closed instead of deleting. Adopting unchanged committed skills (microsoft/apm#3564, microsoft/apm#3638) is only on
main, not released yet.Fix
APM Materialize: a new
scripts/apm-external-skills.py preparestep removes the external skill folders recorded inapm.lock.yamlbeforeapm install. APM then redeploys them from the pinned commit, and a missing target needs no receipt. Git still holds the committed copy, so the materialize PR diff still shows any upstream change. The review gate is unchanged, andapm-overrides.jsonis still re-applied after the install.Guard:
apm-external-skills.py checkruns after the install and fails the run, rather than opening a PR, in any of these cases:SKILL.md;.gitignorenegation);HEADin Materialize; the PR base in PR CI viacheck --base);.agents/skills/file isn't claimed by the lock.tests/test_apm_external_skills.pyruns the same check against the committed tree on PRs (Markdown Validation) and in lefthook. On chore: materialize APM primitives #233's head it fails withcloudflare/security-audit-skill/skills/security-audit deploys no files.preparerefuses to delete through a symlinked.agentsor.agents/skills.validate-coverage-ledger.cjsandvalidate-findings.cjsregain their upstream100755mode, which the Windows commit in feat: add cloudflare security-audit skill #226 dropped. Otherwise every Linux materialize run would show a mode change.docs/apm.mdexplains the workaround and when to drop it.Testing
Run in separate testing contexts on WSL Debian (aarch64):
HOME, onmain(7e4ff4b):apm install --update --only apmexits 0 but deletes all 20 skill files (Cleaned 21 stale deployed files).preparestep, then the overrides andcheck: the skill is untouched andcheckexits 0. The only lock change isgenerated_at, plus the stalerequire-push-approval.shhash from feat(hooks): let git-push-approval allow feature-branch pushes #230, which is legitimate drift. A second run is deterministic.test_apm_external_skills.py(11 tests),test_apm_overrides.py,validate-prompts.shandcheck-index.shall pass. Manual Linux scenarios also fail as expected: a renamed skill, a partial file loss, a symlinked parent. Every signed commit was made with all lefthook hooks passing on Linux.This PR doesn't touch
apm.yml, so merging it won't trigger APM Materialize. Dispatch it manually, or wait for the next release bump. It should then open a PR with only that lock refresh.