Skip to content

Customize Copilot code review with org-wide and per-repo standards #41

Description

@gavinbee

Problem

Copilot code review runs on our PRs against generic defaults rather than SwimBlocks standards
(ruff rule set, venv standard, fixture pseudonymization, PR sizing, design-doc workflow). We
want an org-wide baseline that every repo inherits, plus per-repo layering on top.

Constraints

  • Code review on github.com reads .github/copilot-instructions.md,
    .github/instructions/**/*.instructions.md (applyTo globs), and AGENTS.md (nearest in
    tree wins). All applicable sets are combined, not exclusive.
  • Instructions are read from the head branch, so a PR that changes them is reviewed under them.
  • Copilot does not follow links out of an instruction file. Our current chain
    (copilot-instructions.mdAGENTS.mdCONTRIBUTING.mdswimblocks/.github) breaks at
    the cross-repo hop. Content must physically exist in each repo.
  • The .github repo's community-health inheritance does not cover Copilot instruction files.
  • Org-level instructions require Copilot Business/Enterprise. Org is on free with 0 seats and
    seat_management_setting: unconfigured — unavailable to us. Also ~4000 chars and not
    version-controlled.
  • AGENTS.md is honored by code review on github.com only; copilot-instructions.md is honored
    on all platforms including IDE review.

Current state

Repo copilot-instructions.md AGENTS.md .github/instructions/
officials-admin
.github yes
swim-club-tech-survey yes yes
rems-sync yes yes
deck-eval-gen yes yes
deck-eval-parser yes yes
rems-sync-apps-script

The four existing copilot-instructions.md files are thin pointers to AGENTS.md and carry no
content of their own.

Plan

  1. Write the canonical org-wide review criteria in this repo — review rules, not authoring
    prose. Covers ruff rule set (Ruff selection is documented org-wide but enforced per-repo, inconsistently #38), venv standard, fixture pseudonymization (CONTRIBUTING: test fixtures must use synthetic names; document the multi-pattern PII scrub #17), PR sizing
    (PR-sizing and progress-PR guidance in CONTRIBUTING.md and elsewhere is not sufficiently clear #7), design-doc workflow (Add design-doc workflow to org-wide CONTRIBUTING.md and AGENTS.md #23), squash-only / PR body conventions. Keep under ~2 pages.
  2. Sync that block into .github/copilot-instructions.md in every org repo via the reconciler
    (scripts/apply-settings.py, reconcile-repo-defaults.yml), inside begin/end fences so
    per-repo content below the fence survives reconciliation. Depends on reconcile-repo-defaults workflow is disabled, so new repos miss org settings #34.
  3. Backfill AGENTS.md + copilot-instructions.md in officials-admin and rems-sync-apps-script.
  4. Add .github/instructions/*.instructions.md where a repo has genuinely path-scoped
    conventions (parser fixtures, Apps Script, migrations).
  5. Validate on a real PR.

Decisions

  • copilot-instructions.md holds review criteria; AGENTS.md stays authoring-focused. They
    answer different questions, excludeAgent frontmatter exists for the split, and only
    copilot-instructions.md works for IDE review.
  • Not buying Copilot Business seats for org-level instructions. The file approach is
    version-controlled and reviewable; the org settings box is neither.

Open

  • Does the reconciler own file content, or only settings? Expanding its remit likely warrants a
    design doc under docs/design/.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions