Skip to content

Nightly security sweep: turn Dependabot and code-scanning (CodeQL) alerts into board-tracked issues #2542

Description

@cliffhall

Problem

Security alerts reach this repo from two GitHub sources, and only one of them becomes tracked work:

Source What produces it Swept into issues today?
Dependabot alerts vulnerable dependency ranges in a lockfile Yes, daily (dependabot-alerts.yml, #2233). It files issues, but cannot board them: PROJECT_TOKEN does not exist (#2462).
Code scanning alerts CodeQL (default setup), plus any other tool that uploads SARIF No. Nothing reads them.

CodeQL is not a third source: it is the tool behind the code-scanning alerts on Security → Code scanning. A sweep reads them all through one API, GET /repos/{owner}/{repo}/code-scanning/alerts, and can split by tool.name.

What that gap cost during the v2.9.0 release:

The deeper problem: CodeQL never looks at v2/main

CodeQL runs as default setup (languages: actions, javascript-typescript, schedule: weekly). Default setup analyzes the default branch (main) and PRs that target it. Every analysis on record is refs/heads/main or a PR into main, and the only PRs analyzed are the four milestone merges (#2304, #2381, #2456, #2536).

So no v2 development work is scanned until the milestone merge. A nightly sweep of the alerts cannot fix that on its own: an alert for code on v2/main does not exist until that code reaches a PR against main. It is the same blind spot AGENTS.md already records for Dependabot ("a vulnerable dependency introduced on v2/main and not yet merged to main produces no alert at all"), and for CodeQL it covers every line of first-party code.

Goal

A nightly sweep turns open Dependabot and code-scanning alerts into board-tracked issues, one per actionable finding, with no duplicates, the same way dependabot-alerts.yml does for Dependabot today.

Scanning v2/main itself, so that those alerts exist for v2 code before the release PR, is split out as #2545.

Design sketch

The sweep

Extend scripts/dependabot-alerts.mjs into a security-alert sweep, or add a sibling scripts/code-scanning-alerts.mjs with a shared filing and boarding core. It keeps the existing sweep's invariants:

  • Re-check against v2/main before filing. Dependabot re-checks each vulnerable range against v2/main's lockfile. For code scanning, the equivalent is to take the alert on the v2/main ref once Scan v2/main with CodeQL: switch from default setup to a committed advanced-setup workflow #2545 makes that ref exist. Until then, check that the alert's file and line still hold the flagged code on v2/main, and skip one already fixed there.
  • One issue per finding, deduplicated by a marker that encodes tool + rule + alert number, with markers trusted only on automation-authored issues (the AGENTS.md rule, since the repo is public).
  • Board it directly at Todo / High once the credential exists, the same standing override the Dependabot sweep uses. Without the credential, fall back to the labeled, milestoned triage hand-off.
  • Fail loudly on a bad listing (rate limit, truncated or 5xx response) and write nothing, as dependabot-alerts.mjs's openAlerts() has no error handling for a rate-limited or partially-failed GitHub API response #2425 made the Dependabot sweep do.
  • Never close an issue. A fixed alert closes itself; closing its issue is a maintainer act.
  • No model in any write-capable job. This sweep is deterministic and needs none.

Out of scope unless we choose it: secret scanning (GET …/secret-scanning/alerts). It is a third alert type, would need its own read permission, and its alerts should not be copied into a public issue body. Decide explicitly rather than by omission.

Credential

The GitHub App is now its own issue, #2544, and its permission list there already includes what this sweep needs:

Permission (GitHub App) Needed for Workflow permissions: equivalent
Repository → Dependabot alerts: read GET …/dependabot/alerts vulnerability-alerts: read
Repository → Code scanning alerts: read GET …/code-scanning/alerts security-events: read
Repository → Administration: read (optional) turns the sweep's automated-security-fixes check from UNVERIFIED into a real assertion none: GITHUB_TOKEN cannot hold it
Repository → Secret scanning alerts: read (only if secret scanning is in scope) GET …/secret-scanning/alerts secret-scanning-alerts: read

Both reads are available to GITHUB_TOKEN through workflow permissions:, so the listing half does not strictly need the App. Only the board write does. Putting everything on the App still keeps the sweep to one identity with one scope list, so add the two read permissions when the App is created.

Blocked on

#2544 (the GitHub App), for the board write. Part of tracker #2543.

Acceptance

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedchoreMaintenance: deps, build tooling, CI, cleanup — no user-facing behavior changev2Issues and PRs for v2

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions