Repository navigation
doc: document CodeQL alert triage and re-raised alerts - #1716
Merged
Pedro Henrique Penna (ppenna) merged 1 commit intoNov 17, 2026
Merged
Conversation
A dismissed CodeQL alert does not stay dismissed when its code moves to another file or the code at its location changes: GitHub matches each result to an alert by its rule, the path of its file, and its primaryLocationLineHash fingerprint, so such a result gets a new, open alert. Renaming nvx-rs/ to nvx/ in #1671 replaced alerts 23, 26, and 28 to 32 with alerts 33 to 39 this way, at the same rules and lines; the pull request's own analysis raised them before it merged, and they kept their numbers on dev. Nothing documented how to triage CodeQL alerts, or that this happens. Add a Code scanning section to doc/ci.md. It says how the default setup runs and which checks it adds, and records the convention that the reviews of #1655's alerts followed: an area: security issue, then a fix, or a dismissal with its reason and a comment that cites the issue. It explains why codeql[...] comments and evasive renames are no answer, and how to handle a re-raised alert: dismiss it again in the pull request that raises it, with the original reason and issue, once the original review still holds. A bullet in the Copilot instructions points the agents that make most renames to it. Evidence: dev's analyses at 6594932 (2186479668, 2186487685, and 2186499772) give every result a primaryLocationLineHash and none a suppression. A reimplementation of the CodeQL Action's fingerprint reproduces alert 27's a4d54c21248b79dd:1 from scripts/test_nvx_tools.py at that revision: inserting lines above it or re-indenting it keeps the hash, and editing line 18765 or 18766 changes it. The default-setup API reports the actions, python, and rust languages, the default suite, and a weekly schedule, and dev's rules require only Required status check. No test reads this prose; check_markdown_links.py checks its links. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified for the documentation updates covering CodeQL alert triage and re-raised alerts.
No open issues 🎉
What changed in this PR
This documentation-only PR advances #1655 by documenting NVX’s CodeQL alert triage process and how to handle dismissed alerts that reappear after code changes.
Changes:
- Documents the current CodeQL setup, analysis checks, and relationship to required CI checks.
- Defines evidence-backed triage and dismissal guidance, including private reporting for vulnerabilities.
- Explains alert matching and directs contributors and agents to review re-raised alerts before merging.
| File | Description |
|---|---|
doc/ci.md |
Adds Code scanning guidance covering setup, triage, dismissals, and re-raised alerts. |
.github/copilot-instructions.md |
Adds agent guidance for handling re-raised alerts and links to the CI documentation. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-1655-codeql-reraised-alerts
branch
November 17, 2026 16:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1655
What does this pull request change?
It documents how this repository's CodeQL alerts are triaged, and what to do
when a dismissed alert comes back under a new number. Start with the new
Code scanning section of
doc/ci.md, after Issue link, which:the default suite, on pull requests to
dev, pushes to it, and weekly),which checks it adds (
Analyze (...)andCodeQL), and thatRequired status checkdoes not wait for them;in an
area: securityissue, or privately underSECURITY.mdfor avulnerability, then fix the code, or dismiss the alert with the reason that
fits and a comment that cites the issue. It also says that
codeql[...]andlgtm[...]comments suppress nothing here, and that renaming code only toescape a query is no answer;
and a hash of the first 100 characters, other than spaces and tabs, from the
start of the flagged line), which changes therefore re-raise a dismissed
alert as a new one, and how to handle them: dismiss them again in the pull
request that raises them, with the original reason and issue, once the
original review still holds.
A bullet under Pull Requests And CI in
.github/copilot-instructions.mdpoints the agents, which make most renames, to it.
Left out, and still open in #1655: the choice between default setup and an
advanced workflow (draft #1675 proposes one, and would then update this
section's first paragraph), the Rust coverage limits, how scan results
interact with the merge queue and the required checks, and an owner for new
alerts. Nothing here automates a re-dismissal or suppresses an alert, so a
re-raised alert still needs someone to dismiss it.
Why is this change needed?
A dismissal does not stop GitHub from raising the same false positive again.
GitHub matches each result of an analysis to an alert by its rule, its file's
path, and its
primaryLocationLineHashfingerprint, not by its line number.So a result whose file is renamed, moved, or split, or whose flagged code is
edited, gets a new, open alert, while the dismissed one matches nothing. This
has already happened: #1671 renamed
nvx-rs/tonvx/, and its own analysisreplaced alerts 23, 26, and 28 to 32 with alerts 33 to 39, at the same rules
and lines, which kept their numbers on
devonce it merged. Those alerts werestill open; a dismissal would not have carried over either. Nothing
documented this, or how to triage an alert at all.
Rejected alternatives, from the review of alert 27 in #1667:
# codeql[RULE]comments. Default setup ignores them. Making themwork needs an advanced workflow that adds CodeQL's
AlertSuppression.qlquery and runs
advanced-security/dismiss-alertsondevpushes withsecurity-events: write. That action dismisses as "won't fix" with thecomment "Suppressed via SARIF", and anyone could silence an alert by adding
a comment. Worth revisiting only if ci: enable CodeQL code scanning for Actions, Python, and Rust #1655 moves to an advanced workflow.
advanced-security/filter-sarif, filtering by path and rule. It dropsthe results before upload, which leaves no alert and no record: the silent
suppression that ci: enable CodeQL code scanning for Actions, Python, and Rust #1655 rules out.
query-filtersorpaths-ignore. Too broad, since they disable a rulefor production code too, or every rule for the test files, and they also
need an advanced workflow.
instead of recording its review.
How was this change validated?
Checks and results, on Windows:
python scripts/local_gates.py --base origin/dev --run --keep-goingtest-changes,markdown-links, andtest-layoutpassed.nvx-cliran 2,302 tests, skipping 91, with 6 errors, each asubprocess.TimeoutExpiredafter the 10 seconds that twoMicrovmTestsgive Git'sshon this loaded host: 5 subtests oftest_host_loopback_allow_script_checks_publication_and_trafficand[tcp-missed]oftest_private_network_ingress_script_checks_policy_and_traffic. Both tests passed twice when rerun alone, and the[tcp-missed]timeout also occurs atorigin/dev(742ef7c4) on this hostnvx-cliafter the one that failednvx.py ... --helpchecks passed.scripts/test_adversarial.pyran 84 tests with 1 failure,test_process_timeout_bounds_blocked_stdin_writermeasuring 1,875 ms against its 1,500 ms bound, which passed 3 times when rerun alonepython scripts/check_markdown_links.py, after the last editpython -m unittest scripts/test_tool_versions.py scripts/test_markdown_links.py -vtest_consumers_and_guides_hard_code_no_pincovers both edited filespython scripts/check_test_changes.py --base origin/devNone of the tests that failed locally reads the changed files, and CI ran
them all on this head: run 44112601088
passed, with
NVX CLI / Linux,NVX CLI / Windows,Lint, typing, and formatting, andRequired status check, and theCodeQLcheck found no new alerts.The facts that the section states, checked against
devat659493256316:GET /repos/microsoft/nvx/code-scanning/default-setup: configured;actions,python,rust;default;weeklyAnalyze (actions),Analyze (python),Analyze (rust), andCodeQLcheck runs, the last linking to the branch's open alerts;dev's rules require onlyRequired status checkprimaryLocationLineHashand none asuppressionsentrya4d54c21248b79dd:1fromscripts/test_nvx_tools.py; inserting lines above it or re-indenting it keeps the hash, and editing line 18765 or 18766 changes itnvx-rs/are fixed; alerts 33 to 39 undernvx/were raised onrefs/pull/1671/headbefore #1671 merged, and are the same alerts ondevfalse positiveorused in tests, with a comment that cites #1658, #1659, #1660, #1661, #1662, or #1667Backend, host OS, host type, and guest: none, since the change is
documentation only.
Left to CI: nothing more; it ran the Linux and Windows CLI suites and the
quality job, which passed.
Checklist
python3 scripts/check_test_layout.pychecks.python3 scripts/check_test_changes.py --base origin/devlists no changeto an existing test, or a Changes to existing tests section below
explains each one.
.github/copilot-instructions.mdand the skills when a convention orworkflow changes.
nvxupdatesnvx-rs/public-api/withpython3 scripts/public_api.py --update.description says how the change versions or migrates them.