Add AI skills: analyze-debug-files, find-related-github-issues, trace-code-introduction - #2584
Add AI skills: analyze-debug-files, find-related-github-issues, trace-code-introduction#2584David Paulson (dpaulson45) wants to merge 5 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Adds a Copilot skill that maps a CSS-Exchange script name + version stamp (YY.MM.DD.HHMM) back to the earliest GitHub release tag that shipped that build, by walking releases in ascending tag-date order and matching the File + Version pair in each release's ScriptVersions.csv. Includes: - Find-ReleaseTagForScriptVersion.ps1 helper with structured result (Script, Version, ConfirmedTag, SHA256Hash, Status, WindowExhausted, EarlierGaps, Tried[]) and status enum distinguishing match-earliest, match-possibly-not-earliest, not-found-complete, not-found-inconclusive, and not-found-no-candidates. - SKILL.md documenting the workflow, status semantics, per-candidate Tried statuses, and WorkFolder rejection rules. Security posture: - Repository pinned to github.com/<owner>/<repo> so an inherited GH_HOST cannot redirect requests or leak an ambient enterprise token. - WorkFolder rejects UNC/extended-UNC/provider-qualified paths, non-FileSystem PSDrives, network/CD-ROM/unknown drive types, SUBST/raw-DOS-device aliases (via QueryDosDevice), and paths whose volume root or any existing ancestor is a filesystem reparse point. - Downloaded CSVs go through strict header, per-row, and per-field validation; File comparison is ordinal, and Version/SHA256 formats are regex-checked before any value is surfaced. - All caller-visible strings from downloaded content are size-capped and stripped of control characters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds a Get-CommitShaForTag helper that resolves the matched release tag to its target commit SHA via the GitHub API, and surfaces it on the result object as ConfirmedCommitSha alongside ConfirmedTag. The resolver pins to github.com and validates the SHA shape before returning; any failure yields $null rather than throwing so the primary tag match is never blocked. Downstream skills (analyze-debug-files) require a 40-character SHA to build a scratch worktree deterministically and to key the per-SHA dependency cache; a tag alone is ambiguous once tags are re-pointed. Also adds five cspell dictionary entries (metacharacters, misattributed, misrouting, triaging, worktree) used across the new skills. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New Copilot skill that, given an exception message and script name, searches microsoft/CSS-Exchange issues for prior reports of the same failure. Invoked directly when triaging a fresh exception, or as a Step 7 sub-invocation of analyze-debug-files after each unhandled finding. Provides: - SKILL.md contract: input shape, output shape, scoring criteria, hard cap on results returned to the caller. - Find-RelatedGitHubIssues.ps1: sanitizes user-controlled exception content against gh CLI argument injection (Get-SafeQueryPhrase strips quotes, search-syntax metacharacters, backticks, newlines), runs 3-4 randomized query variants, and deduplicates by issue number before returning ranked candidates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New Copilot skill that traces a specific source line back to the commit that introduced it. Used by analyze-debug-files Step 7 to determine whether an unhandled exception was authored inside the failing script's own history (own-repo introduction) or inherited from a Shared/ file. Provides: - SKILL.md contract: input (repo-relative path + line range + SHA), output shape, caveat that git log -L reports the whole enclosing function/block so sibling statements added later can be misattributed as the introducing commit. - Trace-CodeIntroduction.ps1: validates repository and path against a strict allow-list (no shell metacharacters, no traversal, no absolute or UNC paths), invokes git log -L and git blame against the pinned worktree, and returns a structured record with commit SHA, author, date, and message. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New Copilot skill that analyzes CSS-Exchange debug log files, identifies the source script + release-tag baseline, and produces a root-cause report for unhandled exceptions. Pipeline: - Step 1a: streaming inventory (Get-DebugFileMetadata.ps1) — detects CSS-Exchange output shape; hard-stops if the file is not a recognized script log. - Step 2-4: identifies the script + version and confirms the release tag + commit SHA via find-release-tag-for-script-version. - Step 5: obtains the dependency graph. Loads from a per-SHA cache at $env:LOCALAPPDATA\CSS-Exchange\dependency-cache\<sha>\ when available; otherwise materializes a scratch worktree, runs .build/Build.ps1, and populates the cache. Cache hit avoids the ~107-second Build.ps1 run (measured 74x speedup on the primed path). XML keys are normalized from absolute worktree paths to repo-relative form so Steps 6-8 can read source with git show <sha>:<path> regardless of which branch produced the XML. - Step 6-7: per-finding source reads and BFS across the dependency graph, with optional sub-invocations of trace-code-introduction and find-related-github-issues. - Step 8: renders DebugAnalysis-<timestamp>.md into the caller-supplied directory. Includes STRICT source-slice fidelity rules and a post-render assertion (regex-enforced format, forbidden ellipses, consecutive-line check) so cited source cannot be silently truncated or paraphrased. Trust model: personal machine, no ownership preflight, report written directly to the caller-supplied directory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
3188dce to
dd9382a
Compare
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical and moderate findings remain.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds three Copilot skills for CSS-Exchange debug-log triage, GitHub issue correlation, and code-introduction tracing, plus confirmed commit-SHA support for release lookup.
Changes:
- Adds debug-log metadata parsing, dependency caching, and report generation.
- Adds related GitHub issue search and source-history tracing.
- Extends release resolution with
ConfirmedCommitShaand updates spelling data.
File summaries
| File | Description |
|---|---|
.github/skills/trace-code-introduction/Trace-CodeIntroduction.ps1 |
Traces source ranges to introducing commits. |
.github/skills/trace-code-introduction/SKILL.md |
Documents code-introduction tracing. |
.github/skills/find-release-tag-for-script-version/SKILL.md |
Documents confirmed commit-SHA output. |
.github/skills/find-release-tag-for-script-version/Find-ReleaseTagForScriptVersion.ps1 |
Resolves release tags and commit SHAs. |
.github/skills/find-related-github-issues/SKILL.md |
Documents related-issue lookup. |
.github/skills/find-related-github-issues/Find-RelatedGitHubIssues.ps1 |
Searches and classifies related issues. |
.github/skills/analyze-debug-files/SKILL.md |
Defines the end-to-end analysis workflow. |
.github/skills/analyze-debug-files/Get-DebugFileMetadata.ps1 |
Parses and sanitizes debug logs. |
.build/cspell-words.txt |
Adds skill-specific dictionary terms. |
Review details
Suppressed comments (12)
.github/skills/analyze-debug-files/Get-DebugFileMetadata.ps1:555
- The sanitizer removes C0 controls and DEL, but it leaves the C1 range U+0080–U+009F. Those characters can be returned in snippets and violate the report's explicit no-stray-C0/C1 assertion. Extend the character class to cover
\x7F-\x9F.
'[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]',
.github/skills/analyze-debug-files/Get-DebugFileMetadata.ps1:672
- The regex-timeout fallback uses
code -ge 0x20as its printable test, which also accepts C1 controls U+0080–U+009F. A timeout therefore bypasses the normal sanitizer's intended output guarantee and can still place those controls in the report. Use the same printable range as the normal path here.
if (($code -ge 0x20 -and $code -ne 0x7F) -or $code -eq 0x09) {
.github/skills/analyze-debug-files/Get-DebugFileMetadata.ps1:992
- HealthChecker's
Get-ErrorsThatOccurredalso emits----Errors that occurred that was not handled remotely----after the normal unhandled footer (seeDiagnostics/HealthChecker/Helpers/Get-ErrorsThatOccurred.ps1:37-40). This state machine only recognizes the handled/unhandled headers, so a run with hidden remote errors is still markedSummaryCompletewithUnhandledCount=0, causing the analyzer to report a clean run while silently dropping those errors. Add a remote-section signal/count or downgrade completion when that header is present.
} elseif ($Script:UnhandledSummaryHeaderRegex.IsMatch($line)) {
$summaryState = 'unhandled'
$summaryUnhandledCount = 0
$unhandledHeaderLine = $lineNumber
$unhandledHeaderCount++
.github/skills/analyze-debug-files/Get-DebugFileMetadata.ps1:417
- The logger formats
[System.DateTime]::Nowwith the machine's culture (Shared/LoggerFunctions.ps1:60-63), so the common en-US output includes anAM/PMsuffix. This regex and theAcceptedTimestampFormatsbelow reject that form, leaving$tsnull on every timestamped line; version candidates, completion signals, summary events, and body-evidence correlation then become unavailable for ordinary HealthChecker logs. Parse the logger's culture-dependent format (or add the 12-hour forms) consistently to the framing regex and accepted formats.
'\A\s*\[(?<ts>[0-9]{1,2}/[0-9]{1,2}/[0-9]{4}\s+[0-9]{1,2}:[0-9]{2}:[0-9]{2}(?:\.[0-9]+)?)\]',
.github/skills/analyze-debug-files/Get-DebugFileMetadata.ps1:490
- This explanation cites
Write-Host, butGet-ErrorsThatOccurred.ps1emits both summary headers withWrite-Verbose. The headers are untimestamped because the message starts with CRLF and the logger prefixes the timestamp before those newlines; documenting the actual mechanism is important because this comment justifies the anchored header regex.
# cannot forge an authoritative summary. HealthChecker emits these as
# untimestamped lines (the writers in Get-ErrorsThatOccurred.ps1 use
# `Write-Host`, not `Write-Verbose`), so anchor to line start/end.
.github/skills/analyze-debug-files/SKILL.md:576
- The suffix loop does not actually require a unique match: an absolute key can end with multiple repository paths, and the first hashtable key wins. For example, a key ending in
.../Shared/OutputOverrides/Write-Error.ps1can also match a shorter nested path if one exists, causing the dependency graph to resolve to the wrong source file. Collect suffix matches and accept only the unique/longest path (or reject ambiguity).
if ($winKey.EndsWith($k, [System.StringComparison]::OrdinalIgnoreCase)) {
return $Index[$k]
.github/skills/analyze-debug-files/SKILL.md:479
- A cache miss can be caused by an existing corrupt or incomplete
$cacheDir(for example, invalid metadata). This branch deletes the newly built temp directory whenever the final directory exists, leaving the bad entry in place, so every subsequent run rebuilds and remainsBuildOnly. Distinguish a valid concurrent winner from a stale invalid entry and replace or quarantine the latter.
if (Test-Path -LiteralPath $cacheDir -PathType Container) {
Remove-Item -LiteralPath $tempDir -Recurse -Force -ErrorAction SilentlyContinue
} else {
Move-Item -LiteralPath $tempDir -Destination $cacheDir
$materializationSource = 'BuildAndCached'
.github/skills/analyze-debug-files/SKILL.md:1041
find-related-github-issuesdefinesPartialLookupas usable retained results with incomplete coverage, not as an unavailable lookup. Treating every non-Okstatus as unavailable would hide those matches from the report and contradict the sibling skill's contract; render retained results plus an incompleteness note forPartialLookup, reserving "unavailable" for the failure statuses.
**Failure modes.** Both skills return a `Status` field. When
`Status -ne 'Ok'`, render a single-line "lookup unavailable" note in
the corresponding subsection and continue with the report. A failing
provenance lookup MUST NOT abort the report.
.github/skills/analyze-debug-files/SKILL.md:437
- This invocation is still subject to the caller's
$PSNativeCommandUseErrorActionPreference. When that preference is$trueandBuild.ps1exits nonzero—the condition this step explicitly says may be cosmetic—PowerShell throws before the XML existence checks, so the documented BuildOnly/cache flow aborts instead of using the generated XML. Disable native-error promotion only around this invocation and restore the caller's value infinally, or invoke the process through an API that captures the exit code without throwing.
& pwsh -NoProfile -File (Join-Path $worktreeRoot '.build\Build.ps1')
.github/skills/find-related-github-issues/Find-RelatedGitHubIssues.ps1:88
Mandatorydoes not reject whitespace-only input. In that case normalization produces no query, the final status falls through toOkwith empty results, and callers can incorrectly report that no related issue exists instead of an invalid/failed lookup. Reject an empty or whitespace-only top-level exception before building the query list.
[string]$TopLevelException,
.github/skills/find-related-github-issues/Find-RelatedGitHubIssues.ps1:128
- These replacements create literal placeholder tokens, but the subsequent exact GitHub query is built from the normalized phrase. For
Failure at C:\a\b, the query becomes"Failure at path"after delimiter stripping; GitHub does not treatpathas a wildcard, so an issue sayingFailure at C:\other\dis never returned and normalization cannot provide the advertised stable matching. Build search phrases that omit volatile spans or use unquoted invariant terms, while keeping normalization for local classification.
$s = [regex]::Replace($s, '[A-Za-z]:\\[^\s"'']+', '<path>')
# Strip UNC paths.
$s = [regex]::Replace($s, '\\\\[^\s"'']+', '<unc>')
# Strip randomized temp module names like tmpEXO_3fzjpepe.o0p
$s = [regex]::Replace($s, 'tmpEXO_[A-Za-z0-9]+(?:\.[A-Za-z0-9]+)?', 'tmpEXO_<random>')
.github/skills/find-release-tag-for-script-version/Find-ReleaseTagForScriptVersion.ps1:358
- For a valid empty
gh release listresponse ([]),ConvertFrom-Jsoncan produce$null(especially on Windows PowerShell 5.1).@($parsedReleases)then creates an array containing one null element, so the loop below throwsUnexpected release entry shape from ghinstead of reaching thenot-found-no-candidatesresult. Preserve$nullas an empty array before iterating.
$releases = @($parsedReleases)
- Files reviewed: 9/9 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| $Script:ErrorIndexRegex = [regex]::new( | ||
| # Iter-23 (RD-branch-10): require a strict timestamp shape | ||
| # (matching $Script:TimestampRegex) inside the brackets. See | ||
| # SummaryFooterRegex above for rationale. | ||
| '\A\s*\[[0-9]{1,2}/[0-9]{1,2}/[0-9]{4}\s+[0-9]{1,2}:[0-9]{2}:[0-9]{2}(?:\.[0-9]+)?\]\s*:\s*Error\s+Index\s*[:=]', |
| try { | ||
| # FileShare.ReadWrite so a still-running writer (rare for | ||
| # post-run debug artifacts but supported) does not lock us out. | ||
| $stream = [System.IO.File]::Open($FileInfo.FullName, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite) |
| if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($originUrl)) { | ||
| throw "Could not read local git origin URL." | ||
| } | ||
| if ($originUrl -notmatch '(?i)github\.com[:/]+microsoft/CSS-Exchange(\.git)?$') { |
| if ($blameOk) { | ||
| # Porcelain first line of each entry is `<sha> <origLine> <finalLine> [<groupSize>]`. | ||
| foreach ($ln in $blameOut) { | ||
| if ($ln -is [string] -and $ln -match '\A([0-9a-fA-F]{40})\s+\d+\s+\d+') { |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Summary
Adds three new Copilot AI skills for CSS-Exchange debug-log triage, plus a supporting extension to the existing
find-release-tag-for-script-versionskill.New skills
analyze-debug-files— end-to-end pipeline that ingests a CSS-Exchange debug log, identifies the source script + release-tag baseline, walks the dependency graph via.build/Build.ps1output, and writes a root-cause report (DebugAnalysis-<timestamp>.md) into the caller-supplied directory.find-related-github-issues— searchesmicrosoft/CSS-Exchangeissues for prior reports of a given exception. Invocable directly during triage or as a Step 7 sub-invocation ofanalyze-debug-files.trace-code-introduction— resolves a specific source line + range at a pinned SHA to the commit that introduced it (viagit log -Landgit blame). Used byanalyze-debug-filesto attribute unhandled exceptions to their originating change.Extension
find-release-tag-for-script-versionnow emits aConfirmedCommitShafield alongsideConfirmedTag.analyze-debug-filesneeds the 40-character SHA to build a scratch worktree deterministically and to key its per-SHA dependency cache; a tag alone is ambiguous once tags are re-pointed.Notable design decisions
%LOCALAPPDATA%\CSS-Exchange\dependency-cache\<sha>\. On cache hit,analyze-debug-filesskips the ~107 s.build/Build.ps1invocation (measured ~74× speedup). The cache is worktree-independent — XML keys are normalized from absolute worktree paths to repo-relative paths so Steps 6-8 can read source withgit show <sha>:<path>regardless of which run produced the cache.# L<n>trailing annotations, no ellipses, consecutive-line check) with a post-render regex assertion so cited source cannot be silently truncated or paraphrased.ghCLI arguments, git pathspecs, and shell-metacharacter surfaces before invoking subprocesses (seeGet-SafeQueryPhrase, allow-list regexes inTrace-CodeIntroduction.ps1).Testing
.build\SpellCheck.ps1— clean (0 issues, 810 files).-ForceLegacyscenario). Cache miss + cache hit both verified; normalized XML keys resolve viagit show <sha>:<path>.Commits
find-release-tag-for-script-versionwithConfirmedCommitSha+ cspell dictionary additions.find-related-github-issuesskill.trace-code-introductionskill.analyze-debug-filesskill.