Skip to content

Close only CI's own LabVIEWs and harden the CI scripts and workflow templates - #46

Merged
JanGoebel merged 9 commits into
mainfrom
ci-runner-safety
Oct 30, 2026
Merged

JanGoebel merged 9 commits into
mainfrom
ci-runner-safety

Conversation

@JanGoebel

Copy link
Copy Markdown
Owner

Summary

Review findings on the CI tooling. The main one: with -CleanStart CI force-closed every running LabVIEW on the runner whenever a VIPM Desktop was open. The runner is the development machine, so that included the maintainer's own LabVIEW and anything unsaved in it. Now CI closes only the test and comparison LabVIEWs it chose for the job, and none with unsaved changes. Alongside: the workflow templates get least-privilege tokens, pinned actions and a release guard, the report scripts stop ignoring git and gh failures, and LabVIEW and Caraya text is escaped in the reports. No VI is changed.

Before After
VIPM Desktop open at job start every running LabVIEW on the machine force-closed only the job's test and comparison LabVIEWs, and none with a * (unsaved changes) in a window title unless CI_DEDICATED_MACHINE=1
LabVIEW on the wrong VI Server port (-Fix) force-closed same rule: kept when it has unsaved changes, LabVIEW.ini then not written while it runs
Release from a v* tag any tagged commit, no approval, whole workflow with contents: write only commits on main, job in the release environment, write access only for that job
Pull request checkouts token persisted in .git/config while the PR's code runs persist-credentials: false; only the publishing step gets GH_TOKEN
Setup or package install fails protocol tests and VI diff run anyway with empty LABVIEW_* they are skipped and reported as "not run"; the last step says why
git or gh call fails in the report scripts mostly ignored (a failed git diff looked like "no VI changed") the step fails with the git/gh error
Report comment lookup first 100 comments, any comment with the marker overwritten all pages, only the comment with the marker written by the same user
|, <, >, @ in labels and LabVIEW texts broke table rows, vanished as tags, mentioned users shown as written
Caraya assert level deepest indent: one deeper line hid all tests; assert labels cut at " - " by the report's structure; full labels

Changes

All files are PowerShell scripts, the two workflow templates and docs. .github/workflows/ is not touched.

  • tools/ci/LabVIEWCi.psm1 (modified)
    • Select-LabVIEWToStop decides which running LabVIEWs may be closed, with no side effects: only paths the job uses, none with * in a window title unless CI_DEDICATED_MACHINE=1. Get-ProcessWindowTitle reads the titles of all visible top-level windows of a process (EnumWindows), Get-RunningLabVIEW and Stop-CiLabVIEW apply it, Test-CiDedicatedMachine reads the variable.
    • Stop-LeftoverVipm and Resolve-ViServerPort -Fix use that rule; their doc comments say that they force-close.
    • Invoke-NativeCommand (one wrapper for console tools, replacing four copies), Invoke-Utf8Command and Invoke-Git (UTF-8 output, Invoke-Git throws on a non-zero exit code), ConvertFrom-GitNameStatus (parses git diff -z --name-status), Get-VipmPath, Get-GhPath.
    • ConvertTo-MarkdownText (escapes & < > |, breaks @ mentions with a zero-width space) and Format-MarkdownCodeBlock (fence longer than any backtick run in the text).
    • Get-AnsiEncoding replaces [Text.Encoding]::Default for LabVIEW.ini and VIPM's Settings.ini (GetEncoding(0), 1252 as fallback); Add-GitHubFileText writes GITHUB_ENV/PATH/summary as UTF-8 without BOM.
  • tools/ci/Resolve-CiEnvironment.ps1 (modified): the clean start runs after the test and comparison LabVIEWs are chosen and passes only those two. Closing the test LabVIEW follows the same rule, and the VIObjCache is kept while that LabVIEW still runs. Uses the shared lookups and UTF-8 writes.
  • tools/ci/New-ViDiffReport.ps1 (modified): git worktree prune first; every git call checked; file list from git -c core.quotePath=false diff -z; LabVIEW's difference texts, frame captions and folder names escaped (tags dropped); CLI output in a safe fence; -LiteralPath.
  • tools/ci/Publish-PrReport.ps1 (modified): every git call checked; the report commit is built with worktree add --no-checkout, write-tree and commit-tree, so no temporary branch exists; it is pushed by its id. When the checkout has no credentials, the push sends GH_TOKEN as the AUTHORIZATION header actions/checkout would store, through GIT_CONFIG_COUNT/KEY_0/VALUE_0 of that one git call (not on a command line, not in a config file). When a header is already configured, or for a local run without GH_TOKEN, git pushes as before. gh exit codes are checked on every call; comments are read with gh api --paginate (REST: page by page); only a marked comment of the current user (github-actions[bot] in Actions, whose token cannot read /user) is edited.
  • tools/ci/Convert-CarayaReport.ps1 (modified): parses the report by its structure (callers, then tests at one depth with their asserts one level deeper; assert level = the depth most lines share, deeper lines belong to the current test; only test lines are split into label and VI), escapes cells, safe fence for the protocol output, reads UTF-8 with -LiteralPath, accepts not-run for both statuses.
  • tools/ci/Invoke-CliTest.ps1, tools/ci/Test-InstalledPackage.ps1, tools/setup/Install-Dependencies.ps1 (modified): shared wrapper and lookups, -LiteralPath. Test-InstalledPackage.ps1 writes its initialize request to a temp file of its own and deletes it, and finds vipm under ProgramW6432. Install-Dependencies.ps1 keeps a VIPM_DESKTOP_LIVELINESS_TIMEOUT that is already set.
  • tools/ci/release.workflow.yml (modified template): see the exact differences below.
  • tools/ci/pull-request.workflow.yml (modified template): see below.
  • tools/ci/README.md, tools/setup/README.md (modified): port resolution as Resolve-ViServerPort does it (VIPM's port first); git restore -- tests after a mass compile, with "check git status first", instead of git checkout -- .; the release template described as it is; the isolation claim corrected (the PR's VIs and tester run on the same runner; its own scripts are used while main lacks them); CI_DEDICATED_MACHINE documented.

No VI changed, so there is no code summary and no VI diff report.

Differences between the templates and the live workflows

tools/ci/release.workflow.yml against .github/workflows/release.yml:

  1. Top-level permissions: contents: read (was contents: write).
  2. build-and-release: environment: release and permissions: contents: write.
  3. actions/checkout@v4 → @11d5960a326750d5838078e36cf38b85af677262 # v4.4.0 (both jobs), actions/upload-artifact@v4 → @ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2, softprops/action-gh-release@v2 → @3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2.6.2. These are the commits v4, v4 and v2 point to today (looked up with gh api repos/<owner>/<repo>/git/ref/tags/<tag>; all three are lightweight tags on commits), so the action code is unchanged.
  4. build-and-release checkout: fetch-depth: 0.
  5. New step "Check that the Tag Is on main" (only for v* tags): fetches main, fails unless git merge-base --is-ancestor HEAD origin/main.
  6. Build step: TAG_NAME: ${{ github.ref_name }} in env:, the script reads $env:TAG_NAME (was "${{ github.ref_name }}"); [Console]::OutputEncoding set to UTF-8 before the git calls; a failed tag fetch stops the step; tag text from %(contents:subject)%0a%0a%(contents:body), trimmed (was %(contents)); version pattern '^v(\d+)\.(\d+)\.(\d+)(?:\.(\d+))?$' (was unanchored "v?(\d+)...").
  7. Release notes: every $ doubled before -replace (this was already in the template on main, not in the live file).
  8. VIP_PATH written with [IO.File]::AppendAllText as UTF-8 (was echo ... >> $env:GITHUB_ENV).
  9. Comments added or reworded at the changed places; lines that contained only spaces are now empty.

tools/ci/pull-request.workflow.yml against .github/workflows/pull-request.yml:

  1. Header comment on env: use and UTF-8 writes.
  2. All four checkouts pinned to 11d5960… # v4.4.0 with persist-credentials: false; upload-artifact (both) pinned to ea165f8… # v4.6.2.
  3. Step ids setup, install, recheck on the three setup steps.
  4. "Check that Every Test VI Is Called" step (Test-TestsWired.ps1), and steps.wired.outcome in the last step. Already in the template on main, not in the live file.
  5. status= lines to GITHUB_OUTPUT with [IO.File]::AppendAllText (was >>, UTF-16 in Windows PowerShell).
  6. Run Protocol Tests: if: ${{ !cancelled() && steps.recheck.outcome == 'success' && steps.tests.outputs.status != '2' }} (was always() && ...).
  7. Build Test Report: if: ${{ !cancelled() }}; outcomes and outputs through env:; a skipped step is passed as not-run; summary appended as UTF-8.
  8. Build VI Diff Report: if: ${{ !cancelled() && steps.setup.outcome == 'success' && steps.recheck.outcome != 'failure' }}; base branch through env: BASE_REF.
  9. Upload Reports and Post Report: if: ${{ !cancelled() }}. Post Report gets PR_NUMBER, HEAD_SHA, DIFF_OUTCOME through env: and builds the run URL from GITHUB_SERVER_URL/GITHUB_REPOSITORY/GITHUB_RUN_ID; when the diff was skipped, the footer starts with "VI diff: not run".
  10. Last step: if: ${{ !cancelled() && (setup/install/recheck not success || tests/protocol failure || diff not success || wired failure) }}, all values through env:, an extra error line for failed setup steps and for a diff that did not run, "not run" instead of "skipped".
  11. test-build: if: github.event.pull_request.head.repo.full_name == github.repository.
  12. test-build build step: PR_NUMBER, HEAD_SHA through env:, run number from $env:GITHUB_RUN_NUMBER; summary and VIP_PATH written as UTF-8.
  13. "Test Installed Package" step (Test-InstalledPackage.ps1), already in the template on main, not in the live file; its summary is now written as UTF-8 inside a four-backtick fence.
  14. Comments added or reworded at the changed places; newline at the end of the file.

Verified

All on this machine, Windows PowerShell 5.1.26100, without LabVIEW and without stopping any process:

  • Every .ps1/.psm1 in the repository parsed with [System.Management.Automation.Language.Parser]::ParseFile: 0 errors. All changed scripts are ASCII.
  • Every run: block of both templates extracted and parsed with the same parser: 14 + 9 blocks, 0 errors. No ${{ }} left inside any run: block, every uses: pinned to a 40-character SHA with a version comment, no tabs, final newline.
  • Select-LabVIEWToStop with mocked processes: clean job LabVIEW stopped, unsaved one kept (reason names the window), LabVIEW outside the job kept also with CI_DEDICATED_MACHINE=1, path compare ignores case, CI_DEDICATED_MACHINE only for 1 (9 + 3 checks).
  • Stop-LeftoverVipm with Get-Process/Stop-Process replaced inside the module (guarded so the test aborts if a mock is not active, process ids that cannot exist on Windows): stops exactly VIPM and the clean job LabVIEW, keeps the unsaved and the developer LabVIEW, notes the kept one.
  • Resolve-ViServerPort -Fix with mocks: unsaved LabVIEW not stopped, LabVIEW.ini not written, running port used; with CI_DEDICATED_MACHINE=1 and for a clean LabVIEW stopped and the port written; without -Fix unchanged (10 checks).
  • Get-ProcessWindowTitle: the EnumWindows type compiles in PowerShell 5.1 and reads a process's windows (read only).
  • Invoke-NativeCommand: stdout and stderr as text, exit code, no stop with ErrorActionPreference = Stop, missing program gives -1 and a "not recognized" line.
  • Get-AnsiEncoding gives the same code page (1252) as Encoding.Default did; LabVIEW.ini round trip keeps a non-ASCII line; Add-GitHubFileText writes UTF-8 without BOM.
  • ConvertTo-MarkdownText, Format-MarkdownCodeBlock, ConvertFrom-GitNameStatus (also on a real scratch repository with a rename and a file named with ü, [1], ' and @), Invoke-Git throws on a failing call.
  • Publish-PrReport.ps1 functions taken from its syntax tree, GET requests only, against PR Show every case frame of changed VIs in the detailed diff report #29 with pages of 2: gh --paginate and REST paging return the same comments; author lookup through gh and REST, fallback to github-actions[bot] only inside Actions; marker plus author filter; push header passed through GIT_CONFIG_* and restored, not doubled when the checkout already has one, not used without GH_TOKEN (git push replaced by a recorder).
  • Publish-PrReport.ps1 run end to end offline: local bare repository as origin, gh replaced by a stub that only records its arguments. Pushed a commit without parent containing README.md, images/..., report.md to pr-assets/pr-7; no local branch or worktree left; checkout clean; gh called for user, --paginate and one POST.
  • Convert-CarayaReport.ps1 on the most recent real report of this machine (64 tests): same tests and results as the version on main; differences are the full assert labels (main cut them at " - ") and the escaping. On crafted reports: one deeper assert line no longer hides tests, | < > @ escaped in labels and checks, not-run for both statuses, fence longer than backticks in the protocol output.
  • Release step on a scratch tag object with a signature block: the new format gives subject and body without the signature (%(contents) included it), ü read correctly with the UTF-8 console; the anchored pattern matches v1.2.3 and v1.2.3.4 and not test-1.2.3 (the old one did).

Not verified

  • No workflow ran: the templates are not active, and no CI script was run for real (Resolve-CiEnvironment.ps1, New-ViDiffReport.ps1, Invoke-CliTest.ps1, Test-InstalledPackage.ps1, Publish-PrReport.ps1 against GitHub). The push with GH_TOKEN as header has not been tried against GitHub.
  • The YAML of the templates was not parsed with a YAML parser: none is installed here (no PyYAML, ruamel, js-yaml or ruby). Only the structural checks above.
  • No LabVIEW was involved: whether LabVIEW 2020/2026 marks unsaved changes with * in every window title, and that the check sees a real LabVIEW's windows, was not observed.
  • New-ViDiffReport.ps1 escaping was not run against a real LabVIEW comparison report.
  • PowerShell 7 is not installed; GetEncoding(0) there is not tested.
  • The release environment, the tag guard and !cancelled() behaviour on GitHub.

Known rough edges

  • A test or comparison LabVIEW without unsaved changes is still force-closed, and VIPM Desktop is still ended at the clean start. Unsaved changes in a VI whose window is not open cannot be seen; a * in a title for another reason keeps that LabVIEW running (the safe side).
  • Test-InstalledPackage.ps1 still closes the test LabVIEW at its end (LabVIEW CLI CloseLabVIEW, then Stop-Process) without the unsaved check; it is documented as "do not point it at a LabVIEW someone works in" and was not in scope.
  • %(contents:subject) folds the first paragraph of a tag message into one line. A tag message whose first lines are not followed by a blank line gets them joined.
  • The guard runs for v* tags only; test-* tags build only a test package. Because the environment is set on the whole job, test-* builds also wait for approval.
  • The release template's test job and its checkouts still persist credentials (not part of the findings).
  • git worktree prune also runs when the scripts are used locally; it only removes worktree entries whose folders no longer exist.
  • Gating: a failed package install still lets the VI diff run (diagrams then show ? for missing subVIs), but not the tests and protocol tests.

Maintainer steps

  1. Merge this pull request first: the templates rely on the new scripts on main (not-run, the push with GH_TOKEN). The current live workflows keep working with the new scripts.
  2. Create the environment release (Settings > Environments) with yourself as required reviewer, and optionally a deployment rule for tags v*. Do this before step 3: a workflow that names a missing environment creates it without protection.
  3. Copy both templates on main:
    git switch main; git pull
    Copy-Item tools\ci\pull-request.workflow.yml .github\workflows\pull-request.yml
    Copy-Item tools\ci\release.workflow.yml .github\workflows\release.yml
    git add .github\workflows\pull-request.yml .github\workflows\release.yml
    git commit -m "Activate the hardened CI workflows"; git push
  4. Add a tag ruleset for v* (Settings > Rules > Rulesets, target tags, v*): restrict creation, update and deletion, with only yourself in the bypass list.
  5. Optional: on a runner nobody works on, set CI_DEDICATED_MACHINE=1 in the runner's .env file and restart the runner. Leave it unset on the development machine.

🤖 Generated with Claude Code

CalmyJane and others added 9 commits October 25, 2026 21:35
Resolve-CiEnvironment.ps1 -Fix -CleanStart passed every installed LabVIEW
to Stop-LeftoverVipm. Whenever a VIPM Desktop was open at the start of a
job, every running LabVIEW on the machine was force-closed, including the
maintainer's development LabVIEW on the runner machine, with whatever was
unsaved in it.

Now the clean start runs after the test and comparison LabVIEWs have been
chosen and only ever closes those two. Of these, a LabVIEW one of whose
windows has a "*" in its title (LabVIEW's mark for unsaved changes) is left
running, with a note in the job summary, unless the runner's environment
has CI_DEDICATED_MACHINE=1. The same rule applies where else CI force-closes
a LabVIEW: closing the test LabVIEW at the clean start, and
Resolve-ViServerPort -Fix closing a LabVIEW that does not listen on its
port (which Invoke-CliTest.ps1 and New-ViDiffReport.ps1 also call). When
that LabVIEW is kept, its LabVIEW.ini is not written while it runs, and the
compiled object cache of a test LabVIEW that is still running is kept.

New functions in LabVIEWCi.psm1: Select-LabVIEWToStop (the decision, with
no side effects), Get-ProcessWindowTitle (titles of all visible top-level
windows of a process), Get-RunningLabVIEW, Stop-CiLabVIEW and
Test-CiDedicatedMachine. The doc comments now say that these LabVIEWs are
force-closed. tools/setup/README.md documents CI_DEDICATED_MACHINE.

A test or comparison LabVIEW without unsaved changes is still closed, and
VIPM Desktop is still ended at the clean start; unsaved changes in a VI
whose window is not open cannot be seen from outside.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four copies of the same code ran a console program with stderr merged
into its output while keeping Windows PowerShell from stopping on the
stderr lines (Invoke-CliTest.ps1, two in New-ViDiffReport.ps1,
Test-InstalledPackage.ps1), and three scripts had their own list of places
to look for vipm or gh.

LabVIEWCi.psm1 now has Invoke-NativeCommand (returns ExitCode and Output),
Get-VipmPath and Get-GhPath, and the scripts call them. Behaviour is the
same, with three small differences: an empty stderr line from the VI
comparison is reported as its text, as the other scripts already did; a
program that is not installed returns exit code -1 with a "not recognized"
line in the output; and Test-InstalledPackage.ps1 and Publish-PrReport.ps1
now also look for vipm and gh under ProgramW6432, like
Resolve-CiEnvironment.ps1 did already.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LabVIEW.ini and VIPM's Settings.ini were read and written with
[Text.Encoding]::Default, which is the ANSI code page only in Windows
PowerShell and UTF-8 in PowerShell 7. Get-AnsiEncoding in LabVIEWCi.psm1
now names the system code page (GetEncoding(0), 1252 if that fails), and
Get-IniValue reads with it too. The lines written to GITHUB_ENV,
GITHUB_PATH and the job summary go through Add-GitHubFileText as UTF-8
without BOM; Add-Content wrote them in the ANSI code page, so a path with
non-ASCII characters reached the next steps garbled.

Test-InstalledPackage.ps1 writes its initialize request to a temporary
file of its own run and deletes it at the end. Paths given by the caller
are passed with -LiteralPath in Invoke-CliTest.ps1,
Test-InstalledPackage.ps1, Install-Dependencies.ps1 and
Resolve-CiEnvironment.ps1, so a "[" in a folder name is not read as a
wildcard. Install-Dependencies.ps1 keeps a VIPM_DESKTOP_LIVELINESS_TIMEOUT
that the caller has set instead of replacing it with 600 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New-ViDiffReport.ps1 ignored the exit codes of git diff, merge-base and
worktree add. A failed diff looked like "no VI changed" and passed; a
failed worktree add left every old version missing. The file list was
parsed from quoted, tab-separated output, so a path with non-ASCII
characters or quotes came back in git's escaped form and was not found.

Now every git call goes through Invoke-Git (LabVIEWCi.psm1), which reads
git's output as UTF-8 and throws on a non-zero exit code. The file list
comes from "git -c core.quotePath=false diff -z" and is split on NUL
(ConvertFrom-GitNameStatus). Stale worktrees of a killed earlier run are
pruned first, and a failing clean-up only warns.

Difference headings and details from LabVIEW's HTML report, frame
captions and folder names are now shown as plain text
(ConvertTo-MarkdownText): tags are dropped, and | < > & and @ are
escaped, so "<Default>" stays visible, a "|" no longer splits a table row
and "@name" in a label does not mention anyone. LabVIEW CLI output is
wrapped in a code fence longer than any run of backticks in it
(Format-MarkdownCodeBlock). Paths are passed with -LiteralPath.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Publish-PrReport.ps1 checked only the exit code of the push. A failed
worktree add, commit or rev-parse went unnoticed; it built the report
commit on a fixed temporary branch, assets-tmp, which a cancelled earlier
run could leave behind and which was force-deleted afterwards; a failed
comment lookup through gh was taken as "no comment yet" and posted a
second report; only the first 100 comments were searched; and any comment
with the marker was overwritten, whoever had written it.

Now every git call goes through Invoke-Git and fails the step on an
error. The report commit is built without any branch: a worktree with
--no-checkout (empty index), write-tree and commit-tree, pushed by its id
to pr-assets/pr-<number>. The comment lookup checks gh's exit code, reads
all pages (gh api --paginate, or page by page through the REST API) and
edits only a comment that carries the marker and was written by the user
the token belongs to (github-actions[bot] in the workflow, whose token
cannot read /user). Otherwise a new comment is posted.

For checkouts without persisted credentials the push uses GH_TOKEN, sent
as the same Authorization header actions/checkout would store, through
GIT_CONFIG_COUNT/KEY/VALUE of that one git call: it is not on a command
line and not written to a config file. A checkout that still has its own
header, or a local run without GH_TOKEN, pushes as before.
Invoke-Utf8Command (LabVIEWCi.psm1) runs git and gh with their output read
as UTF-8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Convert-CarayaReport.ps1 took the deepest indented lines as the asserts.
A single line one level deeper than the rest moved the assert level down,
and every test disappeared from the table. It also cut every assert label
at its last " - ", so a label such as "fake_tool_arrays - intarray has
items of type integer" was shown as "fake_tool_arrays". Labels went into
the table as they were, so a "|" split the row, "<x>" vanished as an HTML
tag and "@name" mentioned someone, and the protocol tester's output could
close its own code fence.

The report is now read by its structure, described in the script: caller
lines each at their own depth, then the tests at one depth, each followed
by its asserts one level deeper. The assert level is the depth most lines
share, the tests are one level above it, and anything deeper belongs to
the current test. Only test lines are split into label and VI. Cells go
through ConvertTo-MarkdownText, the protocol output through
Format-MarkdownCodeBlock, and both reports are read as UTF-8 with
-LiteralPath.

TestStatus and ProtocolStatus accept "not-run", which the pull request
workflow passes when it skipped the step because finding LabVIEW or
installing the packages failed; the report then says so instead of "no
per-test report was produced".

On a real report from the runner machine (64 tests) the table is the same
apart from the full assert labels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Proposed changes to the release workflow template (the maintainer copies
it to .github/workflows/release.yml):

- The build-and-release job fails for a v* tag whose commit is not an
  ancestor of origin/main (git merge-base --is-ancestor), so a release is
  always built from reviewed code. test-* tags, which only build a test
  package, may still point anywhere. The checkout fetches the full
  history for this check.
- The job runs in the "release" environment, so it waits for the approval
  of its required reviewer.
- The workflow token is read-only (permissions: contents: read); only
  build-and-release gets contents: write, to create the release.
- github.ref_name reaches the script through env: (TAG_NAME) instead of
  being pasted into it with ${{ }}.
- actions/checkout, actions/upload-artifact and softprops/action-gh-release
  are pinned to the commits of v4.4.0, v4.6.2 and v2.6.2, the newest
  releases of the major versions used so far.
- The release text is %(contents:subject) and %(contents:body), so the
  signature block of a signed tag is no longer part of it, and git's
  output is read as UTF-8. A failed tag fetch stops the step.
- The version pattern is anchored (^v1.2.3[.4]$): a test-* tag with a
  number in it no longer rewrites the package version.
- VIP_PATH is written to GITHUB_ENV as UTF-8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Proposed changes to the pull request workflow template (the maintainer
copies it to .github/workflows/pull-request.yml):

- Every checkout uses persist-credentials: false, so no token is left in
  .git/config while the pull request's tests, VIs and protocol tester run
  on the self-hosted runner. Only the step that publishes the report gets
  the token, as GH_TOKEN; Publish-PrReport.ps1 now pushes the report
  images with it (commit "Check every git and gh call when publishing the
  review report").
- always() is replaced by !cancelled(), so a cancelled run stops instead
  of posting a half report.
- Step outputs, the base branch, the pull request number and head commit
  reach the scripts through env: instead of being pasted into run: blocks;
  the run URL and run number come from the runner's GITHUB_* variables.
- The finding, install and re-check steps have ids. Protocol tests and the
  VI diff check their outcomes, so they no longer run with empty LABVIEW_*
  values after a failed setup; the test report says "not run" for skipped
  steps, the comment says when the VI diff did not run, and the last step
  also fails and says why when one of the setup steps failed or the diff
  did not run.
- GITHUB_OUTPUT, GITHUB_ENV and the job summary are written as UTF-8
  without BOM ([IO.File]::AppendAllText); ">>" wrote UTF-16 and Add-Content
  the ANSI code page.
- test-build has the same fork guard as review.
- The actions are pinned to the same commits as in the release template.
- The file ends with a newline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…late

- tools/setup/README.md: the VI Server port is taken from VIPM's settings
  first, then LabVIEW.ini, then the repository variable, then 3363, as
  Resolve-ViServerPort does; the table and the "VI Server port" section
  said the running LabVIEW's port came first and that CI read the port
  from LabVIEW.ini. The table also says now that a LabVIEW on the wrong
  port is force-closed, and when not.
- The mass compile recipe ended with "git checkout -- .", which discards
  every uncommitted change in the checkout. It now restores only tests
  and says to check git status first.
- tools/ci/README.md: the release template is described as it is (tests
  on every push, build with the chosen test LabVIEW rather than a fixed
  LabVIEW 2020 64-bit, v* tags on main only, release environment), with
  the environment and tag ruleset it expects. "A pull request cannot
  change how its own report is produced" was not true: the pull request's
  VIs and tester run on the same runner, and its own scripts are used
  while main lacks them. The section says that now, and how
  CI_DEDICATED_MACHINE changes what CI closes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Tests

✅ All 64 tests passed.

Per-test results
Test VI Result Checks
Test 1: Initialize Test Initialize.vi ✅ pass Assert True: pass
Test 2: Initialized Test Initialized.vi ✅ pass Assert True: pass
Test 3: tools/list Test tools list.vi ✅ pass Assert True: pass
Test 4: prompts/list test prompts list.vi ✅ pass Assert True: pass
Test 5: resources/list test resources list.vi ✅ pass Assert True: pass
Test 6: tools call test tools call.vi ✅ pass Assert True: pass
Test 7: empty JSON object is an invalid request test empty msg.vi ✅ pass Assert True: pass
Test 8: call broken tool test broken tool.vi ✅ pass Assert True: pass
Test 6: tools call test tools call array.vi ✅ pass Assert True: pass
Test 9: GET is rejected with 405 test http get rejected.vi ✅ pass GET -> 405 with Allow: POST and error -32600: pass
Test 10: Origin validation test http origin.vi ✅ pass Origin http://evil.example -> 403 with error -32600: pass
Origin http://localhost:6274 -> 200 with result: pass
Test 11: MCP-Protocol-Version validation test http protocol version.vi ✅ pass MCP-Protocol-Version 1999-01-01 -> 400 with error -32022 (Unsupported protocol version, the MCP 2026-07-28 code in both eras): pass
MCP-Protocol-Version 2025-06-18 -> 200 with result: pass
Test 12: notification answered with 202 test notification accepted.vi ✅ pass notifications/initialized -> 202 with empty body: pass
Test 13: request with id answered with 200 test request status.vi ✅ pass tools/list with id 1 -> 200 with result and id 1: pass
Test 14: unknown method test unknown method.vi ✅ pass does/not/exist with id 5 -> 200 with error -32601 and id 5: pass
Test 15: unknown notification test unknown notification.vi ✅ pass notifications/cancelled -> 202 with empty body: pass
Test 16: ping test ping.vi ✅ pass ping with id 3 -> 200 with result {} and id 3: pass
Test 17: string request id test string id.vi ✅ pass tools/list with id "abc-1" -> 200 with result and id "abc-1": pass
Test 18: unknown prompt test unknown prompt.vi ✅ pass prompts/get no-such-prompt with id 7 -> JSON-RPC error -32602 with id 7, no status/source: pass
Test 19: unknown resource test unknown resource.vi ✅ pass resources/read of an unknown uri with id 8 -> JSON-RPC error -32002 with id 8, no status/source: pass
Test 20: resources/read returns UTF-8 text unchanged test resource text utf8.vi ✅ pass text of labview://fixture_utf8 decodes to the bytes of fixture_utf8.txt: pass
Test 21: resources/read returns a binary file as Base64 test resource blob.vi ✅ pass blob of labview://fixture_binary is the Base64 of the bytes of fixture_binary.png: pass
Test 22: unknown tool test unknown tool.vi ✅ pass tools/call no_such_tool -> error -32602 with message Unknown tool: no_such_tool and id 22: pass
Test 23: tools/call with arguments that are not an object test tool call without arguments.vi ✅ pass tools/call fake_tool_add with arguments 5 -> error -32602 with message Invalid arguments... and id 23: pass
Test 24: broken tool is a tool execution error test broken tool message.vi ✅ pass tools/call fake_tool_broken_vi -> result with isError true and a text that says not executable: pass
Test 25: prompts/get without arguments test prompt without arguments.vi ✅ pass prompts/get fake_prompt_concatanate_strings without arguments -> result with messages: pass
Test 26: request without method test request without method.vi ✅ pass a message with id 26 and no method -> error -32600 with id 26: pass
Test 27: malformed JSON test malformed json.vi ✅ pass truncated JSON -> error -32700 with id null: pass
Test 28: array inputs say what their items are test tools list array items.vi ✅ pass fake_tool_arrays: pass
Test 29: protocol version negotiation test protocol version negotiation.vi ✅ pass initialize asking for 2024-11-05 -> protocolVersion 2024-11-05: pass
initialize asking for 1999-01-01 -> protocolVersion 2025-11-25 (the newest legacy version): pass
Test 30: resources/templates/list test resource templates list.vi ✅ pass resources/templates/list -> result with resourceTemplates []: pass
Test 31: tools/call without arguments test tool call default arguments.vi ✅ pass tools/call fake_tool_add without arguments -> result with the sum 0, no error, id 31: pass
Test 32: a tool that returns an error sets isError test tool error result.vi ✅ pass fake_tool_return_error with status true -> result with isError true, code and source still in the content: pass
fake_tool_return_error with status false -> result without isError: pass
Test 33: internal errors show no LabVIEW internals test internal error message.vi ✅ pass prompts/get fake_prompt_broken_vi -> error -32603 with a message without call chain, markup, tags or path: pass
Test 34: prompts/list says which arguments are required test prompts list required.vi ✅ pass fake_prompt_required_argument: pass
Test 35: prompts/get checks required arguments test prompt required argument.vi ✅ pass prompts/get fake_prompt_required_argument without topic -> error -32602 that names topic, id 36: pass
prompts/get fake_prompt_required_argument with topic -> result whose text contains both argument values: pass
Test 36: JSON-RPC batch test batch.vi ✅ pass batch of ping (id 38), a notification and tools/list (id b) -> 200 with an array of two answers in that order: pass
batch of two notifications -> 202 with empty body: pass
empty batch -> error -32600 with id null: pass
Test 37: a JSON value that is not an object test non object message.vi ✅ pass body 5 -> error -32600 with id null: pass
body "text" -> error -32600 with id null: pass
body null -> error -32600 with id null: pass
Test 38: tools/call checks required arguments test tool required arguments.vi ✅ pass tools/call fake_tool_required_input without arguments -> result with isError true and the text Missing required argument(s): b, a (the order of inputSchema.required), no JSON-RPC error, id 40: pass
tools/call fake_tool_required_input with arguments {} -> result with isError true naming b and a, no JSON-RPC error, id 41: pass
tools/call fake_tool_required_input with only a -> result with isError true naming only b, no JSON-RPC error, id 42: pass
tools/call fake_tool_required_input with a 2 and b 3 -> result 6, no error, id 43: pass
Test 39: initialize with an unknown protocol version test protocol version fallback.vi ✅ pass initialize asking for 1999-01-01 -> protocolVersion is the first entry of Supported Protocol Versions.vi: pass
Test 40: initialize reports the LabVIEW version test labview version.vi ✅ pass initialize -> serverInfo.additionalDetails.labviewVersion equals the Version Number of this LabVIEW: pass
Test 41: tools/list shows the MCP names of special labels test tools list mcp names.vi ✅ pass tools/list fake_tool_special_labels -> Length (mm) and Factor #1 & 50% are listed as length__mm_ and factor__1___50_ in properties and in required, id 44: pass
tools/list fake_tool_special_labels -> every name in properties and required matches ^[a-zA-Z0-9_-]{1,64}$: pass
Test 42: tools/call reaches special labels by their MCP names test tools call mcp names.vi ✅ pass tools/call fake_tool_special_labels with length__mm_ 2.5 and factor__1___50_ 4 -> result {"result__mm_":10}, no error, id 45: pass
Test 43: Add Tool refuses inputs whose MCP names clash test add tool name clash.vi ✅ pass Add Tool fake tool name clash.vi with the inputs a b and a_b -> error 5010 naming both controls and the MCP name a_b: pass
tools/list after the refused Add Tool -> fake_tool_name_clash is not listed, id 46: pass
Test 44: Add Prompt refuses arguments whose MCP names clash test add prompt name clash.vi ✅ pass Add Prompt fake prompt name clash.vi with the arguments a b and a_b -> error 5010 naming both controls and the MCP name a_b: pass
prompts/list after the refused Add Prompt -> fake_prompt_name_clash is not listed, id 47: pass
Test 45: 3D arrays and arrays of paths, enums and clusters say what their items are test tools list array shapes.vi ✅ pass fake_tool_array_shapes: pass
fake_tool_array_shapes: pass
fake_tool_array_shapes: pass
fake_tool_array_shapes: pass
Test 46: timestamps, clusters and types without a JSON type are described in tools/list test tools list schema details.vi ✅ pass fake_tool_array_shapes: pass
fake_tool_array_shapes: pass
fake_tool_schema_shapes: pass
fake_tool_schema_shapes: pass
fake_tool_schema_shapes: pass
fake_tool_schema_shapes: pass
fake_tool_schema_shapes: pass
Test 47: tools/call converts timestamps, clusters, enums and 2D and 3D arrays test tools call schema shapes.vi ✅ pass fake_tool_array_shapes with a 3D array [[[1,2],[3,4]],[[5,6],[7,8]]] -> no isError, the tool gets all 8 elements: pass
fake_tool_schema_shapes: pass
fake_tool_schema_shapes: pass
fake_tool_schema_shapes: pass
fake_tool_schema_shapes with timestamp 2026-10-17T12:34:56Z -> no isError, time out 2026-10-17T12:34:56.000Z: pass
Test 48: every array schema in tools/list says what its items are test tools list every array has items.vi ✅ pass tools/list of the test server: pass
Test 49: tools/list gives element descriptions, integer ranges, data entry limits, units and defaults test tools list descriptions defaults ranges.vi ✅ pass fake_tool_schema_shapes: pass
fake_tool_schema_details: pass
fake_tool_schema_details: pass
fake_tool_schema_details: pass
fake_tool_schema_details: pass
fake_tool_schema_details: pass
fake_tool_schema_details: pass
fake_tool_schema_details: pass
Test 50: tools/call returns structuredContent without picture outputs, and none on isError results test tools call structured content.vi ✅ pass tools/call fake_tool_required_input without arguments (id 503) -> isError true and no structuredContent: pass
tools/call fake_tool_return_error with status true (id 501) -> isError true and no structuredContent: pass
tools/call fake_tool_return_error with status false (id 502) -> structuredContent carries the error cluster output as an ordinary object: pass
fake_tool_picture_outputs: pass
fake_tool_picture_outputs: pass
tools/call fake_tool_picture_outputs (id 50) -> structuredContent is present and equals the text JSON without its picture placeholder: pass
Test 51: tools/list gives an outputSchema that matches structuredContent test tools list output schema.vi ✅ pass tools/list fake_tool_no_inputs_outputs (no outputs) -> no outputSchema: pass
tools/list fake_tool_picture_outputs -> outputSchema is an object schema whose properties and required are count, label, mode and time: pass
fake_tool_picture_outputs: pass
fake_tool_picture_outputs: pass
fake_tool_picture_outputs: pass
Test 52: tools/list sends title and annotations only when they are set test tools list title annotations.vi ✅ pass tools/list fake_tool_add (registered without title and annotations) -> neither title nor annotations: pass
tools/list fake_tool_annotated (registered with a title) -> title "Annotated Fixture Tool": pass
fake_tool_annotated (readOnlyHint true, destructiveHint false, the others not set) -> annotations with exactly readOnlyHint true and destructiveHint false: pass
Test 53: modern results carry resultType, cache hints and serverInfo; legacy results do not test modern tools list.vi ✅ pass modern tools/list -> _meta io.modelcontextprotocol/serverInfo with name and version: pass
modern tools/list (id 53, MCP 2026-07-28 headers and _meta) -> 200 with tools, resultType complete, ttlMs 60000 and cacheScope public: pass
legacy tools/list (id 531, no _meta) -> tools without resultType, ttlMs, cacheScope or _meta: pass
Test 54: server/discover, and initialize never negotiates 2026-07-28 test server discover.vi ✅ pass initialize asking for 2026-07-28 (id 541) -> the newest legacy version 2025-11-25, never 2026-07-28: pass
server/discover -> capabilities tools, prompts and resources (as initialize), _meta serverInfo, ttlMs 60000, cacheScope public: pass
server/discover (id 54) -> 200, resultType complete, supportedVersions with 2026-07-28 and 2025-11-25: pass
initialize asking for 2025-11-25 (id 542) -> 2025-11-25: pass
Test 55: modern requests are refused when their headers do not match the body test modern headers.vi ✅ pass tools/call with an Mcp-Name that is not params.name (id 554) -> 400 with error -32020: pass
MCP-Protocol-Version 2025-11-25 while _meta says 2026-07-28 (id 553) -> 400 with error -32020: pass
tools/call without an Mcp-Name header (id 551) -> 400 with error -32020: pass
tools/call with Mcp-Name =?base64?...?= of fake_tool_add, lower-case header names (id 552) -> 200 with the tool result: pass
Mcp-Method prompts/list on a tools/list request (id 55) -> 400 with error -32020: pass
Test 56: unsupported versions, missing capabilities and unknown methods in the modern era test modern errors.vi ✅ pass modern resources/read of an unknown uri (id 565) -> error -32602 (legacy keeps -32002): pass
modern request with the unknown method foo/bar (id 563) -> 404 with error -32601: pass
modern request with the legacy version 2025-11-25 in _meta (id 561) -> 400, error -32022, data.supported with 2025-11-25: pass
unknown version 2027-01-01 in header and _meta (id 56) -> 400, error -32022, data.supported with 2026-07-28, data.requested 2027-01-01: pass
modern ping (id 564), which MCP 2026-07-28 removed -> 404 with error -32601: pass
modern request without io.modelcontextprotocol/clientCapabilities (id 562) -> 400 with error -32602: pass
Test 57: elicitation by input_required and retry with inputResponses test elicitation.vi ✅ pass legacy tools/call (id 574) -> outcome not supported: pass
retry with inputResponses confirm_action accept (id 571) -> complete result with outcome confirmed: pass
modern tools/call of fake_tool_confirm with the elicitation capability (id 57) -> resultType input_required with inputRequests.confirm_action: pass
modern tools/call without the elicitation capability (id 573) -> outcome not supported, no input_required: pass
retry with inputResponses confirm_action decline (id 572) -> outcome declined: pass
Test 58: progress notifications as Server-Sent Events for tools/call test progress notifications.vi ✅ pass legacy tools/call with progressToken "t58" and Accept text/event-stream (id 581) -> HTTP 200, text/event-stream: pass
id 581: exactly 10 notifications/progress events, each with progressToken "t58" and total 10, progress 1..10 in increasing order: pass
the tool finished all 10 steps (fixture record): pass
id 581: the JSON-RPC result is the last data event, after progress 10: pass
the same call without progressToken (id 582) -> plain application/json, no progress events: pass
modern tools/call with the numeric progressToken 583 (id 583) -> text/event-stream, 10 events with "progressToken":583 in order, then the complete result: pass
Test 59: closing the stream cancels a streamed tools/call test progress cancel by close.vi ✅ pass modern streamed tools/call over raw TCP (id 591) -> text/event-stream, the first progress event arrives: pass
after the client closed the connection the tool stopped early (fewer than 10 steps) and saw cancelled? TRUE: pass
Test 60: legacy notifications/cancelled stops a streamed tools/call test progress legacy cancel.vi ✅ pass legacy streamed tools/call over raw TCP (id 601) -> text/event-stream, the first progress event arrives: pass
the tool stopped early (fewer than 10 steps) and saw cancelled? TRUE: pass
the stream ended without a result for id 601: pass
notifications/cancelled for id 601 on a second connection -> HTTP 202 within 1 s, while the call runs: pass
Test 61: a client that is not in Allowed clients gets 403 test access allowed clients.vi ✅ pass tools/list from this computer to a server that allows only 192.0.2.1 -> HTTP 403, application/json: pass
the second server (port 37061, Allowed clients = [192.0.2.1]) started: pass
the 403 body is the JSON-RPC error -32600 "Forbidden: client not allowed." with id null: pass
Test 62: Access token requires Authorization: Bearer test access bearer token.vi ✅ pass the token never appears in a response: pass
the second server (port 37062, Access token set) started: pass
authorization: Bearer test-token-62 (lower-case header name) -> HTTP 200 with the tools/list result: pass
the 401 body is the JSON-RPC error -32600 "Unauthorized: missing or invalid bearer token." with id null: pass
Authorization: Bearer wrong-token -> HTTP 401 with WWW-Authenticate: Bearer: pass
no Authorization header -> HTTP 401 with WWW-Authenticate: Bearer (header names compared case-insensitively): pass
Authorization: bearer test-token-62 (lower-case scheme) -> HTTP 200 with the tools/list result, no WWW-Authenticate: pass
Test 63: tools/list leaves out LabVIEW's default error cluster descriptions test tools list error descriptions.vi ✅ pass fake_tool_error_descriptions: pass
tools/list fake_tool_error_descriptions -> error in (no error) and error out are in inputSchema and outputSchema without a description key, and so are their status, code and source; their type is still there: pass

Protocol tests

✅ Test-McpServer.ps1 passed against the test server.

Protocol tester output (0 failed required checks)
Testing MCP server at http://127.0.0.1:36987/mcp/server

[PASS] (required) Lifecycle - initialize answers 200 with a JSON result
[PASS] (required) Lifecycle - initialize answer is JSON-RPC 2.0 with the request id
[PASS] (required) Lifecycle - initialize result has protocolVersion, capabilities and serverInfo.name
[PASS] (required) Lifecycle - Content-Type of the answer is application/json
[PASS] (advisory) Lifecycle - a client asking for 2024-11-05 gets that version back
[PASS] (required) Lifecycle - notifications/initialized is accepted with 202 and no body
[PASS] (required) Transport - GET answers 405 or opens an event stream
[PASS] (advisory) Transport - 405 answer names the allowed method in an Allow header
[PASS] (required) Transport - a request from a foreign Origin is rejected (403)
[PASS] (required) Transport - a request from a localhost Origin is accepted
[PASS] (required) Transport - an unsupported MCP-Protocol-Version header is rejected (400)
[PASS] (required) Transport - the negotiated MCP-Protocol-Version header is accepted
[PASS] (required) JSON-RPC - ping is answered with an empty result
[PASS] (required) JSON-RPC - an unknown method gets error -32601 with the request id
[PASS] (required) JSON-RPC - an unknown notification is not answered
[PASS] (required) JSON-RPC - a string request id is returned unchanged
[PASS] (required) JSON-RPC - malformed JSON gets error -32700
[PASS] (advisory) JSON-RPC - a request without a method gets error -32600
[PASS] (required) Tools - tools/list returns a tools array
[PASS] (required) Tools - every tool has a name and an inputSchema of type object (1 tools)
[PASS] (advisory) Tools - array parameters say what their items are
[PASS] (required) Tools - calling an unknown tool is answered with an error
[PASS] (required) Prompts - prompts/list returns a prompts array
[PASS] (required) Prompts - every prompt has a name (1 prompts)
[PASS] (required) Prompts - getting an unknown prompt is answered with an error
[PASS] (required) Resources - resources/list returns a resources array
[PASS] (required) Resources - every resource has a uri and a name (2 resources)
[PASS] (required) Resources - reading an unknown resource is answered with an error
[PASS] (advisory) Resources - resources/templates/list returns a resourceTemplates array

Required: 24 of 24 passed. Advisory: 0 failed.

VI diff report

No VI, VIM or CTL files changed in this pull request.

Report for commit f354e18. The installable test package is under Artifacts of this run once the build job has finished.

@JanGoebel
JanGoebel merged commit b57feef into main Oct 30, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants