Skip to content

[#366] Added macOS to scaffold devtools CI matrix. - #373

Merged
AlexSkrypnyk merged 3 commits into
1.xfrom
feature/366-add-macos-ci
May 28, 2026
Merged

AlexSkrypnyk merged 3 commits into
1.xfrom
feature/366-add-macos-ci

Conversation

@AlexSkrypnyk

@AlexSkrypnyk AlexSkrypnyk commented May 28, 2026 •

Copy link
Copy Markdown
Member

Closes #366

Summary

Adds macos-latest as a second OS dimension to the scaffold-test-devtools CI job matrix, doubling the combinations from 5 (1 OS × 5 groups) to 10 (2 OS × 5 groups). Because macOS GitHub Actions runners do not support Docker, the Selenium/Chromium container is moved from a job-level services: block to a conditional step that runs only on Linux when the group requires it (p3 or p4). macOS runs receive SCAFFOLD_SKIP_FUNCTIONAL_JAVASCRIPT=1 so the test harness skips both the dedicated FunctionalJavascript step and the broader ahoy test/make test smoke step (which would otherwise try to reach Selenium at localhost:4444). The sqlite3 upgrade is split into a Linux step (source build with SQLITE_ENABLE_COLUMN_METADATA) and a macOS step (brew install sqlite3 with build flags exported via $GITHUB_ENV). The pre-existing Install Ahoy step's inline case syntax inside $(...) is split across lines so it parses under macOS's bash 3.2. Coverage artifact names are disambiguated by OS so the two runners do not collide on upload.

Changes

.github/workflows/scaffold-test.yml

  • Replaced runs-on: ubuntu-latest with runs-on: ${{ matrix.os }} and added os: [ubuntu-latest, macos-latest] to the matrix.
  • Removed the job-level services: block (Docker service containers are unsupported on macOS runners).
  • Added a conditional Start Chromium (Linux only) step that runs docker run only when runner.os == 'Linux' and matrix.group is p3 or p4, with a 30-second readiness poll before proceeding.
  • Split the sqlite3 upgrade into a Linux step (unchanged wget + source build) and a macOS step (brew install sqlite3 with LDFLAGS/CPPFLAGS/PKG_CONFIG_PATH exported via $GITHUB_ENV).
  • Rewrote the Install Ahoy step's os/architecture detection: split the inline case $(uname -m) in ... esac into a multi-line case block. The single-line form parses under bash 4+ on Linux but errors on macOS bash 3.2 (syntax error near unexpected token ';;').
  • Added SCAFFOLD_SKIP_FUNCTIONAL_JAVASCRIPT: ${{ runner.os == 'macOS' && '1' || '0' }} to the test step's env block.
  • Disambiguated the coverage artifact name from ...-${{ matrix.group }} to ...-${{ matrix.os }}-${{ matrix.group }}.

.scaffold/tests/src/Functional/AhoyTest.php and MakeTest.php

  • Added an early-return guard at the top of runFunctionalJavascriptTests() that checks getenv('SCAFFOLD_SKIP_FUNCTIONAL_JAVASCRIPT') === '1' and writes a marker line to STDERR before returning.
  • Added the same guard inline around the ahoy test/make test call in testWorkflow(). ahoy test and make test invoke PHPUnit with no filter and therefore run the FunctionalJavascript suite as well; without Selenium reachable, that step errors with Could not connect to localhost port 4444. The guard skips both that command and its browser_output directory assertion when the env var is set.
  • Default behaviour when the env var is unset or set to any other value is unchanged.

Before / After

Before - 1 OS × 5 groups = 5 matrix jobs, Selenium always pulled as a job-level service:

OS              | p0 | p1 | p2 | p3 | p4
ubuntu-latest   |  X |  X |  X |  X |  X     <- Selenium service always on

After - 2 OS × 5 groups = 10 matrix jobs, Selenium started only where needed:

OS              | p0 | p1 | p2 | p3        | p4
ubuntu-latest   |  X |  X |  X |  X + DOCK |  X + DOCK    <- step-level docker run
macos-latest    |  X |  X |  X |  X SKIP   |  X SKIP      <- SCAFFOLD_SKIP_FUNCTIONAL_JAVASCRIPT=1

X = group runs. + DOCK = Selenium container started in a Linux-only step before the test step. SKIP = both ahoy test/make test and the dedicated test-functional-javascript step are skipped via env var; every other workflow step (assemble, start, provision, lint, test-unit, test-kernel, test-functional) still runs.

@coderabbitai

coderabbitai Bot commented May 28, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

This PR extends CI support to macOS runners for the scaffold-test-devtools job. It adds OS-conditional dependency setup (Homebrew SQLite for macOS, apt for Linux), refactors Ahoy CLI installation with platform detection, enables Selenium+Chromium startup for Linux testing, and gates resource-intensive functional JavaScript tests on macOS via the SCAFFOLD_SKIP_FUNCTIONAL_JAVASCRIPT environment variable.

Changes

macOS CI Support with Conditional Test Gating

Layer / File(s) Summary
CI Matrix and OS-Specific Dependencies
.github/workflows/scaffold-test.yml
Job matrix expanded to include macos-latest. Linux SQLite3 upgrade is gated by OS check; new macOS-specific step installs SQLite via Homebrew and exports build environment variables (LDFLAGS, CPPFLAGS, PKG_CONFIG_PATH).
Ahoy CLI and Chromium Setup
.github/workflows/scaffold-test.yml
Ahoy CLI installation refactored to derive OS and architecture from uname, then download matching binary. Linux-only Chromium startup added with Selenium container launch, readiness polling, and timeout failure. SCAFFOLD_SKIP_FUNCTIONAL_JAVASCRIPT set for macOS test runs. Coverage artifact naming updated to include job name, OS, and matrix group.
Functional JavaScript Test Gating
.scaffold/tests/src/Functional/AhoyTest.php, .scaffold/tests/src/Functional/MakeTest.php
testWorkflow() in both test classes conditionally skips ahoy test/make test execution and browser output directory assertions when SCAFFOLD_SKIP_FUNCTIONAL_JAVASCRIPT=1. runFunctionalJavascriptTests() methods added early return with STDERR skip notice to prevent Selenium-backed test execution.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

Poem

🐰 A rabbit hops cross platforms wide,
macOS joins the testing tide,
Skip the FunctionalJavascript dance,
When Selenium containers prance—
Cross-OS balance, now in stride! 🌳

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding macOS to the scaffold devtools CI matrix as requested in issue #366.
Linked Issues check ✅ Passed The PR fulfills issue #366's objective to expand the CI matrix to include macOS runners alongside Ubuntu runners for scaffold dev workflows.
Out of Scope Changes check ✅ Passed All changes are directly related to adding macOS support: workflow matrix expansion, OS-specific setup steps, Selenium skip logic for macOS, and artifact naming updates.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/366-add-macos-ci

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented May 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.88%. Comparing base (574aeaf) to head (aeae9f4).

Additional details and impacted files
@@           Coverage Diff           @@
##              1.x     #373   +/-   ##
=======================================
  Coverage   55.88%   55.88%           
=======================================
  Files           4        4           
  Lines         467      467           
=======================================
  Hits          261      261           
  Misses        206      206           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.github/workflows/scaffold-test.yml (1)

27-28: ⚡ Quick win

Consider setting persist-credentials: false for security.

The checkout action does not explicitly set persist-credentials: false. This means the GitHub token persists in the local git config and remains available to subsequent steps, which increases the attack surface if a later step is compromised.

🔒 Proposed fix
       - name: Checkout code
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
+        with:
+          persist-credentials: false

As per static analysis, the zizmor tool flagged this as a credential persistence risk through GitHub Actions artifacts (artipacked).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/scaffold-test.yml around lines 27 - 28, The checkout step
named "Checkout code" currently uses
actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd without disabling
credential persistence; update that step (the actions/checkout usage) to include
persist-credentials: false so the GITHUB_TOKEN is not written to the repo git
config and cannot be used by later steps.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/scaffold-test.yml:
- Around line 60-71: The "Install Ahoy" step incorrectly remaps macOS names and
lacks explicit download-failure handling; update the os/architecture mapping so
`os` remains the raw `uname -s` lowercased (allowing `darwin`), keep the
architecture logic for `architecture`, and wrap the `sudo wget
"https://github.com/ahoy-cli/ahoy/releases/download/v2.1.1/ahoy-bin-${os}-${architecture}"
-O /usr/local/bin/ahoy` invocation with an explicit failure check (e.g., test
wget exit and echo a clear error + exit 1) so download errors abort clearly;
reference the "Install Ahoy" step and the variables `os`, `architecture`, and
the `wget` invocation when making the change.

---

Nitpick comments:
In @.github/workflows/scaffold-test.yml:
- Around line 27-28: The checkout step named "Checkout code" currently uses
actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd without disabling
credential persistence; update that step (the actions/checkout usage) to include
persist-credentials: false so the GITHUB_TOKEN is not written to the repo git
config and cannot be used by later steps.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e71f9b5c-a15e-49fe-a0a8-a6277ac538ed

📥 Commits

Reviewing files that changed from the base of the PR and between 574aeaf and aeae9f4.

📒 Files selected for processing (3)
  • .github/workflows/scaffold-test.yml
  • .scaffold/tests/src/Functional/AhoyTest.php
  • .scaffold/tests/src/Functional/MakeTest.php

Comment thread .github/workflows/scaffold-test.yml
@AlexSkrypnyk
AlexSkrypnyk merged commit d4124f3 into 1.x May 28, 2026
52 checks passed
@AlexSkrypnyk
AlexSkrypnyk deleted the feature/366-add-macos-ci branch May 28, 2026 10:36
@AlexSkrypnyk AlexSkrypnyk added this to the 4.15 milestone May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add macos to CI for scaffold dev jobs

1 participant