Skip to content

ci: shard Node integration tests and increase browser Playwright workers - #24676

Merged
nicohrubec merged 11 commits into
developfrom
feat/node-integration-timings
Sep 25, 2026
Merged

nicohrubec merged 11 commits into
developfrom
feat/node-integration-timings

Conversation

@nicohrubec

@nicohrubec nicohrubec commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Improve CI speed by running Node integration tests in three shards and browser Playwright with six CI workers. I am aware benchmarking in CI is quite unreliable, but I tested this multiple times yesterday and again today and speedups were quite consistent so I think we can be reasonably confident. Most of the wall time gain definitely comes from the node integration test sharding because these were a heavy bottleneck before, now end times are much more evenly distributed.

Today's run wall time from workflow start (excluding the optional SvelteKit timeout from the required-check comparison):

Milestone Baseline This PR Faster by
Required checks passed 15m20s 11m27s 3m53s (25%)
All Node integration tests finished 15m15s 10m46s 4m29s
All browser tests finished 11m38s 9m42s 1m56s

Co-Authored-By: GPT-6 <codex@openai.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/workflows/build.yml Outdated
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

Path Size % Change Change
@sentry/browser 29.24 kB - -
@sentry/browser - with treeshaking flags 27.5 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 27.4 kB - -
@sentry/browser (incl. Tracing) 51.15 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 51.17 kB - -
@sentry/browser (incl. Tracing, Profiling) 54.18 kB - -
@sentry/browser (incl. Tracing, Replay) 90.76 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 79.86 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 95.46 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 108.41 kB - -
@sentry/browser (incl. Feedback) 46.76 kB - -
@sentry/browser (incl. sendFeedback) 34.3 kB - -
@sentry/browser (incl. FeedbackAsync) 39.41 kB - -
@sentry/browser (incl. Metrics) 30.25 kB - -
@sentry/browser (incl. Logs) 30.51 kB - -
@sentry/browser (incl. Metrics & Logs) 31.18 kB - -
@sentry/react 31 kB - -
@sentry/react (incl. Tracing) 53.45 kB - -
@sentry/vue 36.74 kB - -
@sentry/vue (incl. Tracing) 53.7 kB - -
@sentry/svelte 29.26 kB - -
CDN Bundle 30.93 kB - -
CDN Bundle (incl. Tracing) 51.69 kB - -
CDN Bundle (incl. Logs, Metrics) 33.2 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 53.66 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 73.92 kB - -
CDN Bundle (incl. Tracing, Replay) 89.28 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 91.25 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 95.45 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 97.42 kB - -
CDN Bundle - uncompressed 91.4 kB - -
CDN Bundle (incl. Tracing) - uncompressed 153.77 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 97.97 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 159.73 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 227.54 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 273.5 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 279.44 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 287.2 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 293.13 kB - -
@sentry/nextjs (client) 55.77 kB - -
@sentry/sveltekit (client) 51.59 kB - -
@sentry/core/server 39.95 kB - -
@sentry/core/browser 13.63 kB - -
@sentry/node 133.9 kB +0.01% +6 B 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection) 82.43 kB - -
@sentry/node - without tracing 90.45 kB +0.02% +14 B 🔺
@sentry/node - without channel injection 112.42 kB +0.02% +12 B 🔺
@sentry/aws-serverless 98.72 kB +0.01% +4 B 🔺
@sentry/cloudflare (withSentry) - minified 206.49 kB - -
@sentry/cloudflare (withSentry) 513.73 kB - -

View base workflow run

Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: collect Node integration test timings ci: shard Node integration tests and collect timings Sep 24, 2026
nicohrubec and others added 2 commits September 24, 2026 13:13
Co-Authored-By: GPT-6 <codex@openai.com>
Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: shard Node integration tests and collect timings ci: parallelize browser bundles and shard Node integration tests Sep 24, 2026
Restore serial browser bundling while retaining the Node integration test shards.

Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: parallelize browser bundles and shard Node integration tests ci: shard Node integration tests across three runners Sep 24, 2026
Generate unminified and minified debug bundles from one Rollup build, retaining a separate production build and the existing output plugin order.

Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: shard Node integration tests across three runners ci: shard Node integration tests and reuse debug bundle builds Sep 24, 2026
Increase browser Playwright CI workers to six. Remove Node timing reports and restore the original bundle build, retaining three shards for every Node integration configuration.

Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: shard Node integration tests and reuse debug bundle builds ci: shard Node integration tests and increase browser Playwright workers Sep 24, 2026
Restore three Rollup processes on the existing bundle runner to measure their effect alongside the current test parallelism.

Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: shard Node integration tests and increase browser Playwright workers ci: parallelize Node integration tests, browser tests, and bundles Sep 24, 2026
Restore serial browser bundling and retain three Node integration shards.

Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: parallelize Node integration tests, browser tests, and bundles ci: shard Node integration tests and increase Playwright workers Sep 24, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit cda6116. Configure here.

Comment thread dev-packages/test-utils/src/playwright-config.ts Outdated
Comment thread dev-packages/browser-integration-tests/playwright.config.ts Outdated
Restore the previously passing configuration with three Node integration shards and original bundling.

Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec nicohrubec changed the title ci: shard Node integration tests and increase Playwright workers ci: shard Node integration tests and increase browser Playwright workers Sep 24, 2026
Co-Authored-By: GPT-6 <codex@openai.com>
@nicohrubec
nicohrubec marked this pull request as ready for review September 25, 2026 07:43
@nicohrubec
nicohrubec requested a review from a team as a code owner September 25, 2026 07:43
@nicohrubec
nicohrubec requested review from logaretm and msonnb and removed request for a team September 25, 2026 07:43
@nicohrubec
nicohrubec marked this pull request as draft September 25, 2026 07:44
@nicohrubec
nicohrubec marked this pull request as ready for review September 25, 2026 08:18
@nicohrubec
nicohrubec requested a review from mydea September 25, 2026 08:18

@andreiborza andreiborza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image

@nicohrubec
nicohrubec merged commit 0808c61 into develop Sep 25, 2026
336 of 337 checks passed
@nicohrubec
nicohrubec deleted the feat/node-integration-timings branch September 25, 2026 10:59
nicohrubec added a commit that referenced this pull request Sep 25, 2026
Similar to #24676, split Cloudflare integration tests across two CI
jobs. Atm this doesn't get us an overall faster wall time but as we'll
soon be running more and more tests on cloudflare this could quickly
become a bottleneck, getting ahead of this by doing this simple
optimization. We can also always add more shards if it becomes
necessary.

Both shards passed. Compared with a run off develop today:

| Cloudflare | Baseline | Two shards | Faster by |
| --- | --- | --- | --- |
| Test execution (slowest shard) | 5m56s | 3m15s | 2m41s (45%) |
| **All Cloudflare jobs finished, from CI start** | **9m31s** |
**6m19s** | **3m12s** |

Observed timings from one run each.

- Baseline: [CI
run](https://github.com/getsentry/sentry-javascript/actions/runs/36110233929)
· [Sentry
trace](https://sentry.sentry.io/explore/traces/trace/c7cd7471a660956b591fea4f99270e09/?project=5899451)
- This PR: [CI
run](https://github.com/getsentry/sentry-javascript/actions/runs/36114956684)
· [Sentry
trace](https://sentry.sentry.io/explore/traces/trace/f2b405b0238b2a25f431bd23da2f130f/?project=5899451)

Co-authored-by: GPT-6 <codex@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants