Skip to content

fix(ci): make Playwright apt retries recover - #19133

Merged
gr2m merged 2 commits into
mainfrom
codex/fix-playwright-apt-retries
Aug 19, 2026
Merged

gr2m merged 2 commits into
mainfrom
codex/fix-playwright-apt-retries

Conversation

@gr2m

@gr2m gr2m commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Background

The Test RSC e2e job failed three times on August 19 while Playwright installed Chromium system dependencies:

In each run, apt-get update stalled against azure.archive.ubuntu.com until the 15-minute timeout. Playwright had spawned apt-get through a nested sudo process, so apt-get survived the timeout and retained /var/lib/apt/lists/lock. The retry guard only watched /var/lib/dpkg/lock-frontend, causing attempts two and three to fail immediately.

This matches the active GitHub runner-images incident: actions/runner-images#14594

Summary

  • run Playwright and its timeout as root so apt-get remains in the timeout process group
  • add apt acquisition timeouts and a bounded retry so stalled mirrors fail over promptly
  • wait for the apt lists, archives, and dpkg locks before every attempt
  • report an explicit error if locks remain held for five minutes

End-to-End Verification

This is a CI-only change. The intermittent mirror timeout cannot be forced locally; this pull request CI exercises the normal Test RSC e2e path. The timeout path now keeps the Playwright process and its privileged apt descendants in one process group and checks the exact lock observed in all three failures before retrying.

Validation

  • pnpm check
  • pnpm type-check:full
  • workflow YAML parsing
  • embedded Bash syntax check
  • git diff --check

Checklist

  • All commits are signed (PRs with unsigned commits cannot be merged)
  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

Related Issues

@gr2m
gr2m enabled auto-merge (squash) August 19, 2026 22:31
@gr2m
gr2m merged commit d6339f4 into main Aug 19, 2026
48 checks passed
@gr2m
gr2m deleted the codex/fix-playwright-apt-retries branch August 19, 2026 22:45
gr2m added a commit that referenced this pull request Aug 19, 2026
## Background

Backport of #19133 to the actively maintained v6 release branch. The RSC
e2e Playwright dependency install could outlive the outer timeout,
retain an apt lock, and make every retry fail immediately.

## Summary

- bound apt download retries and network timeouts
- wait for all apt/dpkg lock files before every attempt
- run the Playwright dependency install as root in its own process group
so timeout cleanup reaches apt descendants
- run RSC e2e when its workflow definition changes

## Validation

- the main PR completed the full CI matrix successfully, including Test
RSC e2e
- pnpm check
- pnpm type-check:full
- workflow YAML parse and RSC shell syntax checks

No changeset: this only changes CI infrastructure.
gr2m added a commit that referenced this pull request Aug 19, 2026
## Background

Backport of #19133 to the actively maintained v5 release branch. The
linked v5 RSC e2e failure showed the Playwright dependency install
outliving the outer timeout, retaining an apt lock, and making every
retry fail immediately.

Failure:
https://github.com/vercel/ai/actions/runs/32302334578/job/96228309554

## Summary

- bound apt download retries and network timeouts
- wait for all apt/dpkg lock files before every attempt
- run the Playwright dependency install as root in its own process group
so timeout cleanup reaches apt descendants
- run RSC e2e when its workflow definition changes

## Validation

- the main PR completed the full CI matrix successfully, including Test
RSC e2e
- pnpm check
- workflow YAML parse and RSC shell syntax checks
- pnpm type-check:full was run; release-v5.0 currently reports unrelated
React type-resolution errors across examples, while this diff only
changes .github/workflows/ci.yml

No changeset: this only changes CI infrastructure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant