Skip to content

Stop a release wait at its last look, however little the clock moved - #88

Merged
codefusion-automerge[bot] merged 1 commit into
mainfrom
fix/release-wait-flake
Nov 12, 2026
Merged

codefusion-automerge[bot] merged 1 commit into
mainfrom
fix/release-wait-flake

Conversation

@XeonFX

@XeonFX XeonFX commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

a_commit_with_no_release_is_refused_and_one_that_comes_out_while_it_waits_is_taken failed every time on a loaded Mac and passed on CI. The cause is in ReleaseUpdate::release_of, not the test.

Cause

The last look was decided by began.elapsed() + every > wait. With a zero wait (the bench's default), that is 0 > 0 whenever the monotonic clock has not moved between Instant::now() and elapsed(), which a coarse or busy clock allows. The loop then looked a second time (4 GitHub requests instead of 2). On a clock that does not move at all it never stopped asking.

Fix

  • The last look is the one after which the next would come at the end of the wait or past it: >=. A zero wait asks once. In production (45 min, every 3 min) the waits end after 15 looks instead of 16, the last at 42 min.
  • release_of reads an injected cf_ghapp::Monotonic clock (the trait the token source already uses), SystemMonotonic in production.
  • Tests run on a TestClock, frozen by default and stepped where a test needs time to pass. It fails after 10,000 reads, so a wait that never ends fails instead of hanging CI.
  • a_wait_reads_the_newest_releases_only_now_and_then now asserts exactly 14 looks (40 ms steps, 600 ms wait) instead of >= 10, which pins the boundary.

Evidence

  • Before the fix, on the frozen clock: the named test fails with "the wait read the clock 10000 times without ending".
  • After: 50 of 50 runs in a row pass (cargo test -p codefusion-runner -j2 <name> -- --test-threads=2). All 16 release_update tests pass.
  • Putting > back makes the 14-looks assertion fail (15 looks).
  • cargo fmt --check and cargo clippy --all-targets -D warnings on the crate are clean.

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

With no wait, the loop's last look was `elapsed + every > wait`: 0 > 0 when
the clock had not moved since the wait began, so it asked GitHub again, and
on a clock that does not move at all it never stopped. A coarse monotonic
clock on a loaded Mac made the test see 4 requests instead of 2.

The last look is now the one after which the next would come at the end of
the wait or past it (>=), and the wait reads an injected Monotonic clock, so
the tests run it on a frozen or stepped clock.
@XeonFX XeonFX added the automerge Merges itself once every check is green label Nov 12, 2026
@codefusion-automerge
codefusion-automerge Bot merged commit 1a15d36 into main Nov 12, 2026
6 checks passed
@codefusion-automerge
codefusion-automerge Bot deleted the fix/release-wait-flake branch November 12, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Merges itself once every check is green

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant