macOS builders randomly stops working on GitHub Actions

[emilyalbini]

We started noticing that occasionally one of our macOS builders stop working, marking the job as either failed or canceled and providing no logs for the build. Examples of such failed builds are:

• Rollup of 6 pull requests #71934 - https://github.com/rust-lang-ci/rust/runs/648528503
• Rollup of 7 pull requests #71951 - https://github.com/rust-lang-ci/rust/runs/649952440
• Update cargo #71925 - https://github.com/rust-lang-ci/rust/runs/651724911
• upgrade chalk and use chalk-solve/chalk-ir/chalk-rust-ir #69406 - https://github.com/rust-lang-ci/rust/runs/656100319
• [WIP] Rollup of 5 pull requests #72083:
  • https://github.com/rust-lang-ci/rust/runs/661212134
  • https://github.com/rust-lang-ci/rust/runs/661326448

[emilyalbini]

New one: https://github.com/rust-lang-ci/rust/runs/656100319

[emilyalbini]

Two new ones in a row from #72083:

• https://github.com/rust-lang-ci/rust/runs/661212134
• https://github.com/rust-lang-ci/rust/runs/661326448

[RalfJung]

This broke the 2nd rollup in a row: #72187
Makes it kind of hard to land anything, really.^^

There's no error, just a red cross next to "upload artifacts to S3".

[emilyalbini]

We have a call with Github today, if there is no news we'll rollback the gate on GHA.

[emilyalbini]

GitHub is going to continue looking into this ❤️

In the meantime I'll remove the double-gate, to let the queue run more smoothly.

[emilyalbini]

Removed the gate on GHA.

[RalfJung]

@pietroalbini thanks!
Does this take effect on the already running #72202 (comment) ?

[emilyalbini]

It should.

[Aaron1011]

Possibly related: https://github.com/rust-lang-ci/rust/runs/685622726

[emilyalbini]

Just got another failure related to this:

• https://github.com/rust-lang-ci/rust/runs/834243531