SoushAI analysis. Drafted by Soushi's AI assistant, reviewed and posted by @Soushi888.
Two CI reliability problems surfaced while stabilising #132. Neither is an application defect and neither blocks that PR, but both are the kind of thing that becomes a mystery failure on somebody else's branch months from now.
1. The resource Sweettest shard is drifting toward the job timeout
Observed on ndo-layer1 across four consecutive runs:
| head |
sweettest (resource) |
note |
ec1240e |
failure |
Run Sweettest target step ended with conclusion: null after a 59m51s run: a timeout, not an assertion |
f57ef43 |
success, 15m15s |
|
b3bf2ae |
success, ~24m |
|
That leaves roughly 24 minutes of headroom against a 60-minute ceiling on a suite that only grows. #132 alone adds four Sweettests to this target, and the resource target already takes about 790 seconds locally for a single pass.
The failure mode is nasty because it does not look like a timeout from the outside. gh run list reports the run as red, the job reports as failed, and the step that "failed" is the one that runs the tests, so the natural reading is that a test broke. It took reading conclusion: null on the step to see what actually happened.
It also has a second-order effect: e2e is gated behind the shards, so a timed-out shard silently skips e2e entirely. During #132's review that produced a run where e2e appeared to have been failing for several commits when in fact it had not run at all.
Options, roughly in order of cost: shard resource further the way the targets are already split; raise the job timeout and accept longer feedback; or look at why individual Sweettests are slow, since conductor startup dominates and tests that could share a fixture currently do not.
2. The empty-lobby precondition flakes on leaked conductor state
Error: precondition failed: expected an empty lobby, found 1 group(s) in the sidebar
(E2E Circle). Some earlier test leaked conductor state — check for a clone cell or
group that was created and never torn down.
core-flows.spec.ts:159 › empty lobby shows the create-or-join onboarding CTA failed twice on retry in CI and then passed locally on a full-file run in 29ms, so it is ordering or environment specific rather than an app defect. The helper's own message names the likely cause. This is the same class as the clone-cell leak fixed previously in #127, so it is worth checking whether that fix covers every creation path or only the one it was written for.
One green local run is weak evidence for a flake, so this should be watched rather than closed on the next passing build.
Two CI reliability problems surfaced while stabilising #132. Neither is an application defect and neither blocks that PR, but both are the kind of thing that becomes a mystery failure on somebody else's branch months from now.
1. The
resourceSweettest shard is drifting toward the job timeoutObserved on
ndo-layer1across four consecutive runs:sweettest (resource)ec1240eRun Sweettest targetstep ended withconclusion: nullafter a 59m51s run: a timeout, not an assertionf57ef43b3bf2aeThat leaves roughly 24 minutes of headroom against a 60-minute ceiling on a suite that only grows. #132 alone adds four Sweettests to this target, and the
resourcetarget already takes about 790 seconds locally for a single pass.The failure mode is nasty because it does not look like a timeout from the outside.
gh run listreports the run as red, the job reports as failed, and the step that "failed" is the one that runs the tests, so the natural reading is that a test broke. It took readingconclusion: nullon the step to see what actually happened.It also has a second-order effect:
e2eis gated behind the shards, so a timed-out shard silently skips e2e entirely. During #132's review that produced a run where e2e appeared to have been failing for several commits when in fact it had not run at all.Options, roughly in order of cost: shard
resourcefurther the way the targets are already split; raise the job timeout and accept longer feedback; or look at why individual Sweettests are slow, since conductor startup dominates and tests that could share a fixture currently do not.2. The empty-lobby precondition flakes on leaked conductor state
core-flows.spec.ts:159 › empty lobby shows the create-or-join onboarding CTAfailed twice on retry in CI and then passed locally on a full-file run in 29ms, so it is ordering or environment specific rather than an app defect. The helper's own message names the likely cause. This is the same class as the clone-cell leak fixed previously in #127, so it is worth checking whether that fix covers every creation path or only the one it was written for.One green local run is weak evidence for a flake, so this should be watched rather than closed on the next passing build.