Skip to content

Refuse a blank owner, and assemble the claim where the owner lives - #1550

Merged
aaylward merged 3 commits into
mainfrom
blank-owner-refused
Sep 14, 2026
Merged

aaylward merged 3 commits into
mainfrom
blank-owner-refused

Conversation

@aaylward

@aaylward aaylward commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Two layers under #1549. Both queues refuse an empty owner with InvalidArgument before touching the row, and V004 adds CHECK (owner_id <> '') to both request tables, nulling existing blanks on the deploy that adds it. Java only ever writes NULL there. PlatformAdmission::Claim no longer sees the owner; ClaimOne assembles the Claim after the queue's last read of the string. The new gated test drives IndexPool through the gate over the real queue SQL to COMPLETED and kills a mutation that blanks the owner for the duration of the claim.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 14, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ast-lol 111139f Commit Preview URL

Branch Preview URL
Sep 14 2026, 07:56 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 14, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
1d4-web 111139f Commit Preview URL

Branch Preview URL
Sep 14 2026, 07:55 PM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 14, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
iili 111139f Commit Preview URL

Branch Preview URL
Sep 14 2026, 07:56 PM

@github-actions github-actions Bot added service:one_d4 Changes the one_d4 image service:one_d4_migrate Changes the one_d4_migrate image service:one_d4_worker Changes the one_d4_worker image labels Sep 14, 2026
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Review panel

Four read-only agents, four lenses (correctness/control flow; data access/SQL/resource safety; tests/docs/CI; altitude). Each hunted then tried to refute. Aggregation re-checked survivors against HEAD 1cdcc7fb. CI on that commit: build-and-test green; sanitize (asan) red; ubsan green; tsan still running at review time.

Headline: the change is the right second layer under #1549 — queue refusal + V004 CHECK + assembling Claim only in ClaimOne after the last owner read. Correctness, SQL, and altitude found nothing that survived refute. One fold-in survives: this head's required build-and-test never ran the new suites.

1 finding in the inline comments; soft notes below.

Verified clean (not reported):

  • Empty-owner check is local and before SQL on both queues; no TOCTOU with FOR UPDATE SKIP LOCKED
  • ClaimOne moves owner only after admission/take returns; removes the Claim under the owner the run will fence on #1549 by-value hazard by API shape, not by a comment on parameter type
  • V004 UPDATE-then-CHECK in one DO block; NULL still allowed (NULL <> '' is unknown); both request tables covered; idempotent via IF NOT EXISTS
  • RefusesToClaimUnderABlankOwner on both queues pins InvalidArgument (not CHECK → InternalError) and leaves the row PENDING / attempts=0
  • Gate e2e ARunTakenThroughTheGateIsFencedAllTheWayToCompleted drives admission + real SQL to COMPLETED, with a 10s bound that bites a blank-owner spin
  • Java constraint test: blank refused on both tables; named + NULL accepted; constraint names visible to boot verify; step idempotent
  • DESIGN.md “NULL or never blank (V004 CHECK)” matches the constraint
  • asan failure is not a memory finding on this change: Resolve target set died on rust-redist 504 Gateway Time-out before any instrumented test ran

Soft notes (not blockers; fold or ignore)

  1. rerunningTheMigrationClearsABlankOwnerStoredBeforeTheConstraint plants a blank only on indexing_requests. The FOREACH is shared; same gap V003's platform test has. A one-line twin on reanalysis_requests would close it.
  2. Whitespace-only owners (" ") pass both owner.empty() and CHECK (owner_id <> ''). Not the Claim under the owner the run will fence on #1549 failure mode.

The code is ready. The CI receipt for it is not — see the inline comment.


Review panel via Cursor

// progress report inside the run, and the terminal write after it. Every one
// of those is fenced on the id the row was claimed under, so this is where a
// claim made under any other id shows up — as a lease nobody took.
TEST_F(PgQueueTest, ARunTakenThroughTheGateIsFencedAllTheWayToCompleted) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should-fix (CI receipt): these suites — and OwnerNeverBlankConstraintTest — have no execution receipt on this head's required build-and-test.

On run 34859493099, scripts/diff-build reported 533 first-party impacted targets, then:

./scripts/diff-build: line 92: /usr/local/bin/bazel: Argument list too long
--- [diff-build] tests to run: 0
./scripts/diff-build: line 97: /usr/local/bin/bazel: Argument list too long
--- [diff-build] targets to build: 0

The bazel query "… set($TARGET_SET) …" failure is swallowed by | grep … || true, so the job exits 0 having run only //:buildifier_test. Postgres was up; nothing asked it to exercise the blank-owner or gate→COMPLETED paths. (#1549's green run had ~64 impacted → real one_d4_worker suites.)

Not a defect in the tests themselves — they look correctly shaped. Either:

  1. fix scripts/diff-build to fail loud (and/or feed targets via --target_pattern_file / query-from-file so ARG_MAX cannot empty the set), or
  2. land that fix and re-run, so this PR's claims have a CI receipt before merge.

asan on the same head is a separate infra miss (rust-redist 504 during Resolve target set) — also not a finding on this code, but another required check that never reached the new tests.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Receipt on the rebased head (https://github.com/muchq/MoonBase/actions/runs/34889778959/job/104129247549): 533 impacted → tests to run: 73, targets to build: 56, executed 73 of 73, all pass — pg_queue_test, reanalysis_queue_test and OwnerNeverBlankConstraintTest among them, against the postgres service.

@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

CI update on 1cdcc7fb: all 24 checks green after asan retry.

asan’s first fail was the rust-redist 504; the retry ran ~45m and passed — confirms it was infra, not a memory finding on this change. Its sanitizer target set also never included pg_queue_test / reanalysis_queue_test (postgres-gated) or the Java constraint suite.

build-and-test is still the same green-with-empty-set run: 533 impacted → ARG_MAX → tests to run: 0. The fold-in stands — these suites still have no CI receipt on this head.

@aaylward

Copy link
Copy Markdown
Collaborator Author

build-and-test on this head ran nothing — 533 impacted targets blew the inlined query past ARG_MAX and the failure was swallowed. Fix is #1551; I'll rebase this once it lands so the new suites get a real run.

@github-actions github-actions Bot added the service:unknown Impacted services could not be computed label Sep 14, 2026
Both queues refuse an empty owner with InvalidArgument before touching
the row, and V004 adds CHECK (owner_id <> '') to both request tables,
nulling existing blanks on the deploy that adds it. A blank is an owner
no fence matches, so a row claimed under one neither completes nor
retires.

PlatformAdmission::Claim returns the job and its slot; ClaimOne moves
the owner into the Claim after the queue's last read of it.
A run that never completes now fails the pool test at its assertion
inside ten seconds rather than as a timeout on the whole target.
@aaylward

Copy link
Copy Markdown
Collaborator Author

Rebased onto main with #1551; build-and-test is re-running and should show the new suites this time. Heal test now covers both tables. Not taking the whitespace-only owner: a " " owner is fenceable (its own heartbeat matches it), so it isn't the case that loops — the CHECK is scoped to the one that does.

@github-actions github-actions Bot removed the service:unknown Impacted services could not be computed label Sep 14, 2026
@aaylward
aaylward merged commit 238f21e into main Sep 14, 2026
24 checks passed
@aaylward
aaylward deleted the blank-owner-refused branch September 14, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

service:one_d4_migrate Changes the one_d4_migrate image service:one_d4_worker Changes the one_d4_worker image service:one_d4 Changes the one_d4 image

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant