Skip to content

feat(mcp)!: the job ticket is Leadbay's own id; delete the file we kept (product#4005) - #197

Open
milstan wants to merge 7 commits into
mainfrom
milstan/4005-v3
Open

feat(mcp)!: the job ticket is Leadbay's own id; delete the file we kept (product#4005)#197
milstan wants to merge 7 commits into
mainfrom
milstan/4005-v3

Conversation

@milstan

@milstan milstan commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Closes leadbay/product#4005. Closes leadbay/product#4039.

Replaces #196, which was closed. Same goal, clean history — that branch was replayed onto a base rather than rebased and silently reverted two merged fixes while deleting their tests. This one is rebased on current main and verified against that specific failure.

The problem, in plain terms

Three tools — qualify a batch of leads, import leads, import-and-qualify — fail on the Leadbay server that Claude and ChatGPT connect to. Nine customers hit it, 53 times. leadbay_import_and_qualify has never once worked there.

The cause: the MCP keeps its own file listing the jobs it has started, so it can hand the assistant a ticket to check on them later. That file is only ever created by the version people install on their own computer. The server version runs a different entry point and never creates it, so every job that needs a ticket fails instantly.

Why we delete the file instead of fixing it

The file exists because of a note in its own header: "while the Leadbay backend doesn't yet issue a real bulk_id." It does now, in two places:

Covers What it gives
GET /1.6/mcp/jobs/{id} search and qualify server-generated id, kept 30 days, returns 404 to another organization
GET /1.6/notifications enrichment, import, qualification per-account, progress counts, a finished flag

Every job the MCP starts already receives one of these and already hands it to the assistant. Our file was a second, worse copy: an id we invented, no protection between customers, and it dies whenever the server restarts.

So the ticket becomes the Leadbay id. Nothing is stored. That also removes the whole question of where to store it — no disk to attach, no volume to configure, and none of the customer's imported rows sitting on our server.

Proven live, twice

Before writing any code, against staging: starting a job returns an id, that id is listable one second later with progress counts, and it resolves from three separate processes with fresh logins.

After, through the MCP server itself: start a job, then check on it from two brand-new server processes. Resolved both times. That is a server restart, live, through the real tools.

What changes for the assistant

Tool Before After
enrich_titles, bulk_qualify_leads our invented id Leadbay's notification_id + the lead ids
import_leads, import_and_qualify our invented id Leadbay's importIds
bulk_enrich_status, qualify_status demanded our id take notification_id
import_status takes importIds, which it already accepted

Breaking: bulk_id, handle_id and qualify_id are gone as inputs, with no transition period. They were valid for minutes on a local install and never worked on the server, so there is nothing to transition.

What is kept

A small in-memory guard, alive for five minutes, so that if the assistant fires the same request twice we do not charge the customer twice. It does not need to survive a restart: losing it costs at most one duplicate inside a five-minute window, which is the right trade for not owning a datastore.

This also closes product#4039. The earlier version of that guard had a real flaw, raised in review on #196: while a job was still being started, a second identical request was told "it's running" and handed a ticket that did not exist yet. On the server that is an unrecoverable answer — importIds is the only ticket, so an empty one means the rows are uploaded somewhere the assistant can never find. Fixed by making "still starting" an explicit state the callers have to read, rather than something they infer.

Guarding the failure that caused #4005 in the first place

The original bug was the server being built without the file while every test passed. packages/mcp/test/http-no-store-e2e.test.ts now drives the real server on a real socket and checks that a second, separate request can resolve the id. There is no longer an option to leave out, so the two versions cannot drift apart again.

Checked against how #196 went wrong

Also

Eight tool descriptions, four prompts and three shared snippets rewritten. qualify_status had been telling the assistant its ticket was "persisted to ~/.leadbay/bulks.json with a 30-day TTL and survives MCP restart" — every clause of that was false.

Version 0.35.0 (renumbered after main released 0.34.0): this is a breaking change and should read as one.

Not in this change

product#4032 and product#4033 land next, on top of this.

🤖 Generated with Claude Code

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Synced with main; @leadbay/mcp is 0.34.0 (main shipped 0.33.2).

milstan added a commit that referenced this pull request Sep 2, 2026
…b dead end (PR #197 review)

Five defects from the #197 review. The first two are the ones that mattered:
the rewrite lost capability it never had to lose.

Per-lead progress and channel-aware 'done' were computed from the bulk store,
so they went out with it. They never needed the store — they needed lead_ids,
titles, email and phone, all of which enrich_titles already returns and the
agent already carries. Restored on the lead_ids path, along with the
title-scoping that stops a lead's pre-existing CFO email inflating a CEO run,
and the rule that a contact is done only once the channel THIS run requested
has landed. A phone-only run no longer counts an email-enriched contact as
finished. The paragraph in enrich-titles telling the agent to redo that by
hand with get_contacts is gone; the tool does it again.

bulk_enrich_status now takes the same anyOf as qualify_status, so lead_ids
alone is a valid call. That removes a dead end: the job lookup is a scan, not
a fetch-by-id — the backend routes no GET /notifications/{id} — so an
archived job was unfindable, and we ship the tool that archives it. New
shared readNotificationById also searches the archived set and pages, and the
two error hints no longer claim a 30-day org-scoped retention that does not
exist.

Also removed four instructions to use handles that no longer exist —
import_leads' handle_id in its param description and outputSchema,
enrich-titles' bulk_id and durability ('persisted bulks.json'), and dead
handle_id spreads in import_and_qualify. enrich-titles declared re_used while
the code returned reused, and listed a mode the code no longer returns while
omitting one it does. Restored the ctx.progress ticks and the titles / email /
phone echo, so the reply states its own scope.

Ten new tests across three files, covering the phone-only case, title
scoping, partial_failures, and lookup of an archived or paged-past job —
none of which had coverage.

2187 tests passing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@milstan

milstan commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

[Claude]: all five defects fixed. build-and-test green, 2,191 tests.

1 + 2 — per-lead progress and channel-aware done, restored. You were right that neither needed the store. They needed lead_ids, titles, email, phone — all returned by enrich_titles and carried by the agent. bulk_enrich_status takes them as params now and computes the rollup exactly as main did: enrichment_progress:{done,total} per lead, counting only contacts whose requested channel has landed, scoped to the titles this run enriched. A phone-only run no longer counts an email-enriched contact as done. The paragraph telling the agent to redo this by hand with get_contacts is deleted — it existed only to paper over the loss.

3 — the archived dead end is closed. bulk_enrich_status now has the same anyOf as qualify_status, so lead_ids alone is a valid call and answers fully. New shared notifications/read-by-id.ts also searches the archived set and pages, so a job the agent archived with our own acknowledge_notification is still findable. And I removed the two false claims — there is no GET /notifications/{id} on the backend, so the hint now says what the lookup actually does instead of promising a 30-day org-scoped retention.

4 — the four dead instructions are gone, plus two you did not list: enrich-titles declared re_used while the code returns reused, and listed launched_tracker_pending (never returned) while omitting launch_in_flight (returned).

5 — ctx.progress ticks restored with the fan-out, titles/email/phone echoed again, and partial_failures now has coverage.

Tests: 10 new across 3 new files — the phone-only case, title scoping, partial_failures, and finding an archived or paged-past job. None of those had a test.

What a user can do after this that they could not before: start an enrichment, qualification or import on mcp.leadbay.app and poll it — all of it threw BULK_TRACKER_UNAVAILABLE there. Come back to a job in a later conversation or the next day. Poll a job they have already archived.

What they can no longer do: pass bulk_id / handle_id / qualify_id. Those handles lived minutes on local installs and never worked on hosted. No other capability is reduced — that was the point of this round.

Verified: runWithRequestSignal 3/3 vs main, fetchTerminalNotifications 2/2, and the only test files absent from main are the three that tested the deleted store. Live hosted verification after deploy, as you asked.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid rewrite overall — the launch-guard claim/settle split correctly closes the product#4039 race, and readNotificationById's archived+paginated scan fixes the dead-end from the prior review round. Two issues before merge, both stemming from the bulk_idnotification_id rename not being swept everywhere it needed to be:

  1. enrich-titles.ts accidentally added the new notification_id/lead_ids/reused output fields to inputSchema instead of outputSchema — the tool's own declared contract no longer documents the job id it mints. Worse, at least one real consumer (packages/components/src/runtime.ts's enrichment() helper) still reads the deleted bulk_id field and will now silently report every enrichment launch as instantly complete with no job ever polled. getting-started.ts has the same stale reference in its onboarding-tour prompt text.
  2. bulk-enrich-status.ts's outputSchema.required still claims notification_id is always returned, but the new lead_ids-only path can omit it.

Neither is exercised by this PR's own tests since the boundary is mocked, so a repo-wide grep for bulk_id/handle_id/qualify_id before merge would be worthwhile given this is explicitly a no-transition-period breaking change.

reused: {
type: "boolean",
description: "True when an identical launch inside the 5-minute window was reused instead of spending quota again.",
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

notification_id, lead_ids and reused are declared under inputSchema.properties here, not outputSchema. Their descriptions ("Carry it to leadbay_bulk_enrich_status", "True when an identical launch...was reused") are plainly describing return values — and execute() does in fact return all three (lines 88/94/145). But outputSchema (line 260+) previously documented the equivalent fields as bulk_id/re_used/durability and this PR deletes them without adding replacements back — so the schema for the very field this whole PR introduces as the new job handle (notification_id) is now undocumented in the output, while incorrectly implied to be a launch parameter. EnrichTitlesParams (top of file) confirms execute() never reads notification_id/lead_ids/reused from input, so as inputs they're just inert — this looks like the three properties were pasted into the wrong schema block.

Separately: packages/components/src/runtime.ts's enrichment() helper (not touched by this PR) still reads r.bulk_id from the leadbay_enrich_titles result and calls leadbay_bulk_enrich_status({bulk_id}). With bulk_id gone, bulkId is now always null, so every real launch takes the "no job" branch and reports {all_done:true, no_job:true} immediately — silently faking completion for any artifact/widget built on that SDK helper. packages/components/test/domain.test.ts didn't catch it because it mocks the tool call directly with {bulk_id:"b1"} rather than exercising the real (renamed) core output. Worth a repo-wide grep for bulk_id/handle_id/qualify_id before merge — packages/core/src/composite/getting-started.ts:435 also still tells the onboarding-tour agent to poll "with the returned bulk_id".

},
},
required: ["bulk_id", "status", "leads", "overall_progress", "all_done"],
required: ["notification_id", "status", "leads", "overall_progress", "all_done"],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

outputSchema.required lists notification_id as always present, but the new lead_ids-only path (no notification_id passed in) returns an object that omits the notification_id key entirely (...(params.notification_id ? { notification_id: ... } : {}) further down). The declared schema no longer matches what a lead_ids-only call actually returns.

milstan added a commit that referenced this pull request Sep 2, 2026
…s ids (PR #197 review)

Repo-wide sweep for bulk_id / handle_id / qualify_id / BulkTracker after the
store was deleted. What was still reading the old contract:

- enrich_titles declared notification_id, lead_ids and reused under
  inputSchema; they are outputs. Moved.
- The artifact SDK's lb.enrichment() (packages/components) read r.bulk_id and
  polled bulk_enrich_status({bulk_id}), so every widget enrichment reported
  done instantly with no job polled. It now carries notification_id + lead_ids
  and forwards titles/email/phone so the per-lead rollup is scoped. Its test
  mocked the deleted field and had to change; artifact-runtime.generated.ts is
  regenerated from it.
- The live eval harness imported LocalBulkStore from @leadbay/core (no longer
  exported) and plumbed LEADBAY_BULK_STORE_PATH; both removed.
- core's import-and-qualify live smoke imported jobs/bulk-store.js and asserted
  qualify_id / BULK_NOT_FOUND; rewritten to lead_ids + lens_id and
  QUALIFY_JOB_NOT_FOUND. mcp's live smoke polled bulk_enrich_status({bulk_id});
  now LEADBAY_TEST_NOTIFICATION_ID.
- Copy that told the assistant to poll with a handle that no longer exists:
  getting-started tour text, bulk_qualify_leads and import_and_qualify schema
  descriptions, the cancellation paragraph in server instructions + README, the
  qualify_top_n prompt's "BulkTracker-not-configured" rule, and the dead
  LEADBAY_BULK_STORE_ALLOW_MEMORY env in the dxt smoke.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review on commit 6296313 (previous review at 252dfd9 requested changes).

Both prior findings are fixed: notification_id/lead_ids/reused moved from inputSchema to outputSchema in enrich-titles.ts, with the stale bulk_id reference in runtime.ts's enrichment() helper and getting-started.ts's onboarding prompt both updated to the new handle shape; and bulk-enrich-status.ts's outputSchema.required no longer claims notification_id is always present.

The new code in this diff (per-kind ENRICH_JOB_WRONG_KIND/QUALIFY_JOB_WRONG_KIND guards using inferKind, the ENRICH_JOB_NO_COUNTERS path, and the qualify-status not-found fallback when lead_ids+lens_id are supplied) is well covered by the new bulk-enrich-status-kind.test.ts and qualify-status-kind.test.ts unit tests, and I traced the logic against those tests without finding defects. CHANGELOG/README/prompt-template updates are consistent with the code changes.

No new significant issues found. (Note: I was unable to invoke the review-dismissal endpoint directly in this environment — posting this as a COMMENT review instead, since no outstanding issues remain from the prior CHANGES_REQUESTED review.)

@github-actions github-actions Bot added the needs-manual-rebase PR conflicts with a version bump on main — needs a manual rebase label Sep 2, 2026
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

main shipped 0.33.3. Bump this PR's version above 0.33.3 (package.json + server.json) and re-anchor your CHANGELOG section above main's, then push. The sync bot won't edit CHANGELOG prose.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

main shipped 0.33.4. Bump this PR's version above 0.33.4 (package.json + server.json) and re-anchor your CHANGELOG section above main's, then push. The sync bot won't edit CHANGELOG prose.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

main shipped 0.34.0. Bump this PR's version above 0.34.0 (package.json + server.json) and re-anchor your CHANGELOG section above main's, then push. The sync bot won't edit CHANGELOG prose.

milstan and others added 6 commits September 2, 2026 16:02
… (product#4005, product#4039)

Replaces branch milstan/4005-v2, which reverted two merged PRs. Rebuilt on
9580fad by re-applying the change rather than replaying the branch.

The store existed, per its own header, 'while the Leadbay backend doesn't
yet issue a real bulk_id'. It does now: GET /1.6/mcp/jobs/{id} (mcp_jobs)
and GET /1.6/notifications, which the backend ADR calls the single
user-facing unit for asynchronous operations. Both durable and org-scoped.

Launches return the backend's id; status tools poll by it. Deleted
jobs/bulk-store.ts, ToolContext.bulkTracker, both entrypoint wirings, eight
BULK_* codes and three env vars — no volume, nothing at rest on the pod or
on the user's disk.

The double-launch guard now CARRIES its state instead of leaving the caller
to infer it. beginLaunch returns owned | in_flight | settled. Reading an
in-flight claim as settled is what produced status:'running' with a null
notification_id or an empty importIds — and since importIds is the only
handle on hosted, that answer loses the import for good. import_and_qualify
also abandons its claim when the fan-out throws, and no longer logs a skip
it does not perform (product#4039).

Deletions against main are bulk-store.ts and its own tests only. Every test
file from #182, #191, #192, #195 is present and passing; runWithRequestSignal
and fetchTerminalNotifications are intact.

2177 tests passing, typecheck clean.

BREAKING CHANGE: bulk_id, handle_id and qualify_id are gone as inputs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…b dead end (PR #197 review)

Five defects from the #197 review. The first two are the ones that mattered:
the rewrite lost capability it never had to lose.

Per-lead progress and channel-aware 'done' were computed from the bulk store,
so they went out with it. They never needed the store — they needed lead_ids,
titles, email and phone, all of which enrich_titles already returns and the
agent already carries. Restored on the lead_ids path, along with the
title-scoping that stops a lead's pre-existing CFO email inflating a CEO run,
and the rule that a contact is done only once the channel THIS run requested
has landed. A phone-only run no longer counts an email-enriched contact as
finished. The paragraph in enrich-titles telling the agent to redo that by
hand with get_contacts is gone; the tool does it again.

bulk_enrich_status now takes the same anyOf as qualify_status, so lead_ids
alone is a valid call. That removes a dead end: the job lookup is a scan, not
a fetch-by-id — the backend routes no GET /notifications/{id} — so an
archived job was unfindable, and we ship the tool that archives it. New
shared readNotificationById also searches the archived set and pages, and the
two error hints no longer claim a 30-day org-scoped retention that does not
exist.

Also removed four instructions to use handles that no longer exist —
import_leads' handle_id in its param description and outputSchema,
enrich-titles' bulk_id and durability ('persisted bulks.json'), and dead
handle_id spreads in import_and_qualify. enrich-titles declared re_used while
the code returned reused, and listed a mode the code no longer returns while
omitting one it does. Restored the ctx.progress ticks and the titles / email /
phone echo, so the reply states its own scope.

Ten new tests across three files, covering the phone-only case, title
scoping, partial_failures, and lookup of an archived or paged-past job —
none of which had coverage.

2187 tests passing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s ids (PR #197 review)

Repo-wide sweep for bulk_id / handle_id / qualify_id / BulkTracker after the
store was deleted. What was still reading the old contract:

- enrich_titles declared notification_id, lead_ids and reused under
  inputSchema; they are outputs. Moved.
- The artifact SDK's lb.enrichment() (packages/components) read r.bulk_id and
  polled bulk_enrich_status({bulk_id}), so every widget enrichment reported
  done instantly with no job polled. It now carries notification_id + lead_ids
  and forwards titles/email/phone so the per-lead rollup is scoped. Its test
  mocked the deleted field and had to change; artifact-runtime.generated.ts is
  regenerated from it.
- The live eval harness imported LocalBulkStore from @leadbay/core (no longer
  exported) and plumbed LEADBAY_BULK_STORE_PATH; both removed.
- core's import-and-qualify live smoke imported jobs/bulk-store.js and asserted
  qualify_id / BULK_NOT_FOUND; rewritten to lead_ids + lens_id and
  QUALIFY_JOB_NOT_FOUND. mcp's live smoke polled bulk_enrich_status({bulk_id});
  now LEADBAY_TEST_NOTIFICATION_ID.
- Copy that told the assistant to poll with a handle that no longer exists:
  getting-started tour text, bulk_qualify_leads and import_and_qualify schema
  descriptions, the cancellation paragraph in server instructions + README, the
  qualify_top_n prompt's "BulkTracker-not-configured" rule, and the dead
  LEADBAY_BULK_STORE_ALLOW_MEMORY env in the dxt smoke.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nt without counters answers honestly (product#4005 live e2e)

Found on staging, one fresh server process per call:

- bulk_enrich_status fed a QUALIFICATION's notification_id reported it as a
  finished enrichment (2/2, all_done). qualify_status fed an ENRICHMENT's id
  reported a running qualification. Neither checked what the notification was.
  Both now use inferKind (links[].type / file_import_id / bulk_progress) and
  refuse with ENRICH_JOB_WRONG_KIND / QUALIFY_JOB_WRONG_KIND naming the right
  tool. This is the BULK_WRONG_KIND guard the store had, restored on the
  backend's id.
- The finished enrichment row on staging carried in_progress + title + the
  bulk_enrichment link but NO bulk_progress, so a notification_id-only poll
  said "not a bulk job". It now answers ENRICH_JOB_NO_COUNTERS with the
  backend's running/finished flag and asks for the lead_ids the launch
  returned, which always answer (verified: the lead_ids path resolves the same
  job after it was archived).
- qualify_status threw QUALIFY_JOB_NOT_FOUND even when the caller had passed
  lead_ids + lens_id, the fallback its own hint recommends. It falls through
  to the per-lead path now, as bulk_enrich_status already did.
- bulk_enrich_status's outputSchema required notification_id, which the
  lead_ids-only path omits. Dropped from required.
- Both templates and the qualify_status hint no longer claim a 30-day,
  organization-scoped retention the backend does not have.
- CHANGELOG: the 0.34.0 entry was missing.

Tests: bulk-enrich-status-kind.test.ts, qualify-status-kind.test.ts (new
files), using the exact rows staging returned.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The branch was renumbered from 0.34.0 to 0.35.0 after main released 0.34.0
(ChatGPT connector URL). Every release on main carries a customer-facing entry
in the root CHANGELOG; this PR never had one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One line: the notifications-inbox snippet now says notification_id inside
account_status, whose template main changed in #201.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Caught by test/audit/server-json-version.test.ts after the renumbering.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-manual-rebase PR conflicts with a version bump on main — needs a manual rebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant