fix(pump): forward the notifier type discriminator to the data pump

[jbiskur]

Description

buildNotifierOptions() emitted a shape @flowcore/data-pump does not read. The type discriminator was never sent and the key names differed:

// pathways sent
{ dataSource, auth, pollerIntervalMs }   // no `type`
{ dataSource, auth, natsServers }

// data-pump reads
options.notifier?.type === "poller" ? options.notifier.intervalMs : undefined
options.notifier?.type === "nats"   ? options.notifier.servers   : undefined

Both branches resolved to undefined, so every pump fell through to waitWebSocket. poller and nats have been dead options for every consumer. The nested dataSource / auth were also ignored — the notifier takes both from the top-level pump options.

This emits FlowcoreDataPumpNotifierOptions exactly, typed against a local mirror so the shape cannot drift silently across the dynamic-import boundary.

Found while investigating "Lost event.stored.* notifications stall pathway consumers" (board task 5c0bce51-158f-4501-b706-f45f17a447c8), which reports platform-wide 500s on writes that actually succeeded.

⚠️ Behaviour change — read before merging

Consumers on the default or explicit { type: "websocket" } are unaffected (identical code path). Anyone who configured a non-default notifier has been silently running websocket and will now get what they configured:

• nats — the pump really connects. service-tenant-store-api sets NATS_SERVERS in production and has been on websocket since pathways 2.4.0. Confirm reachability from the workload before upgrading; a failing connect drops the pump into restart backoff.
• poller — the pump really polls. Upstream waits Math.min(pollerIntervalMs, 1000), so any interval above one second still polls every second. Worth weighing against current service-event-type-api ingest latency. To be filed separately against data-pump.

Type of Change

• Bug fix (non-breaking change)
• Documentation update
• Breaking change

No public API signature changes. PumpNotifierConfig is unchanged — pollerIntervalMs / natsServers stay the consumer-facing names and are mapped internally.

Testing

• Tests pass locally — 31 passed / 152 steps / 0 failed
• Added tests for new functionality — tests/pathway-pump-notifier.test.ts
• Verified the regression test fails without the fix — 5 of 6 steps fail against the previous implementation
• PostgreSQL tests — not run locally, port 5432 was held by another worktree's container. Unrelated to this diff, but flagging it; CI covers them.

deno fmt --check, deno lint, deno check src/mod.ts all clean.

Also in this PR

• waitForPathwayToBeProcessed now states the timed-out write succeeded and is durable, so callers stop retrying and duplicating a stored event. The message still begins with the original text, so log greps and alerts matching Pathway processing timed out keep working.
• README — new "Notification Delivery and Write Timeouts" section: notifier selection, the notifier-recovery vs pathwayTimeoutMs relationship, fireAndForget on request paths, and an upgrade warning.
• Retargeted the existing per-group event-type assertion at the top-level dataSource, which is the field the notifier actually reads.

Additional Notes

This does not fix the dropped notifications themselves — it makes the workarounds reachable and the failure legible. Two follow-ups remain: forwarding timeoutMs from FlowcoreDataPump.create() into FlowcoreNotifier (currently dead config, pinning recovery at 20s while pathwayTimeoutMs defaults to 10s), and the producer-side investigation into why event.stored.* frames go missing.

🤖 Generated with Claude Code