Skip to content

Verify shared request batching across concurrent flow runs #100

Description

@MasterOfBinary

Outcome

Prove the two independent GoBatch APIs compose: concurrent graph runs share one endpoint-scoped RequestBatcher, combine compatible reads, and still return the correct result to each run.

Acceptance

  • Use fixtures for a bulk market lookup and a nontrading bulk lookup; no live provider account required.
  • Create the batcher at caller/application lifetime scope, not once per candidate/run.
  • Exercise repeated keys with distinct request identities, unordered/partial/missing replies, caller cancellation and different deadlines.
  • Show one canceled flow/request cannot cancel unrelated batch members; no cross-run result mix-up, lost reply or duplicate delivery.
  • Show node concurrency includes nodes waiting on the batcher, and max linger flushes even when the runner's node bound is smaller than the handler's max batch size.
  • Bound both runtimes' accepted work and shutdown them in ownership order: stop flow admission and settle/cancel runs before closing their shared batcher.
  • Measure actual call reduction, realized batch sizes, queue wait and expiry versus direct bounded calls/grouping; do not promise speedup or automatic batching for singleton endpoints.
  • Extend the published examples and applicable batchlab cases; run race-enabled checks.

Dependencies and boundaries

Blocked by #71 and #98. Related to #97; explicitly not a blocker for #99 or standalone use of either API. No generic retries/coalescing, scheduler, durable execution or provider policy in flow.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions