Skip to content

Define source ownership and cancellation without silent in-flight loss or blocked sends #83

Description

@MasterOfBinary

Current outcome

Clarify ownership of a value dequeued from a caller channel but not yet accepted by the engine, and make source shutdown observable and bounded for cooperative consumers.

Replanned on 2026-09-06 against GoBatch master 63ef757 and ShitQuant's recorder, enrichment, paper/replay, and flow workloads. This is an implementation target, not a claim that the behavior already exists.

Required behavior

  • Specify the input-dequeue, source-forward and engine-acceptance boundaries and the terminal disposition of a value between them.
  • Resolve source.Channel and source.Error mid-forward cancellation together under the lifecycle policy.
  • Do not use unconditional forwarding that can block forever when CancelStop or a standalone consumer stops receiving.
  • Document abort/drop limitations and surface cancellation where the source contract permits; do not promise durable or exactly-once delivery from an in-memory channel.
  • Ensure full buffers, saturated admission and stopped output consumers cannot leak an internal source goroutine under the documented shutdown contract.

Verification and completion

Use synchronized dequeue/forward barriers with full buffers and engine/standalone consumers. Assert the chosen ownership/disposition and termination behavior on cancellation, EOF and stop.

Follow the repository's formatting, race-test, vet/lint, package documentation, example and changelog requirements for the changed surface. Report the actual supported behavior and migration; do not treat a passing coverage percentage as proof of these outcomes.

Scope and relationships

Coordinate with lifecycle and PR #76; #73 adds saturation cases. Source acceptance is not the same contract as an admitted RequestBatcher reply.

Design history

The earlier report/prototype remains below for provenance; the requirements above supersede conflicting prescriptions. Existing discussion is preserved.

Original issue: source.Channel / source.Error can drop one in-flight value when ctx is canceled mid-forward

On master @ 58cee73, source/channel.go:48-57: after an item has been consumed from the user's Input channel, the forwarding select races cancellation against delivery:

case item, ok := <-s.Input:
    ...
    select {
    case <-ctx.Done():
        return
    case out <- item:

When ctx is already canceled, Go's pseudo-random select choice can take the ctx.Done() arm even though out has buffer space — the item was irreversibly dequeued from the caller's channel but never enters the pipeline, with no error emitted. That contradicts the library's documented stance that items already read are still processed. The same pattern can drop one in-flight error in source/error.go (~:60).

Trigger: cancel ctx while the source is forwarding; lose up to one item (or one error) per cancellation.

Options: (a) attempt a non-blocking send to out before honoring ctx.Done(); (b) restructure so the dequeue and delivery are a single committed step (only select on ctx.Done() before consuming from Input, then deliver unconditionally — safe for the engine, which drains until close, but should be documented for non-engine consumers); or (c) document the at-most-one-loss semantics explicitly. (a) or (b) preferred — silent loss is the worst of the three.

Relations: complements #76 (CancelStop mode); found in a deep review of the source package.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions