Skip to content

refactor: split High Throughput Publishing into batching and non blocking confirmation (Enterprise) - #694

Merged
dgafka merged 1 commit into
mainfrom
feat/HighThroughput
Aug 9, 2026
Merged

refactor: split High Throughput Publishing into batching and non blocking confirmation (Enterprise)#694
dgafka merged 1 commit into
mainfrom
feat/HighThroughput

Conversation

@dgafka

@dgafka dgafka commented Aug 9, 2026

Copy link
Copy Markdown
Member

Why is this change proposed?

Warning

Breaking change with no deprecation layer, to an Enterprise feature released in 1.324.0. Every entry in the table below changes name or shape.

High Throughput Publishing is now configured as two named mechanisms — batching and non blocking confirmation — and each provider exposes only the ones it can actually deliver.

Why

One flag stood for two unrelated mechanisms, and only three of the five providers implement both. Kafka, AMQP and SQS publish without blocking on delivery confirmation. DBAL and Redis only coalesce sends into a single round trip that still blocks until the store confirms it. Configuring both groups identically promised users a guarantee two of them cannot give. The flag was also named after asynchrony, which in Ecotone already means background processing over a Message Channel — an unrelated concept.

The failure was visible in the error messages: all five adapters told users to "enable it with withAsyncPublishing()", a method that does not exist on the channel builder they configured.

Before / after

Before After
Kafka, AMQP, SQS channels withHighThroughputPublishing($enabled, $timeout) withHighThroughputPublishing(batchPublishing:, nonBlockingConfirmation:, confirmationTimeoutInMilliseconds:)
DBAL, Redis channels same signature, but only batching ever happened withHighThroughputPublishing() — no parameters
All publisher configurations withAsyncPublishing() withHighThroughputPublishing()
MessagePublisher asyncPublish(): Future publishDeferred(): Future
Namespace Channel\AsyncPublishing Channel\DeliveryConfirmation

Example

// Kafka, AMQP, SQS — both mechanisms, each independently opt-out-able
KafkaMessageChannelBuilder::create('orders')
    ->withHighThroughputPublishing(
        batchPublishing: true,
        nonBlockingConfirmation: true,
        confirmationTimeoutInMilliseconds: 5000,
    );

// DBAL, Redis — batching is all that is on offer, and the signature says so
DbalBackedMessageChannelBuilder::create('orders')
    ->withHighThroughputPublishing();

// Awaiting confirmation explicitly, where the provider can defer it
$publisher->publishDeferred($order)->resolve();

Description of Changes

Not calling the method leaves publishing unchanged. Calling it enables every mechanism the provider supports; each can then be turned off by name.

What each provider does under High Throughput Publishing:

Provider Batching Non blocking confirmation
Kafka producer lingering delivery reports awaited at scope end
AMQP one publisher-confirms round trip confirms awaited at scope end
SQS batch send requests responses awaited at scope end
DBAL multi row insert not offered — the insert blocks
Redis single scripted round trip not offered — the round trip blocks

Migration notes for anyone on 1.324.0:

  • publishDeferred() throws on DBAL and Redis publishers. Use convertAndSend(BatchMessage), which was already the synchronous batch path and is unchanged.
  • The internal namespace rename affects custom outbound adapters: AsyncPublishingRegistry becomes PendingDeliveryRegistry, AsyncPublishingWaiterInterceptor becomes DeliveryConfirmationInterceptor, AsyncPublishingGateway becomes DeferredPublishingGateway.
  • withHighThroughputPublishing(false) changes meaning rather than failing, since the first positional parameter is now batchPublishing. Every other old positional call raises a TypeError or ArgumentCountError.
  • ConfirmedDelivery is removed. It existed only to hand DBAL and Redis an already-resolved Future.

Refines #684, which introduced the feature.

Verified against Ecotone core (1362), AMQP (25), DBAL (14), SQS (8), Redis (10), Kafka (6), Symfony (46) and Laravel (42), plus PHPStan and PHP-CS-Fixer.

Drafted with /ecotone-contributor.

Pull Request Contribution Terms

  • I have read and agree to the contribution terms outlined in CONTRIBUTING.

…king confirmation (Enterprise)

High Throughput Publishing was configured through one flag whose meaning
differed per provider. Kafka, AMQP and SQS genuinely publish without
blocking on confirmation, while DBAL and Redis only coalesce sends into a
single blocking round trip. The flag was also named after asynchrony,
which collides with the Asynchronous attribute.

Configuration is now expressed as two named mechanisms under one method,
and each provider only exposes the ones it can deliver.

BREAKING CHANGE: withAsyncPublishing() is replaced by
withHighThroughputPublishing(); MessagePublisher::asyncPublish() is
replaced by publishDeferred(); the Channel\AsyncPublishing namespace is
renamed to Channel\DeliveryConfirmation. publishDeferred() is no longer
available on DBAL and Redis publishers, which cannot defer confirmation.
@dgafka
dgafka merged commit b9eca82 into main Aug 9, 2026
9 of 10 checks passed
@dgafka
dgafka deleted the feat/HighThroughput branch August 9, 2026 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant