Skip to content

Acknowledge polled outgoing data stream batches once the host has sent them - #1691

Merged
hiroshihorie merged 13 commits into
mainfrom
hiroshi/polled-outgoing-ack-next
Nov 18, 2026
Merged

hiroshihorie merged 13 commits into
mainfrom
hiroshi/polled-outgoing-ack-next

Conversation

@hiroshihorie

@hiroshihorie hiroshihorie commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

The polled outgoing data stream adapter in core-modules acknowledged a send as soon as its packets were buffered. A send_* or write call resolved before anything reached the transport, transport errors were lost, and nothing slowed a producer down. Flutter is the one SDK on this adapter. It shows as unbounded memory growth on a large sendFile and as send errors that never reach the caller. Swift and Android pass their own delegate, which already resolves a send once the packets are handed to the data channel, and this brings Flutter to the same point.

OutgoingPacketQueue.next_packets() is replaced by next_batch(), which returns an OutgoingPacketBatch record: the packets, moved out without a copy, and an ack object with complete() and fail(reason). The send resolves when the host completes the ack. It fails when the ack fails, is released without an outcome, or the queue is closed. The manager already awaits each delegate call, so at most one batch is pending.

The queue is split into two halves. The manager holds the sending half and the host holds OutgoingPacketQueue, the receiving half. Releasing the queue then drops a batch nobody pulled, which fails its send and lets the forwarding task finish. The sending half also watches the token its manager cancels on drop, so the task finishes when the manager goes away, whatever the host still holds. outgoing.rs gains a constructor that takes that token. Its forwarding task is unchanged, so hosts that pass their own delegate are not affected.

This is a breaking change for hosts on the polled adapter, so the changeset is livekit-ffi: major.

Checks

published 0.13.4 #1636 only #1636 and this PR
256 MiB sendFile, peak RSS growth +826 MB +147 MB +45 MB
sendText then disconnect(), localhost never arrives never arrives arrives
other 10 data stream and core scenarios pass pass pass

The same scenarios pass on an iOS simulator and on an iPhone 17 Pro over Wi-Fi, with one exception on the phone: sendText then disconnect() is still not reliable there. A send now resolves when its packets are handed to the data channel, and Flutter's disconnect() does not wait for the channel to drain. The JS, Swift and Android SDKs send the Leave without that wait too, so it is a gap of its own and not part of this PR.

The polled outgoing adapter acknowledged on_packets_available as soon as the
packets were buffered. That broke the delegate contract: send_* and write
resolved before anything reached the transport, transport errors were lost,
and nothing slowed a producer down.

next_packets() is replaced by next_batch(), which hands out an
OutgoingPacketBatch. The delegate call now waits for the host to complete or
fail that batch. A batch released without an outcome, or a closed queue, fails
the pending send. The manager already awaits each delegate call, so at most one
batch is pending and the queue no longer needs depth tracking.
@github-actions

Copy link
Copy Markdown
Contributor

Changeset ✓

This PR includes a changeset covering all affected packages:

Package Bump
livekit-ffi major

The forwarding task awaited on_packets_available outside the race with its
cancellation token. The polled delegate now waits for the host, so a manager
dropped while a batch sat unpulled left the task holding the queue and the
batch forever. The call is raced against shutdown, and a test drops the
manager with an unpulled batch and checks the queue is released.
…uts down"

This reverts commit f03ff49.

The forwarding task is shared with hosts that pass their own delegate, and
they do not have the leak. The fix moves into the polled adapter.
The queue held both ends of its channel and was also the delegate the
manager's forwarding task owns. A batch nobody pulled could then never be
dropped: the task waited on its outcome while keeping the queue, and so the
batch, alive.

The manager now gets a sending half and the host keeps the receiving half.
Releasing the queue drops an unpulled batch, which fails its send and lets
the task finish.
A send is cancelled before its batch is pulled, then the manager and the
queue are released. The forwarding task must finish and drop the sending
half.
@hiroshihorie hiroshihorie changed the title Acknowledge polled outgoing data stream batches after delivery Acknowledge polled outgoing data stream batches once the host has sent them Nov 17, 2026
@hiroshihorie
hiroshihorie requested a review from pblazej November 17, 2026 20:47
A batch was an object whose packets() cloned the list on every call, and
because the batch kept the originals, lowering each packet copied its
payload. A batch is now a record that owns the packets, with a separate
OutgoingBatchOutcome object to report on. Dropping the outcome no longer
builds an error when one was already reported.
The sink waited on the queue's own token only, so a manager dropped while
the host still held the queue or a pulled batch left the forwarding task
waiting. The manager can now be built with the token it cancels on drop,
and the sink watches it. The forwarding task itself is unchanged, so a
host that passes its own delegate sees no difference.
@hiroshihorie
hiroshihorie marked this pull request as ready for review November 17, 2026 21:37
@hiroshihorie
hiroshihorie requested a review from ladvoc as a code owner November 17, 2026 21:37

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@pblazej pblazej left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job 🎉

Two hundred sends go through a polled manager, each ending one of four
ways: the ack completes, it fails, it is released without an outcome, or
the send is cancelled before the ack reports. Every round checks the ack
is freed, and the manager and queue are dropped at the end.
@hiroshihorie
hiroshihorie merged commit 80114cc into main Nov 18, 2026
36 checks passed
@hiroshihorie
hiroshihorie deleted the hiroshi/polled-outgoing-ack-next branch November 18, 2026 09:24
hiroshihorie added a commit to livekit/client-sdk-flutter that referenced this pull request Nov 19, 2026
## Summary

Data streams (`sendText`, `sendBytes`, `sendFile`, `streamText`,
`streamBytes`) on native platforms had three problems since they moved
to the shared Rust core:

- **Memory.** Sending a large file loaded all of it into memory. A 256
MiB file grew the app by about 826 MB. It now grows by about 43 MB.
- **Silent failures.** If a packet could not be sent, the error was only
logged and the call still reported success. The call now throws.
- **Finishing too early.** A send reported "done" as soon as the data
was queued inside the SDK, before any of it went out. It now reports
"done" once the data has been handed to WebRTC, which is what the JS,
Swift and Android SDKs do.

The fix is in the Rust core and shipped in `livekit_ffi` 0.15.0. This PR
moves Flutter to that version and makes the small change needed to use
it. That version also fixes a memory leak in the Dart bindings.

There is no public API change, and the web is not affected.

One thing this does **not** fix: a message sent right before
`disconnect()` can still be lost on a real network. See "Not covered"
below.

## Details

Companion to livekit/rust-sdks#1691. The dependency moves from `^0.13.4`
to `^0.15.0`, which also brings livekit/rust-sdks#1636, the bindings
that free native buffers.

Flutter cannot take callbacks from the Rust core's threads, so it pulls
outgoing packets from a queue. That queue used to acknowledge a send the
moment its packets were queued. In 0.15.0 the host acknowledges each
batch itself.

- The outgoing pump pulls a batch with `nextBatch()`, sends its
`packets`, then calls `ack.complete()`, or `ack.fail()` when the send
throws. The core resolves the send from that. Because the pump only
pulls the next batch after that, a fast producer is now paced by the
data channel.
- Closing a stream writer after the room closed stays quiet. The core
now reports that the trailer could not be sent, and closing is cleanup.

### Not covered

`await sendText()` followed at once by `disconnect()` is still not
reliable on a real network. A send now resolves when its packets are
handed to the data channel, and `disconnect()` does not wait for the
channel to drain. On an iPhone over Wi-Fi a 200 KB text was cut off, and
arrived with a one second pause before the disconnect. The other SDKs
disconnect the same way, so that is a gap of its own.

### Checks

Against the published `livekit_ffi` 0.15.0:

- `flutter analyze` clean, 596 unit tests pass.
- e2e-flutter data stream scenarios. The macOS column is with the
published package. The two iOS columns are from an earlier local build
of the same Rust change:

| | macOS | iOS simulator | iPhone 17 Pro, Wi-Fi |
|---|---|---|---|
| 10 data stream and core scenarios | pass | pass | pass |
| 256 MiB `sendFile`, peak RSS growth | +43 MB | not run | not run |
| `sendText` then `disconnect()` | arrives | arrives | not reliable |

The 826 MB figure is the same file send on macOS with `livekit_ffi`
0.13.4.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants