Repository navigation
Acknowledge polled outgoing data stream batches once the host has sent them - #1691
Merged
Merged
Conversation
The polled outgoing adapter acknowledged on_packets_available as soon as the packets were buffered. That broke the delegate contract: send_* and write resolved before anything reached the transport, transport errors were lost, and nothing slowed a producer down. next_packets() is replaced by next_batch(), which hands out an OutgoingPacketBatch. The delegate call now waits for the host to complete or fail that batch. A batch released without an outcome, or a closed queue, fails the pending send. The manager already awaits each delegate call, so at most one batch is pending and the queue no longer needs depth tracking.
Contributor
Changeset ✓This PR includes a changeset covering all affected packages:
|
The forwarding task awaited on_packets_available outside the race with its cancellation token. The polled delegate now waits for the host, so a manager dropped while a batch sat unpulled left the task holding the queue and the batch forever. The call is raced against shutdown, and a test drops the manager with an unpulled batch and checks the queue is released.
…uts down" This reverts commit f03ff49. The forwarding task is shared with hosts that pass their own delegate, and they do not have the leak. The fix moves into the polled adapter.
The queue held both ends of its channel and was also the delegate the manager's forwarding task owns. A batch nobody pulled could then never be dropped: the task waited on its outcome while keeping the queue, and so the batch, alive. The manager now gets a sending half and the host keeps the receiving half. Releasing the queue drops an unpulled batch, which fails its send and lets the task finish.
A send is cancelled before its batch is pulled, then the manager and the queue are released. The forwarding task must finish and drop the sending half.
A batch was an object whose packets() cloned the list on every call, and because the batch kept the originals, lowering each packet copied its payload. A batch is now a record that owns the packets, with a separate OutgoingBatchOutcome object to report on. Dropping the outcome no longer builds an error when one was already reported.
The sink waited on the queue's own token only, so a manager dropped while the host still held the queue or a pulled batch left the forwarding task waiting. The manager can now be built with the token it cancels on drop, and the sink watches it. The forwarding task itself is unchanged, so a host that passes its own delegate sees no difference.
Contributor
There was a problem hiding this comment.
🔍 Devin Review: 1 flag
Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
Two hundred sends go through a polled manager, each ending one of four ways: the ack completes, it fails, it is released without an outcome, or the send is cancelled before the ack reports. Every round checks the ack is freed, and the manager and queue are dropped at the end.
hiroshihorie
added a commit
to livekit/client-sdk-flutter
that referenced
this pull request
Nov 19, 2026
## Summary Data streams (`sendText`, `sendBytes`, `sendFile`, `streamText`, `streamBytes`) on native platforms had three problems since they moved to the shared Rust core: - **Memory.** Sending a large file loaded all of it into memory. A 256 MiB file grew the app by about 826 MB. It now grows by about 43 MB. - **Silent failures.** If a packet could not be sent, the error was only logged and the call still reported success. The call now throws. - **Finishing too early.** A send reported "done" as soon as the data was queued inside the SDK, before any of it went out. It now reports "done" once the data has been handed to WebRTC, which is what the JS, Swift and Android SDKs do. The fix is in the Rust core and shipped in `livekit_ffi` 0.15.0. This PR moves Flutter to that version and makes the small change needed to use it. That version also fixes a memory leak in the Dart bindings. There is no public API change, and the web is not affected. One thing this does **not** fix: a message sent right before `disconnect()` can still be lost on a real network. See "Not covered" below. ## Details Companion to livekit/rust-sdks#1691. The dependency moves from `^0.13.4` to `^0.15.0`, which also brings livekit/rust-sdks#1636, the bindings that free native buffers. Flutter cannot take callbacks from the Rust core's threads, so it pulls outgoing packets from a queue. That queue used to acknowledge a send the moment its packets were queued. In 0.15.0 the host acknowledges each batch itself. - The outgoing pump pulls a batch with `nextBatch()`, sends its `packets`, then calls `ack.complete()`, or `ack.fail()` when the send throws. The core resolves the send from that. Because the pump only pulls the next batch after that, a fast producer is now paced by the data channel. - Closing a stream writer after the room closed stays quiet. The core now reports that the trailer could not be sent, and closing is cleanup. ### Not covered `await sendText()` followed at once by `disconnect()` is still not reliable on a real network. A send now resolves when its packets are handed to the data channel, and `disconnect()` does not wait for the channel to drain. On an iPhone over Wi-Fi a 200 KB text was cut off, and arrived with a one second pause before the disconnect. The other SDKs disconnect the same way, so that is a gap of its own. ### Checks Against the published `livekit_ffi` 0.15.0: - `flutter analyze` clean, 596 unit tests pass. - e2e-flutter data stream scenarios. The macOS column is with the published package. The two iOS columns are from an earlier local build of the same Rust change: | | macOS | iOS simulator | iPhone 17 Pro, Wi-Fi | |---|---|---|---| | 10 data stream and core scenarios | pass | pass | pass | | 256 MiB `sendFile`, peak RSS growth | +43 MB | not run | not run | | `sendText` then `disconnect()` | arrives | arrives | not reliable | The 826 MB figure is the same file send on macOS with `livekit_ffi` 0.13.4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The polled outgoing data stream adapter in
core-modulesacknowledged a send as soon as its packets were buffered. Asend_*orwritecall resolved before anything reached the transport, transport errors were lost, and nothing slowed a producer down. Flutter is the one SDK on this adapter. It shows as unbounded memory growth on a largesendFileand as send errors that never reach the caller. Swift and Android pass their own delegate, which already resolves a send once the packets are handed to the data channel, and this brings Flutter to the same point.OutgoingPacketQueue.next_packets()is replaced bynext_batch(), which returns anOutgoingPacketBatchrecord: thepackets, moved out without a copy, and anackobject withcomplete()andfail(reason). The send resolves when the host completes the ack. It fails when the ack fails, is released without an outcome, or the queue is closed. The manager already awaits each delegate call, so at most one batch is pending.The queue is split into two halves. The manager holds the sending half and the host holds
OutgoingPacketQueue, the receiving half. Releasing the queue then drops a batch nobody pulled, which fails its send and lets the forwarding task finish. The sending half also watches the token its manager cancels on drop, so the task finishes when the manager goes away, whatever the host still holds.outgoing.rsgains a constructor that takes that token. Its forwarding task is unchanged, so hosts that pass their own delegate are not affected.This is a breaking change for hosts on the polled adapter, so the changeset is
livekit-ffi: major.Checks
cargo test -p livekit-ffi --no-default-features --features core-modulespasses, 78 tests. Newpolled_*tests cover the wait, a failed batch, a dropped batch, a closed queue, a queue released with an unpulled batch, a manager dropped while the host keeps the queue or a pulled batch, and 200 repeated sends that check every ack is released.sendFile, peak RSS growthsendTextthendisconnect(), localhostThe same scenarios pass on an iOS simulator and on an iPhone 17 Pro over Wi-Fi, with one exception on the phone:
sendTextthendisconnect()is still not reliable there. A send now resolves when its packets are handed to the data channel, and Flutter'sdisconnect()does not wait for the channel to drain. The JS, Swift and Android SDKs send the Leave without that wait too, so it is a gap of its own and not part of this PR.