Repository navigation
Description
Describe the bug
While a room recovers from a network drop, it can end with RoomDisconnectedEvent(reason: DisconnectReason.duplicateIdentity), even though only one client uses that identity. The reconnect had already succeeded, and the connection was healthy when the event arrived.
duplicateIdentity is supposed to mean another participant joined with the same identity, so apps treat it as "you were replaced" and don't retry. Here, the other session is one this client opened during its own reconnect.
The server is behaving correctly. The client ends up with two sessions for one identity, the server kicks the older one, and the SDK applies that kick to the session it is actually using.
Root cause
SignalClient.connect() (lib/src/core/signal_client.dart, main @ 2.13.0):
var future = _wsConnector(rtcUri, options: WebSocketEventHandlers(
onData: _onSocketData, onDispose: _onSocketDispose, onError: _onSocketError), ...);
future = future.timeout(connectOptions.timeouts.connection);
_ws = await future;Future.timeoutdoes not cancelio.WebSocket.connect. On a slow link the handshake can finish after the timeout. The server then has a live session S1 for that attempt.- That socket is never assigned to
_ws, socleanUp()can never close it.LiveKitWebSocketIOhas already subscribed it to the shared_onSocketData, so it keeps delivering messages as if it were current. - The engine retries, and the next join creates session S2, which is the one we keep. The server sees two sessions for one identity and sends
Leave(DISCONNECT, DUPLICATE_IDENTITY)to S1, which is the orphaned socket. - The leave comes through
_onSocketDataas an ordinarySignalLeaveEvent. The engine handler (engine.dart:~1603) runssignalClient.cleanUp()anddisconnect(), which tears down S2 and emitsRoomDisconnectedEvent(reason: duplicateIdentity). Signal is already disconnected at that point, so noleaveis sent for S2 either. The server keeps S2 as a stale participant until it times out, and other participants go on seeing it in the room.
To reproduce
Deterministic on an Android emulator (reproduced 5+ times):
- Use a short connect timeout so the race is easy to hit:
ConnectOptions(timeouts: Timeouts(connection: Duration(seconds: 3), ...)). - Turn on verbose logs:
hierarchicalLoggingEnabled = true; Logger('livekit').level = Level.ALL;. - Emulator: Wi-Fi off, so traffic goes over the emulated mobile-data link.
adb emu network delayonly affects that link. - Join a room.
- Airplane mode on for ~40 s, so the server drops the session.
adb emu network delay 5000. Every packet now takes longer than the connect timeout.- Airplane mode off.
About 30 s later the logs show:
Leave received, action: RECONNECT, reason: STATE_MISMATCH // resume rejected, server already dropped us
EngineFullRestartingEvent // full rejoin #1
Received JoinResponse // S1's handshake finished...
Connect Error TimeoutException after 0:00:03 // ...but the attempt had already timed out
EngineFullRestartingEvent // full rejoin #2 -> S2
Leave received, action: DISCONNECT, reason: DUPLICATE_IDENTITY // meant for S1, handled as S2's
RoomDisconnectedEvent(reason: duplicateIdentity)
Expected: the room stays connected on S2, S1's socket is closed, and no RoomDisconnectedEvent is emitted.
Actual: RoomDisconnectedEvent(reason: duplicateIdentity) on a single client.
We also see this in production on phones with poor mobile data: one device, one identity, and a duplicateIdentity disconnect right after a run of reconnect attempts.
Relation to #1211
#1211 adds a per-socket generation so callbacks from a replaced socket are ignored. Once the retry bumps the generation, that would also drop the late leave above. This issue is a different trigger (a timed-out attempt instead of a disposed one), and two gaps remain:
- the late socket is never closed, so it stays open and holds a server session until the server closes it;
- between the timeout and the next
connect()(e.g. during retry backoff), the orphan's callbacks still carry the current generation and are processed.
I have a fix with tests for this locally and plan to open a PR.
Environment
- livekit_client 2.7.0 in production, same code on main (2.13.0)
- Android (emulator repro) and iOS (production)
- LiveKit server (self-hosted)
Activity