Add MatrixRTC + LiveKit voice call support by Nightchicken · Pull Request #670 · ulyssa/iamb

Nightchicken · 2026-09-03T16:18:43Z

Adds an optional voip feature providing voice calls over MatrixRTC, using LiveKit as the SFU.

The new src/voip module contains:

matrix_rtc.rs: homeserver-facing signalling. Discovers the LiveKit focus from m.call.member state, exchanges an OpenID token for an SFU JWT via lk-jwt-service, and publishes our own membership.
livekit_session.rs: the LiveKit room connection, the E2EE key ring, and the published microphone track.
devices.rs: audio device enumeration, matching, and remembered choices.
mod.rs: CallManager/CallSession state machine and the key exchange event types.

Calls in encrypted rooms are end-to-end encrypted. Per-participant keys are distributed over Olm-encrypted to-device messages (io.element.call.encryption_keys), and re-sent to late joiners. In unencrypted rooms audio is sent in the clear, matching the room.

Audio capture and playback are both handled by LiveKit's platform audio device module, so no additional audio dependency is needed.

User-facing surface is the :call command (join, hangup, decline, mute, unmute, devices, device mic|speaker <name>), a participant banner in the room view, and a room-list indicator for rooms with an active call.

The feature is off by default; without --features voip the build is unchanged.

Claude-Session: https://claude.ai/code/session_01MNTPSETtCAzUuLbn4fzG9a

VAWVAW

I haven't looked at this in depth, but here are some of my initial thoughts:

if you add a substantial feature flag, you should also add a CI test to make sure everything works with the feature enabled and disabled
I'm not a fan of the conditional use statements. The imports are already very cluttered and I feel this makes it potentially even worse. I would use fully qualified calls whenever possible and put the imports that are necessary in their own section.

Adds an optional `voip` feature providing voice calls over MatrixRTC, using LiveKit as the SFU. The new `src/voip` module contains: - `matrix_rtc.rs`: homeserver-facing signalling. Discovers the LiveKit focus from `m.call.member` state, exchanges an OpenID token for an SFU JWT via lk-jwt-service, and publishes our own membership. - `livekit_session.rs`: the LiveKit room connection, the E2EE key ring, and the published microphone track. - `devices.rs`: audio device enumeration, matching, and remembered choices. - `mod.rs`: `CallManager`/`CallSession` state machine and the key exchange event types. Calls in encrypted rooms are end-to-end encrypted. Per-participant keys are distributed over Olm-encrypted to-device messages (`io.element.call.encryption_keys`), and re-sent to late joiners. In unencrypted rooms audio is sent in the clear, matching the room. Audio capture and playback are both handled by LiveKit's platform audio device module, so no additional audio dependency is needed. User-facing surface is the `:call` command (`join`, `hangup`, `decline`, `mute`, `unmute`, `devices`, `device mic|speaker <name>`), a participant banner in the room view, and a room-list indicator for rooms with an active call. The feature is off by default; without `--features voip` the build is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MNTPSETtCAzUuLbn4fzG9a

…nd livekit interference

…plete, and added CI tests for voip feature

VAWVAW

I still haven't had time to look fully into this but here are a few more thoughts.

Does this PR only support MSC4075 or also the stable voip spec?

VAWVAW · 2026-09-09T21:15:31Z

This file doesn't seem to be used.

VAWVAW · 2026-09-09T21:35:55Z

+.It Sy ":call"
+Join the voice call in the currently selected room, or answer a call you have been rung about.
+Your audio is encrypted when the room is encrypted, and sent in the clear when it is not.


I feel like it might make sense to have this option named something like :call join. That would leave the possibility to have a call list on :call like with :verify.

There is a :call join, but I had ":call" call the same join command since ":call" felt more idiomatic to imitate a call.

VAWVAW · 2026-09-09T21:49:05Z

@@ -0,0 +1,1206 @@
+//! # Voice/Video calls (VoIP)
+//!
+//! This module implements voice calls for iamb using [LiveKit] for the media


cargo doc complains about this being a dead link. (And a few others in this file.)

Nightchicken · 2026-09-09T23:05:07Z

This only implements the newer MatrixRTC, it does not have code for the legacy VOIP 1:1. The clients I have checked all have the new API. Including element desktop and element X.

VAWVAW · 2026-09-09T22:43:25Z

+                ""
+            };
+
+            out.push_str(&format!("  {}  {}{mark}\n", device.index, device.name));


nitpick (feel free to ignore): Using write!(..) instead of format!(..) removes one unnecessary allocation.

VAWVAW · 2026-09-09T22:57:24Z

+    /// Show the available audio devices.
+    Devices,
+    /// Choose an audio device, by index or by name.
+    SetDevice(DeviceKind, String),


I think there should be an option for an interactive device select using keybindings::dialog::MultiChoice.

VAWVAW · 2026-09-10T20:16:38Z

I am a bit confused by the different matrix specs for doing voice. There seem to be:

1:1 voip calls (stable)
voip group calls (MSC3401)
MatrixRTC
- this seems to supersede MSC3401 and is made up from multiple MSCs
- core: MSC4143
- calls: MSC4195
- livekit: MSC4196
- call notifications: MSC4075

I couldn't find clear information on this but element seems to support a combination of the latter two. Could you please state and link the documentation you used to implement this PR?

Nightchicken · 2026-09-11T01:23:35Z

I was not familiar with rusts docs with their links, I was using markdown links previously, my next commit will contain working links to the related resources.

until that here is the related docs

MSC4143
MSC4195
MSC4196
MSC3401
MSC4075
MSC3757
MSC4310

Some of these are closed and rejected, while others are obsolete, but the use of them did make some of the code easier to implement. For example the MSC3757 was rejected, but implementing it solved some issues I had with key verification.

It was difficult to track down the exact new spec. At least what was required to make it work properly with element's client. These were the set that made things work.

There are some specs that would be great to add, like delayed events in case of a crash, but that would have to interact with the code base more.

VAWVAW · 2026-09-11T14:11:30Z

Ok, the whole ecosystem of call on matrix is a mess.

I will refer to the combination of MSC4143, MSC4195, and MSC4196 as MatrixRTC.

The spec discussions seem to favor MatrixRTC over MSC3401. (MSC4075 changed its focus to MatrixRTC.)

MatrixRTC is still very actively discussed and the only up-to-date implementations seem to be Element Call and synapse with lk-jwt-service. There is work being done to add support for base features of MatrixRTC to the matrix-sdk (1, 2, 3).

Element Web seems to support a version of MSC3401 but isn't listed as an implementer. It also uses an old version of both MSC4075 and the now-closed MSC4143. The same goes for Element-X (android).

Here are the options I see for going forward:

Go for maximum compatibility
- Implement the combination of old/closed MSCs that Element Web does (this PR)
Implement the currently newest version
- Would only be compatible with Element Call for now (as far as I can see)
- Element Web will probably move over at some point
Wait until the discussion has settled a bit
- All the MatrixRTC MSCs (except MSC4075) are currently in their final review period meaning that there soon is a fixed spec to develop against

@Nightchicken, @ulyssa Do you see anything I missed and what are your opinions on this matter?

I don't like option 1 since we would add new support for (almost) obsolete standards and about-to-be obsolete features.

… and write to voip.json when devices change.

…ependancies are now in the CI tests

Nightchicken · 2026-09-11T14:58:29Z

I avoided adding the voip 1:1 because the standard is pretty much obsolete. I would prefer that we do not work on something that will be discontinued soon. I have tried to implement the new MatrixRTC standard to the best of my ability.

I am okay with going with option 3, but as of right now this PR is more or less option 2.
There are people in the community that have been using this implementation and it has been working for them. So it is at least functional.

As far as Element X (android). It has both the MatrixRTC and voip implementations. Which would still make this compatible.

VAWVAW · 2026-09-11T19:29:42Z

I don't agree that this PR falls under option 2. I think it is clearly option 1 as it implements MSC3401 and the removed MSC4310.

I believe the best way forward is to wait until matrix-org/matrix-rust-sdk#7014 is merged and then implement MatrixRTC based on that and possibly ruma/ruma#2555.

Nightchicken · 2026-09-12T17:13:08Z

Honestly that seems fair enough.

VAWVAW reviewed Sep 4, 2026

View reviewed changes

Comment thread src/main.rs Outdated

Comment thread src/main.rs Outdated

Comment thread src/main.rs Outdated

Nightchicken and others added 3 commits September 9, 2026 09:53

added switch function to handle switching devices to avoid the wrtc a…

b5b5673

…nd livekit interference

refactor and clean up

6a21f36

Nightchicken force-pushed the voip-ulyssa branch from c2047ac to 6a21f36 Compare September 9, 2026 14:55

Fixed issue with verification being moved, added call command autocom…

ee2ec94

…plete, and added CI tests for voip feature

Nightchicken requested a review from VAWVAW September 9, 2026 19:22

missing detail to let linter find voip cfg

09a732f

VAWVAW reviewed Sep 9, 2026

View reviewed changes

Nightchicken added 2 commits September 11, 2026 09:22

Fixed dead links, worked on refactor, cached the voip device settings…

814fef5

… and write to voip.json when devices change.

dependency documentation within the readme and nix file. The proper d…

343ba42

…ependancies are now in the CI tests

ulyssa added this to the v0.0.13 milestone Sep 12, 2026

Uh oh!

Conversation

Nightchicken commented Sep 3, 2026

Uh oh!

VAWVAW left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

VAWVAW left a comment

Choose a reason for hiding this comment

Uh oh!

VAWVAW Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

VAWVAW Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Nightchicken Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

VAWVAW Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Nightchicken commented Sep 9, 2026

Uh oh!

Uh oh!

VAWVAW Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

VAWVAW Sep 9, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

VAWVAW commented Sep 10, 2026

Uh oh!

Nightchicken commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

VAWVAW commented Sep 11, 2026

Uh oh!

Nightchicken commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

VAWVAW commented Sep 11, 2026

Uh oh!

Nightchicken commented Sep 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Nightchicken commented Sep 11, 2026 •

edited

Loading

Nightchicken commented Sep 11, 2026 •

edited

Loading