Skip to content

MSC4195: MatrixRTC Transport using LiveKit Backend#4195

Open
hughns wants to merge 26 commits into
matrix-org:mainfrom
hughns:hughns/matrixrtc-livekit
Open

MSC4195: MatrixRTC Transport using LiveKit Backend#4195
hughns wants to merge 26 commits into
matrix-org:mainfrom
hughns:hughns/matrixrtc-livekit

Conversation

@hughns
Copy link
Copy Markdown
Member

@hughns hughns commented Sep 16, 2024

Rendered

Implementation

Dependencies:

To-do:

  • Define error responses for POST /get_token

SCT Stuff:

No MSC checklist

No FCP tickyboxes

BLOCKED on MSC4143 - #4195 (comment)

@hughns hughns changed the title MatrixRTC using LiveKit backend MSC4195: MatrixRTC using LiveKit backend Sep 16, 2024
@turt2live turt2live added voip proposal A matrix spec change proposal client-server Client-Server API kind:core MSC which is critical to the protocol's success needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Sep 25, 2024
@hughns hughns force-pushed the hughns/matrixrtc-livekit branch from 054a23c to 6072744 Compare December 17, 2024 07:22
Comment thread proposals/4195-matrixrtc-livekit.md
Comment thread proposals/4195-matrixrtc-livekit.md Outdated

The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID token before validating the token.

The `room_id` could be validated too, and checking that the Matrix user from the OpenID token is a member of the room.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding there's no API that allows room_id validation. Without that implementing access control in a federated environment seems infeasible however.

Given that all members in a session use one SFU via oldest_membership, just validating against matrix_server_name (authenticated using TLS / OpenID) would require allowlisting any server that users might want to join from.

Ideally access control could check either for an allowlisted server or for an existing session with the oldest member being on an allowlisted server, gating an SFU to sessions being started by their "own" users.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation in element-hq/lk-jwt-service#67 does allow limiting creation of new SFU rooms based on servers in a given allowlist, but still allows arbitrary users to join

Copy link
Copy Markdown

@networkException networkException Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the context provided in element-hq/synapse#18967, I believe we can do better by not using ID tokens or a lk-jwt-service altogether.

Instead of a client authenticating against all remote SFUs using an ID token, an extended /transports Client-Server API can return matching credentials, obtained by the user's homeserver over federation. This removes the need for the somewhat bespoke ID token mechanism (which needs to do server name lookup over HTTP and DNS as a fallback. By design that also doesn't play nicely with MAS and OIDC native Matrix), reusing existing server to server authentication. Crucially, this allows for proper per-room tokens.

@ara4n ara4n added the matrix-2.0 Required for Matrix 2.0 label Sep 5, 2025
@github-project-automation github-project-automation Bot moved this to Tracking for review in Spec Core Team Workflow Sep 5, 2025
@fkwp fkwp changed the title MSC4195: MatrixRTC using LiveKit backend MSC4195: MatrixRTC Transport using LiveKit Backend Oct 14, 2025
@fkwp fkwp marked this pull request as ready for review October 14, 2025 13:16
Comment on lines +362 to +375
End-to-end encryption is mapped into the LiveKit frame level encryption mechanism described
[here](https://github.com/livekit/livekit/issues/1035).

Where a shared password is used by the application it is used as the `string` input to the LiveKit
key derivation function (which uses PBKDF2) and all participants use the same derived key for
encryption and decryption.

Where a per-participant key is used it is imported as the byte array input to the LiveKit key
derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is
used as the key index for the key provider.

On receipt of the `m.rtc.encryption_keys` event the application can associate the received key with
the LiveKit participant identity by calculating the pseudonymous LiveKit participant identity as
described above.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like more details on how to implement this.
I've been trying to get it to work in matrix-jukebox, but had no success in getting compatibility with Element Call.
(I'm implementing the old version of the spec without slots and multiSFU for now)

@turt2live turt2live added 00-weekly-pings Tracking for weekly pings in the SCT office. 00 to make it first in the labels list. and removed needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Jan 30, 2026
@turt2live turt2live moved this from Tracking for review to Proposed for FCP readiness in Spec Core Team Workflow Jan 30, 2026
encryption and decryption.

Where a per-participant key is used it is imported as the byte array input to the LiveKit key
derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like multiple LiveKit SDKs (Rust, Flutter) only support using PBKDF2, so using HKDF here is going to limit compatibility with non-web based clients out of the box

See:
livekit/rust-sdks#796
livekit/client-sdk-flutter#974

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also affects the SDKs using Rust-FFI (like python and maybe others)

@turt2live turt2live added blocked Something needs to be done before action can be taken on this PR/issue. and removed 00-weekly-pings Tracking for weekly pings in the SCT office. 00 to make it first in the labels list. labels Feb 10, 2026
@turt2live
Copy link
Copy Markdown
Member

This is blocked on MSC4143's block: #4143 (comment)

Comment thread proposals/4195-matrixrtc-livekit.md
Comment thread proposals/4195-matrixrtc-livekit.md Outdated
Comment thread proposals/4195-matrixrtc-livekit.md Outdated
Comment thread proposals/4195-matrixrtc-livekit.md
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
eleboucher pushed a commit to eleboucher/homelab that referenced this pull request May 15, 2026
…0 → 0.4.4) (#485)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/element-hq/lk-jwt-service](https://github.com/element-hq/lk-jwt-service) | minor | `0.3.0` → `0.4.4` |

---

### Release Notes

<details>
<summary>element-hq/lk-jwt-service (ghcr.io/element-hq/lk-jwt-service)</summary>

### [`v0.4.4`](https://github.com/element-hq/lk-jwt-service/releases/tag/v0.4.4)

[Compare Source](element-hq/lk-jwt-service@v0.4.3...v0.4.4)

##### Towards Delegation of Delayed Events

This release updates the service to accept the JSON signature required for the delegation of MatrixRTC membership leave events, as specified in [MSC4195](matrix-org/matrix-spec-proposals#4195).

Note: This update specifically addresses the JSON signature for JWT request. Comprehensive parsing and handling of the additional delegation data are currently in development (PR [#&#8203;171](element-hq/lk-jwt-service#171)).

##### Docker image

The service is available as a Docker image from the [GitHub Container Registry](https://github.com/element-hq/lk-jwt-service/pkgs/container/lk-jwt-service).

```
docker pull ghcr.io/element-hq/lk-jwt-service:0.4.4
```

##### Precompiled binaries

The service is available as static precompiled binaries for amd64 and arm64 on linux attached to this release below.

##### What's Changed

- Update Docker image version in installation steps by [@&#8203;her001](https://github.com/her001) in [#&#8203;175](element-hq/lk-jwt-service#175)
- Adapted the lastest JSON signature from MSC4195 for SFURequest by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;176](element-hq/lk-jwt-service#176)

##### New Contributors

- [@&#8203;her001](https://github.com/her001) made their first contribution in [#&#8203;175](element-hq/lk-jwt-service#175)

**Full Changelog**: <element-hq/lk-jwt-service@v0.4.3...v0.4.4>

### [`v0.4.3`](https://github.com/element-hq/lk-jwt-service/releases/tag/v0.4.3)

[Compare Source](element-hq/lk-jwt-service@v0.4.2...v0.4.3)

##### Security Update and MSC4195 Identity Alignment

This release introduces important security updates and aligns the LiveKit identity calculation with the latest MSC4195 specification.

##### ⚠️ Breaking Change

The update to the LiveKit Identity hash calculation ([#&#8203;172](element-hq/lk-jwt-service#172)) impacts two experimental modes in Element Call: **Compatibility: state events & multi SFU** and **Matrix 2.0**. Note that by default, these modes are not activated and Element Call will remain working with this release.

For these experimental features to function correctly, the versions of Element Call and the JWT Service must be aligned:

- Compatible: Element Call v0.19.2 or higher AND lk-jwt-service v0.4.3 or higher
- Incompatible:
  - Element Call v0.19.1 or lower AND lk-jwt-service v0.4.3 or higher
  - Element Call v0.19.2 or higher AND lk-jwt-service v0.4.2 or lower

##### Docker image

The service is available as a Docker image from the [GitHub Container Registry](https://github.com/element-hq/lk-jwt-service/pkgs/container/lk-jwt-service).

```
docker pull ghcr.io/element-hq/lk-jwt-service:0.4.3
```

##### Precompiled binaries

The service is available as static precompiled binaries for amd64 and arm64 on linux attached to this release below.

##### What's Changed

- Pin element-hq/setup-ess-cluster-action action to [`aac02ff`](element-hq/lk-jwt-service@aac02ff) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;169](element-hq/lk-jwt-service#169)
- Update module github.com/go-jose/go-jose/v3 to v3.0.5 \[SECURITY] by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;170](element-hq/lk-jwt-service#170)
- Update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;173](element-hq/lk-jwt-service#173)
- Update softprops/action-gh-release action to v3 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;174](element-hq/lk-jwt-service#174)
- Adapt LiveKit Identity hash calculation to latest MSC4195 update by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;172](element-hq/lk-jwt-service#172)

**Full Changelog**: <element-hq/lk-jwt-service@v0.4.2...v0.4.3>

### [`v0.4.2`](https://github.com/element-hq/lk-jwt-service/releases/tag/v0.4.2)

[Compare Source](element-hq/lk-jwt-service@v0.4.1...v0.4.2)

##### Healthcheck Improvement & Security Updates

This is a minor release focused on maintenance and small improvements.
Highlights:

Added support for the HEAD method on the healthcheck endpoint, improving compatibility with monitoring tools.
Included multiple security-related dependency updates to ensure the service remains up to date and secure.
No breaking changes are introduced in this release.

##### Docker image

The service is available as a Docker image from the [GitHub Container Registry](https://github.com/element-hq/lk-jwt-service/pkgs/container/lk-jwt-service).

```
docker pull ghcr.io/element-hq/lk-jwt-service:0.4.2
```

##### Precompiled binaries

The service is available as static precompiled binaries for amd64 and arm64 on linux attached to this release below.

##### What's Changed

- Consolidate non-major dependency updates into larger batches by [@&#8203;robintown](https://github.com/robintown) in [#&#8203;160](element-hq/lk-jwt-service#160)
- Update module github.com/pion/dtls/v3 to v3.0.11 \[SECURITY] by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;154](element-hq/lk-jwt-service#154)
- Update module google.golang.org/grpc to v1.79.3 \[SECURITY] by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;162](element-hq/lk-jwt-service#162)
- Add tests against ESS by [@&#8203;gaelgatelement](https://github.com/gaelgatelement) in [#&#8203;164](element-hq/lk-jwt-service#164)
- Update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;165](element-hq/lk-jwt-service#165)
- Update github.com/matrix-org/gomatrixserverlib digest to [`20c9de3`](element-hq/lk-jwt-service@20c9de3) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;134](element-hq/lk-jwt-service#134)
- Allow HEAD method on healthcheck by [@&#8203;VlaDexa](https://github.com/VlaDexa) in [#&#8203;151](element-hq/lk-jwt-service#151)
- Push docker images also to oci.element.io by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;166](element-hq/lk-jwt-service#166)
- Update all non-major dependencies by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;167](element-hq/lk-jwt-service#167)
- Update GitHub Actions (major) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;168](element-hq/lk-jwt-service#168)

##### New Contributors

- [@&#8203;VlaDexa](https://github.com/VlaDexa) made their first contribution in [#&#8203;151](element-hq/lk-jwt-service#151)

**Full Changelog**: <element-hq/lk-jwt-service@v0.4.1...v0.4.2>

### [`v0.4.1`](https://github.com/element-hq/lk-jwt-service/releases/tag/v0.4.1)

[Compare Source](element-hq/lk-jwt-service@v0.4.0...v0.4.1)

##### 🚀 Cont. Towards Matrix 2.0 – MatrixRTC-Authorisation Service

This release is a minor update from 0.4.0 reflecting the lastest changes in **MSC4195: MatrixRTC Transport** which adds unpadded base64 encoding to hashes for room alias and participant ID.

Furthermore, this release ensures compatibility between clients using the old `/sfu/get` endpoint and the new `/get_token` endpoint (details are [here](https://github.com/element-hq/lk-jwt-service/blob/fa226031ac7a9252996f8d038c45fc086d9d960b/main.go#L230)).

##### Docker image

The service is available as a Docker image from the [GitHub Container Registry](https://github.com/element-hq/lk-jwt-service/pkgs/container/lk-jwt-service).

```
docker pull ghcr.io/element-hq/lk-jwt-service:0.4.1
```

##### Precompiled binaries

The service is available as static precompiled binaries for amd64 and arm64 on linux attached to this release below.

##### What's Changed

- README: Use LIVEKIT\_FULL\_ACCESS\_HOMESERVERS consistently by [@&#8203;csett86](https://github.com/csett86) in [#&#8203;140](element-hq/lk-jwt-service#140)
- README: Describe the static binaries by [@&#8203;csett86](https://github.com/csett86) in [#&#8203;141](element-hq/lk-jwt-service#141)
- refactor build.yaml into reusable chunks  by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;142](element-hq/lk-jwt-service#142)
- fix build.yaml date determination by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;143](element-hq/lk-jwt-service#143)
- Use base64 encoded hashes for room alias and participant ID (minimize identifying metadata sent to SFU) by [@&#8203;robintown](https://github.com/robintown) in [#&#8203;144](element-hq/lk-jwt-service#144)
- Add 'synchronize' and 'opened' types to PR trigger by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;146](element-hq/lk-jwt-service#146)
- Update room alias handling in token generation for LegacySfuRequest by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;147](element-hq/lk-jwt-service#147)

##### New Contributors

- [@&#8203;robintown](https://github.com/robintown) made their first contribution in [#&#8203;144](element-hq/lk-jwt-service#144)

**Full Changelog**: <element-hq/lk-jwt-service@v0.4.0...v0.4.1>

### [`v0.4.0`](https://github.com/element-hq/lk-jwt-service/releases/tag/v0.4.0)

[Compare Source](element-hq/lk-jwt-service@v0.3.0...v0.4.0)

##### 🚀 Towards Matrix 2.0 – MatrixRTC-Authorisation Service

This release brings major foundational work towards **MSC4195: MatrixRTC Transport**, introduces the new `/get_token` endpoint, improves validation and error-handling, and enhances deployment flexibility with improved bind controls.

##### ✨ What's New

##### **Implementation of MSC4195: MatrixRTC Transport using LiveKit Backend**

This release introduces the new MSC4195-compliant request flow and `/get_token` endpoint, while keeping `/get/sfu` available for a transition period.

Key changes:

- Added unified error handling via `writeMatrixError`.
- Introduced new Matrix 2.0 `SFURequest` type, alongside `LegacySFURequest`.
- Refactored internal functions for better testability and structure.
- Added extensive new tests across request mapping, processing, and JSON handling.
- Updated logging, error codes, and user-identity checks.

##### **🔌 Allow Binding of Arbitrary Addresses**

This release improves deployment flexibility by allowing the service to bind to any or a specific address.

##### Features & Fixes

- **feat:** Introduced support for binding to arbitrary addresses.\
  *Contributed by [@&#8203;matdup](https://github.com/matdup)*
- **fix:** Documentation updated for environment variable changes.
- **feat:** Reintroduced `LIVEKIT_JWT_PORT` for backward compatibility.
  - Now mutually exclusive with `LIVEKIT_JWT_BIND`.
  - Emits a warning when used.
- **feat:** Enforced mutual exclusivity between `LIVEKIT_JWT_BIND` and `LIVEKIT_JWT_PORT`.

##### ⚙️ Configuration

##### Environment Variables

| Variable           | Description                                                                           |
| ------------------ | ------------------------------------------------------------------------------------- |
| `LIVEKIT_JWT_BIND` | Binds the service to a specific host:port (new preferred method). Defaults to `:8080` |
| `LIVEKIT_JWT_PORT` | Deprecated but still supported. Mutually exclusive with `LIVEKIT_JWT_BIND`.           |

##### Endpoint Changes

| Endpoint     | Status                    | Notes                                                         |
| ------------ | ------------------------- | ------------------------------------------------------------- |
| `/get_token` | **New (Primary)**         | Conforms to MSC4195.                                          |
| `/get/sfu`   | **Legacy (Transitional)** | Auto-maps request types; will be removed in a future release. |

##### 🤝 Client Compatibility

- Legacy MatrixRTC clients continue functioning via `/sfu/get`.
- MSC4195-aware clients should migrate to **`/get_token`**.
- Both request formats remain supported during the transition.
- Stricter validation may require client-side updates:
  - Correct request field shapes
  - Accurate user identity claims (`claimed_user_id` vs OpenID `sub`)

##### Docker image

The service is available as a Docker image from the [GitHub Container Registry](https://github.com/element-hq/lk-jwt-service/pkgs/container/lk-jwt-service).

```
docker pull ghcr.io/element-hq/lk-jwt-service:0.4.0
```

##### Precompiled binaries

The service is available as static precompiled binaries for amd64 and arm64 on linux attached to this release below.

##### What's Changed

- ci: add static go releases for linux arm64 and amd64 by [@&#8203;csett86](https://github.com/csett86) in [#&#8203;111](element-hq/lk-jwt-service#111)
- fix(deps): update github.com/matrix-org/gomatrixserverlib digest to [`a234d6d`](element-hq/lk-jwt-service@a234d6d) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;113](element-hq/lk-jwt-service#113)
- chore(deps): update dependency go to v1.24.5 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;114](element-hq/lk-jwt-service#114)
- refactor README.md by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;115](element-hq/lk-jwt-service#115)
- feat: allow binding of arbitrary addresses by [@&#8203;MatthieuCoder](https://github.com/MatthieuCoder) in [#&#8203;118](element-hq/lk-jwt-service#118)
- fix(deps): update github.com/matrix-org/gomatrixserverlib digest to [`6697d93`](element-hq/lk-jwt-service@6697d93) by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;120](element-hq/lk-jwt-service#120)
- chore(deps): update dependency go to v1.25.3 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;121](element-hq/lk-jwt-service#121)
- Refactor config parsing by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;125](element-hq/lk-jwt-service#125)
- fix(deps): update module github.com/golang-jwt/jwt/v5 to v5.3.0 by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;127](element-hq/lk-jwt-service#127)
- chore(deps): update github actions by [@&#8203;renovate](https://github.com/renovate)\[bot] in [#&#8203;126](element-hq/lk-jwt-service#126)
- Implementation of MSC4195: MatrixRTC Transport using LiveKit Backend by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;128](element-hq/lk-jwt-service#128)
- fix: add `expires_in` field to `OpenIDTokenType` by [@&#8203;fkwp](https://github.com/fkwp) in [#&#8203;130](element-hq/lk-jwt-service#130)

##### New Contributors

- [@&#8203;csett86](https://github.com/csett86) made their first contribution in [#&#8203;111](element-hq/lk-jwt-service#111)
- [@&#8203;MatthieuCoder](https://github.com/MatthieuCoder) made their first contribution in [#&#8203;118](element-hq/lk-jwt-service#118)

**Full Changelog**: <element-hq/lk-jwt-service@v0.3.0...v0.4.0>

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL21pbm9yIl19-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/485
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked Something needs to be done before action can be taken on this PR/issue. client-server Client-Server API kind:core MSC which is critical to the protocol's success matrix-2.0 Required for Matrix 2.0 proposal A matrix spec change proposal voip

Projects

Status: Proposed for FCP readiness

Development

Successfully merging this pull request may close these issues.

10 participants