[PLUGIN] - LiveKit (real-time video chat)

Hi all,

This post is to introduce a brand new audio/video/data real-time chat plugin. I’ve been sitting on this for quite some time now, and I’m glad it’s finally complete! It’s gone through nearly 4 months of development which has involved not only testing the code, but making sure it works and plays nicely across all devices and everything runs smoothly. There’s a ton of features inside, like having the ability to float video elements, drag them around, swap on click (all that you get when you have a WhatsApp call with someone) and then it also includes background image segmentation where you can set a background image or blur it (like you get with Google Meet, but this is a desktop only feature). You can send audio across a WebSocket connection if you want, maybe integrate it with some third party system etc and record meetings uploading them to S3, GCP and Azure (all explained in the docs).

LiveKit (Real-Time Video Chat) Plugin | Bubble

I’ve built three different layouts to choose from (gallery, grid, speaker) which give you different views. The actual UI is quite configurable, but if I’ve missed anything then please let me know. Excuse the mug shots!

Gallery layout (responsive)

Grid layout (set sizes)

Speaker layout (this changes depending on who is speaking)

Screen sharing (participants get shifted to a container which is configurable)

As a part of the building process for this plugin, I’ve collated a list of what everyone has asked for in other plugins of this nature and made sure they are all included. So you can, if you want assign sound effects to specific events or turn on the “knocking” feature where specified room owners have to accept/reject participant requests before they are allowed to join the room.


I was going to release a version of this type of plugin some months back when MUX was developing their MUX Meet feature, but sadly they decided to sunset this and in the process of doing so recommended LiveKit. These guys really have built an awesome infrastructure and they give you a fair bit of data access on the free plan too (50 GB every month). The plugin automatically optimizes the quality of subscribed video tracks (both bandwidth and CPU) with a setting called adaptive stream which means it doesn’t consume a lot of bandwidth or data that gets sent. When video elements become visible, an appropriate resolution will be set. When none of the video elements are visible, it’ll temporarily pause the data flow until they become visible again.

You can even hook it up to an AI agent and have AI agents join your room. The agents use Deepgram, ChatGPT and ElevenLabs. Additional configuration for voice activation (VAD) and Speech to Text (SST) will be available BUT… none of this is in the current version of the plugin, not yet anyway. I will be adding this functionality once LiveKit have completed their JavaScript integration. For now, this only runs on Python backends.

If that’s caught your interest, you can demo it below.
LiveKit Agents Playground: KITT

Let’s move on … … …


Demo Pages

There are three demo pages and another that contains all the documentation. Every documentation section inside all the options/actions has been written up and proof read to make sure they state exactly how things work.

Standard demo page
https://paul-testing.bubbleapps.io/version-test/livekit_demo

Demo page showing the knocking feature
https://paul-testing.bubbleapps.io/version-test/livekit_demo_knocking

Demo page showing the visual effects (only supported on desktop)
https://paul-testing-4.bubbleapps.io/version-test/livekit-effects

Documentation
https://paul-testing.bubbleapps.io/version-test/livekit_instructions


Mobile views can be configured to look like the image below. The floating video element is draggable by click and touch and if you click it, the videos can swap.

Data can be sent between participants.

Locations of where people are (if they agree to sharing) can be plotted on to a map.

Background imaging.



As far as technology goes, LiveKit use the latest and greatest and have a Slack channel to join if you ever need further information - Slack channel

There’s not much more for me to say other than I hope it works well for everyone. If there’s any bugs then it’s genuinely because I’ve missed something, but I’m here for support as always and I’m happy to include any additional options going forward. It goes without saying, if anyone does spot any bugs then I’ll be jumping onto it straight away.

Paul

5 Likes

This is pretty amazing! I’m still fairly new, but very curious about how you mean when you say AI agents can join the call? Do you mean you can have them follow along in a video meeting? Is that possible to do easily with the plugin?

Thanks @M1J1

Exactly that, so think of an AI agent as an additional participant (person in the call) that’s capable of answering any questions etc. It’s not in the plugin right now but it’s being added once complete. Did you check the demo out which LiveKit themselves produced for it?

Paul