[PLUGIN] - VideoJS Audio Recorder (plus voice cloning)

[pork1977gm]

VideoJS Audio Recorder

(uses the latest videoJS library plus integration with Cartesia’s voice cloning endpoint)

---

Hi all,

I’ve released a videoJS Audio Recorder plugin. This is similar to the recorder that appears inside my videoJS Advanced plugin, but with better code management, more up to date scripts (for improved cross browser support) and a recording management state included.

It allows for immediate audio recording and playback by providing access to recordings through an object state, eliminating the need to upload files to S3 first in order to load or play again.

Plugin: https://bubble.io/plugin/videojs-audio-recorder-1741952478933x862514496477069300
Demo: https://paul-testing.bubbleapps.io/version-test/videojs_audio_recorder
Editor: https://bubble.io/page?id=paul-testing&tab=Design&name=videojs_audio_recorder&type=page
Documentation: https://paul-testing.bubbleapps.io/version-test/videojs_audio_recorder_instructions

---

Recordings are stored within a state that looks like this:

Each individual recording within this state holds various data that updates depending on what action gets run. This can be fed into a repeating group for example where individual recordings can then be played, paused, uploaded to S3 (or Wasabi) etc and any values will be updated.

The plugin does come with a visual element which you can hide if you don’t need it. If you do use it, then it has waveSurfer built-in which shows the sound wave as you speak (colors configurable).

There’s a bunch of settings that can be set like setting the max recording length, auto gain control, noise suppression, buffer sizes, capturing microphone noise levels, display settings etc.

---

Cartesia

Since at the time of creation, I had a need for cloning voices, further customizations/support was added to integrate Cartesia’s voice cloning API. You can now feed a recording into an action called “Clone voice” and have it clone your own voice. Cartesia returns a voice ID so if you wanted to utilise their services further and apply this voice to one of their TTS services, then you can do.

The action also allows you send a transcript along with the recording. If you do, it’ll generate an audio file where the transcript is spoken in the same voice as the recording.

Demo page

As far as using the demo page goes, it’s good fun when you clone your own voice and provide a transcript. You can test cloning your own voice and it will return an audio file for you to playback. If you try it, then try and speak as you would normally. It generally takes around 2-3 seconds to generate. You’ll need to use your own API key though I’m afraid, I’ve provided an input field for that.

Hope it’s a useful plugin.

Paul

[ianruta11]

Hi Paul! Thanks for building this. I am quite interested! A few questions:

• Would this work for Bubble’s new mobile app builder?
• Is it easy to use the audio recordings for external api calls (eg. to OpenAIs speech to text model)?
• How many workflow units do each recording action count for?

[pork1977gm]

Hi there,

No problems at all. As far as I’m aware, I think plugins are currently disabled through the new mobile builder at the moment. I’m not entirely sure what the plan is there, but maybe someone will reply to this with some more info. I would imagine exposing plugins could play a part in apps not being approved, especially with Apple.

As for sending the audio file to openAI, it should be straight forward to get at the recordings. You can access the recording’s blobUrl field which will give you a locally stored URL for the recording. I’m pretty sure you’ll be able to use that with the majority of STT API calls, but if not and the service requires a URL to an actual location, then you can use the ‘upload recording’ action which will save the file to S3, and then you run your openAI API call inside the ‘file upload complete’ event, accessing the uploadUrl property of the recording.

If neither of those work then shout at me and I’ll change some bits to make it work

I don’t think any workflow units are consumed at all, because it all runs inside the plugin’s code. The actions might, although not 100% sure (you might want to double check through the logs to see that)

Hope that helps
Paul

[ianruta11]

Thanks Paul! Really appreciate the response! I will probably give this a try in the next couple of weeks or so

[pork1977gm]

I’ve included live speech transcription into the plugin. No API calls needed, no audio files are required to be saved to Bubble first. Speech is just transcribed and put straight into a state called transcript that is a list of text’s.

(Note this may not work across all browsers) - You can test it on the demo page by starting a recording session. Any requirements, please let me know.

Paul