[PLUGIN] - Speech Animation (mic & voice)

Hi all,

Documentation for a new plugin below. It allows you to activate the microphone in your desktop or mobile browser (tested on multiple devices) and use the sound input to animate CSS transitions to produce the type of effect that Google Meet uses when a person is speaking.

Animation

Whilst the above GIF doesn’t do it much justice… the animations and the image can be controlled through the plugin options, like the microphone sensitivity, the ring colors, max expansion widths and other settings.

It uses 2 types of audio streams. The first stream is just whatever the microphone picks up, this is used for the inner circle animation and the sensitivity option controls this. The second stream uses a voice activity detection library and this controls the outer ring. It recognises when the user is speaking and sets a boolean state accordingly.

Because there are essentially two streams opened, it’s possible to detect when a user is speaking but perhaps their microphone is muted and you can use that to conditionally display messages, if you’re using some sort of audio solution.

Demo
https://paul-testing-4.bubbleapps.io/speech_animation

Editor
https://bubble.io/page?type=page&name=speech_animation&id=paul-testing-4&tab=tabs-1

Plugin
https://bubble.io/plugin/speech-animation-mic–voice-1698649389079x200244457278799870

For the more advanced of you, if you’re already using an audio stream and it’s accessible through the browser console, perhaps you’re sending voice data to some other destination, then there’s an alternative stream option that can be used to replace the microphone stream with yours.


Have fun with it! hopefully it will provide some nice functionality to a bunch of apps out there!
As always, if you want anything else adding to this, just shout and I’ll do what I can to accomondate :slight_smile:

Documentation

Fields

Start monitoring
When set to Yes, the audio streams for microphone and voice will be enabled automatically. Setting this value to No, requires you to run the 'Start voice and microphone' action.

Voice circle color
This is the color of the outer ring which appears when voice is detected. Please note: Voice is not the same as microphone sounds being picked up.

Voice circle width
This number measured in pixels (px) is the width of the outer ring that appears when the person speaks. Please note, this cannot be changed dynamically once set.

Mic circle color
This is the color for the inner ring which appears as any sound is picked up through the microphone.

Mic circle max width
This number measured in pixels (px) is the maximum number allowed for which the circle can grow to when talking.

Mic circle sensitivity
This number determines the sensitivity of the microphone (in programmatic way). The number should range in the hundreds, for example a value of 100 would make the sensitivity relatively low, whilst 1000 will make it super sensitive. You can have any number you like here since it's used a multiplier in various calculations.

Mic circle transition
This number measured in seconds, is the linear transition speed between the microphone animation decreasing/increasing in size. It should ideally be set between 0.1 and 1.0 where a lower number will make the animation appear snappier.

Mic circle blur
This number measured in pixels (px) is the blur radius of the microphone animation. The higher the number the more blurred it will be.

Alternative stream
This is an advanced option which allows you to use an alternative audio stream for the microphone. For it to work, that stream must be readily available through the browser's console in the format of window.myStream. The value 'myStream' is just an example here, provide the full global variable name. If this is incorrectly entered, the 'error' state will be populated and the 'voice or microphone initialization has errored' event triggered.


Exposed states

Microphone muted (yes/no)
Current state of the microphone, is Yes when the microphone is disabled or has been muted.

Voice detected (yes/no)
Current state of voice activity, is Yes when voice is detected or the user is speaking.

Microphone level (number)
A number between 0 - 100 which represents the microphone volume level.

Error (text)
Contains any errors.

Microphone animation stopped (yes/no)
Current state of the microphone animation, is Yes when the animation has been stopped.

Voice animation stopped (yes/no)
Current state of the voice animation, is Yes when the animation has been stopped.


Events

Voice and microphone has initialized
Triggered when the voice and microphone initializations are complete.

Voice or microphone initialization has errored
Triggered when either the voice or microphone initializations have errored.


Actions

Mute/Unmute microphone
Mutes or unmutes the microphone.

Start voice and microphone
Start the voice and microphone detection.

Stop voice and microphone
Stops the voice and microphone detection.

Start/Stop microphone animation
Allows you to start/stop the microphone animation.

Start/Stop voice animation
Allows you to start/stop the voice animation.

Paul

4 Likes

Just checked it out. Very cool.