Now, OpenAI Realtime pushes the boundaries by integrating language generation with seamless voice interaction, giving app builders the tools they need to craft more natural and conversational AI experiences.
From creating virtual assistants to powering real-time customer support, this new model opens a vast array of possibilities for voice-driven applications.
This plugin uses an external service to provide streaming capabilities.
Is there a way to push in a user key or a BYOK functionality to this? We’d like to use with our users but wouldn’t want to run from our own key. If we are using a BYOK function then having that exposed prior to firing up the chat isn’t the biggest of issues (as the user will have just entered it)
Amazing work on the plugin! Really easy and inutitive to use.
Just a few comments/questions:
Do I need to do anything to make different voices work? For some reason I can only get it to speak back to me if it’s on the default alloy voice. If I try to change it to anything else, I don’t hear any responses.
What does this field do? It gives me an error if it’s empty and it’s the only one without a documentation tooltip.
Also I can’t seem to get it to work on mobile. When I press detect mic, it opens up the microphone permission pop up in the browser and if you click Allow it doesn’t go away. If you click on it a few times and then off the pop up, you’ll eventually give it permissions.
However, when you click on Start Voice, it doesn’t seem to be hearing me as I can’t hear any responses.
This field must be set if you need to resume an existing conversation that has been interrupted by the Stop action.
You must pass one of the existing conversationId of ALL CONVERSATIONS state.
However there was a bug that I have fixed, this field should not be mandatory - please update.
Is there a reason all six are not supported? This is what they mention within the OpenAI docs and it shows it within the documentation you have as well.
I was really hoping that fable would be available.
Have you tested the plugin on mobile across different browsers? I would be interested to know which ones were looked into, if at all, just so I know what to expect.
Haven’t found one not working (yet!). Tested Chrome, Safari, Firefox on both desktop and mobile.
If you have an issue let me know via DM, I will investigate.
Hi again! I set up the plugin on a test page and replicated the workflows and so on of the demo. Have confirmed the API key and credentials are working by sending a text message and getting an instantaneous voice response.
But, even when confirming that the plugin is listening, I cannot get any response. Here’s a screenshot of what I’m seeing after sending a text message (what’s up), showing the microphone, and the listening state. Demonstrating that everything but the voice input and response seems to be working. Any clue as to what I might look at to get it running?
Thanks!