Hi, I’m using OpenAI’s GPT-3 completion api to create my own type of ChatGPT. Here’s the workflow:
- User submits question > Create new database entry > Call GPT-3 API > Update database entry with results > Database entry is shown in a repeating group on the page
Problem is, sometimes the time it takes to receive the full response is quite long and the user just sees a loading icon while we wait. I want to be able to use the stream function to type the response as it comes just like ChatGPT does.
Has anyone successfully done this? I’ve found resources on it but it’s a bit beyond me to work out how to do it in Bubble!
Thanks!
2 Likes
Has anyone been able to setup the stream?
@rico.trevisan I am sure the plugins shown above work. However, none are using the streaming functionality. The streaming functionality provides a realtime response word by word. Currently, the existing api’s have a delayed response and provide the completion all at once. There is a significant difference in user experience between streamed and non streamed calls.
2 Likes
someone have a solution for this? thx
APIs don’t work like that. In a nutshell - you send an object and receive an object back. There is no “streaming” in APIs.
1 Like
Is there an alternative then?
WebSockets is an alternative to RESTful APIs. I have a feeling OpenAI’s website is using it, although this is just speculation as I haven’t looked.
I always thought that you could do it over API because the “Stream” is actually just a bunch of individual data responses grouped together, so you just grab them and display them as they come in. I don’t know if that’s correct, but that’s what I understood from their documentation: Using server-sent events - Web APIs | MDN
1 Like
Meaningless.
i have a Custom Bubble ChatGPT at work.
1st) you need to enable stream = true
and the return API is almost not recognizable by bubble.
Even if you manage to do it, it almost bring no added value.
A MUCH Simple solution.
while waiting for OpenAI response :
Typing .
Typing …
Typing …
( animate 3 sentense above )
Solved.
2 Likes
Can you please point me in the right direction? I can figure it out from there but need a starting point.
I have my own OpenAI based server set up on AWS and it streams responses back. I need some directions on how this could be done on bubble.
Thanks!
my understanding is that bubble’s API Connector does not support streams, same as bubble’s server side actions.
If you are streaming responses from your own server you can use a client-side action plugin with code to interface with that.
1 Like
this intuitively make sense, but there is no returned value on the client side action - so how im not sure how to get the streamed response back
Yes, that’s one annoying thing with plugins in bubble. The usual workaround is create a state in the plugin element, publish the result in the state, trigger an event to signal that there is a returned value available.
2 Likes
thanks very much! i did come across another one of your posts where you elaborated on this. appreciated!
1 Like
Hi everybody,
I test to call ChatGPT API call with a stream type like this (choose data type: text, not JSON)
API return to me below. Is there any way/ideas to stream content to the user?
Thanks
data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"role":"assistant"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"C"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"ute"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" and"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" furry"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":" f"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"eline"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{"content":"!"},"index":0,"finish_reason":null}]} data: {"id":"chatcmpl-74KJe5fBbwnwOD5jrZffjGWbxc73m","object":"chat.completion.chunk","created":1681265762,"model":"gpt-3.5-turbo-0301","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]} data: [DONE]
