I’ve just released a new ChatGPT plugin that supports real-time streaming! You can now get responses streamed into your application immediately, as in the official ChatGPT interface, so no more waiting 20 seconds for a response, and no more timeouts with GPT-4!
Features:
Immediate, real-time streaming, so that responses come through without long waits
Easy message history, allows for multi-turn conversations. Memory that works out of the box.
Token counting - see how many tokens each request uses, in case you want to monitor usage.
Stream interruption - stop ChatGPT’s generation if it seems to be going in the wrong direction.
The plugin page is here (ChatGPT Toolkit (with streaming) Plugin | Bubble), or you could watch the setup video below. (this is slightly outdated now, and will be updated with the version 6 release, coming soon)
@dorilama - thanks for the feedback, that’s a great point re: disclosure of the third party server. I’ve added a note to both the description and usage page.
re: math.random being insecure, for transient keys, what do you see as the risk?
Math.random is unsafe and predictable, web crypto use unpredictable hardware random generator, plus using a random % alphabet does not generate an even distribution. So a brute force attack will need less number of tries.
It doesn’t look like there is any auth for your ws endpoint (maybe I am mistaken), so I guess if someone wants to try there are more chanches than what you think to connect to someone else’s websocket and receive the messages.
Thanks for the pointers. I’ll look into swapping in web.crypto instead of math.random. Do you have any other suggestions regarding security? I’ll continue to work on making it as robust and secure as possible.
Regarding the chance that someone can guess a websocket id, I believe its pretty slim. The generation space is about 10^53 combinations. ChatGPT’s take on the size of the space is the following:
To put this in perspective, the age of the universe is estimated to be around 13.8 billion years, or approximately 4.35 x 10^17 seconds. Therefore, even if every atom in the universe were to attempt to create a WebSocket connection simultaneously and continuously, it would still take many, many orders of magnitude longer than the age of the universe to reach 10^54 attempts.
that applies only if you use a secure unpredictable random generator (which Math.random is not) and an even distribution in the alphabet used. That’s the kind of thing a brute force attack relies on.
The server is one that we’ve built and are running, to handle the connection between the Bubble app and the OpenAI API. So it’s a “third party” server in the sense that its not directly Bubble → OpenAI, but we’re the third party. (I’ve updated the notes to make this more clear; thanks for pointing out that it wasn’t clear!)
Hey all. Just added a new action - “Stop Generating”, so you can cancel a request if GPT is going off the rails. Also a new state, “Data Container Currently Generating?”, so you can run workflows conditionally, based on whether text is currently streaming.
Hey again all. Got another update: Support for token counting!
You can now use the “Last Message Token Usage” to track how many tokens your requests (or your users’ requests) used. This is available in version 5.4.1, so please update
Hi all,
Sorry about the outage this morning. Seems we’ve had a massive uptick in traffic (props to the successful launchers!), which has brought the server down. It will be back up in a few minutes.
)
