[Updated Plugin] ChatGPT with Real-Time Streaming (No more timeouts!)

Also, one small bug:

I can send a message with the server that doesn’t contain the user content, only the message history (which is good):

But using the without server action, it requires message 1 to not be empty:

The use case here is that when I refresh a user’s message, I remove the last assistant message from the message history, then run the message history so it just generates a new assistant message from the previous user/system message

@georgecollier - Just pushed a patch (5.10.3) to fix that issue. You should be able to send with just message history now.

@georgecollier - the patch isn’t quite working :slightly_frowning_face: Will have to do re-patch once I’ve got this connection issue sorted

Hi all,

There may be some brief disruptions to the plugin today, as the servers are updated. Apologies for any inconvenience this may cause!

Keep on eye on this thread for any updates.

Thanks for your patience!

Is there a way to set a stop sequence with Launchable AI, as is possible with the default Open AI API? I wasn’t able to find a setting for it.

Would help to get the AI chat to come to an end once its followed through with the system instructions, rather than continue using tokens.

@mdajinani - ah, that’s a good point. I haven’t exposed that option. Will add that to the list of improvements. Thanks for the suggestion!

1 Like

Hi all,

Quick note - update coming today with a few new features, as requested above:

  • Set System Message action - to simplify setting a system message that will persist, even as message history grows
  • Set Stop Sequence - to terminate response on a given sequence - (@mdajinani)
  • Update “Send Direct to OpenAI” to allow sending just a message history (@georgecollier)
  • Will try to fix the issues with message history setting/clearing not always working (@buildingsomething)
1 Like

Thanks for the awesome updates!

@launchable How is 6.0 going? Do you have any approximate timeframe?

@hugors00 - 6.0 is basically done. I expect to submit it for review tomorrow :crossed_fingers:

2 Likes

Awesome, thank you!

So this plugin seems to be “user inputs what they want to input and get a response from ChatGPT”. However, in my application, I already have the prompting done, and the user will just Input a short sentence or number for example and then a response will be given. I want THAT response to be real-time, not for the user to type in the entire prompt by themselves. Is that possible?

Make message 1 a system message with your prompt saying ‘using the user’s message below, do XXXXX’ etc, then make message 2 a user message which is user’s input

Hi all,

Update just released - 5.11.1 :parasol_on_ground:

Improvements:

  1. :robot: Set System Message - Action to set a static system message, that will always be part of conversation. Helpful for guiding behavior of GPT, especially GPT-4. You could (and still can) do this by setting Message 1, 2, or 3 to type “system”; this action just simplifies it, especially for long conversations. Available in Send Message w/ Server, Send Message Direct to OpenAI, and Send Message non-streaming (fallback).
  2. :stop_sign: Stop Sequences - specify up to 4 character sequences that will cause generation to terminate.
  3. :shushing_face: Less obnoxious error if “Ensure Connected” fails. Should now just be a message in developer console: “Unable to connect to ChatGPT websocket”.
    Update re: other 2 planned improvements:
  • :x: Update “Send Direct to OpenAI” with just message history - not yet implemented.
  • :orangutan: Issue setting/clearing message history - I have found the most luck wrapping these events in a Custom Event. The issues seems to be Bubble plugin actions running in parallel when possible, causing workflow events to sometimes occur in a different order than that set in the Workflow Editor. Custom Events seem to force running in order, so can be useful to have better control of the timing of workflow events. Still looking for better solution.

Feature development is now on hold until 6.0 is released (major reliability/availability upgrade), which should be in the next few days :european_castle:

1 Like

Any suggestions on how to do a more perfect user experience around auto-scrolling while the text is being generated and displayed in real time? Seems like it sometimes gets cut off. Tried a visibility detector plugin, and it sort of works half the time. Seems the demo also will sometimes get cut off, even with the workflow actions that are in place.

Besides for this issue, what an amazing plugin. Thank you so much for making and maintaining this.

1 Like

Is it possible to buy a license to the server and run it myself (with you auto updating of course)? I might feel more secure doing that. I don’t like the idea of your server being in the middle. Both for security complications (I’m in Canada and might be doing a medical GPT thing), but also then my business setup is not as anti-fragile.

Long shot, but worth asking

2 Likes

@mayur - After the next major upgrade, I think I’ll be developing a self-hostable option, as it’s becoming a pretty common request.

Out of curiosity, is the concern the server specifically, or also the code? In other words, do you think you’d want/need to have the code, or just want/need it running on a server you own/control?

Shouldn’t take too long to release it, maybe a couple of weeks from now? I’ll keep you in the loop!

2 Likes

@davewliu - I’ll give this some thought, I’ve struggled with the same issue. Don’t currently have a good auto-scrolling option.

1 Like

I don’t really need the code - though having it would increase my security. I assume whatever is running on your server is closed source and proprietary. Having it on my own server gives me some comfort knowing its stable, and always accessible and my full responsibility. Doing updates myself would be ideal as well so that I can control version stability and rollbacks in case something breaks.

@launchable related question: do you know whether OpenAI keeps the data we send to it or not? The answer seems unclear. I’m thinking I might need to train and run my own LLM instead (no idea if that’s feasible or not)