The use case here is that when I refresh a user’s message, I remove the last assistant message from the message history, then run the message history so it just generates a new assistant message from the previous user/system message
So this plugin seems to be “user inputs what they want to input and get a response from ChatGPT”. However, in my application, I already have the prompting done, and the user will just Input a short sentence or number for example and then a response will be given. I want THAT response to be real-time, not for the user to type in the entire prompt by themselves. Is that possible?
Make message 1 a system message with your prompt saying ‘using the user’s message below, do XXXXX’ etc, then make message 2 a user message which is user’s input
Set System Message - Action to set a static system message, that will always be part of conversation. Helpful for guiding behavior of GPT, especially GPT-4. You could (and still can) do this by setting Message 1, 2, or 3 to type “system”; this action just simplifies it, especially for long conversations. Available in Send Message w/ Server, Send Message Direct to OpenAI, and Send Message non-streaming (fallback).
Stop Sequences - specify up to 4 character sequences that will cause generation to terminate.
Less obnoxious error if “Ensure Connected” fails. Should now just be a message in developer console: “Unable to connect to ChatGPT websocket”.
Update re: other 2 planned improvements:
Update “Send Direct to OpenAI” with just message history - not yet implemented.
Issue setting/clearing message history - I have found the most luck wrapping these events in a Custom Event. The issues seems to be Bubble plugin actions running in parallel when possible, causing workflow events to sometimes occur in a different order than that set in the Workflow Editor. Custom Events seem to force running in order, so can be useful to have better control of the timing of workflow events. Still looking for better solution.
Feature development is now on hold until 6.0 is released (major reliability/availability upgrade), which should be in the next few days
Any suggestions on how to do a more perfect user experience around auto-scrolling while the text is being generated and displayed in real time? Seems like it sometimes gets cut off. Tried a visibility detector plugin, and it sort of works half the time. Seems the demo also will sometimes get cut off, even with the workflow actions that are in place.
Besides for this issue, what an amazing plugin. Thank you so much for making and maintaining this.
Is it possible to buy a license to the server and run it myself (with you auto updating of course)? I might feel more secure doing that. I don’t like the idea of your server being in the middle. Both for security complications (I’m in Canada and might be doing a medical GPT thing), but also then my business setup is not as anti-fragile.
@mayur - After the next major upgrade, I think I’ll be developing a self-hostable option, as it’s becoming a pretty common request.
Out of curiosity, is the concern the server specifically, or also the code? In other words, do you think you’d want/need to have the code, or just want/need it running on a server you own/control?
Shouldn’t take too long to release it, maybe a couple of weeks from now? I’ll keep you in the loop!
I don’t really need the code - though having it would increase my security. I assume whatever is running on your server is closed source and proprietary. Having it on my own server gives me some comfort knowing its stable, and always accessible and my full responsibility. Doing updates myself would be ideal as well so that I can control version stability and rollbacks in case something breaks.
@launchable related question: do you know whether OpenAI keeps the data we send to it or not? The answer seems unclear. I’m thinking I might need to train and run my own LLM instead (no idea if that’s feasible or not)