[Updated Plugin] ChatGPT with Real-Time Streaming (No more timeouts!)

ah. So that could have been an editing mistake. I initially put the custom state on the wrapper group, but had meant to put it on the page. So I tried to cut the video to skip the mistake. But I may have left it in, or left out both parts :grimacing:

Good job catching it

Brilliant plugin, great work! I’m on the latest version 5.12.5, it’s working great! iMac, Chrome. I’m not using chat history though, just one call at a time.

Error handling and testing
It seems like the only way to check if there are errors is by referencing the previous action step (e.g. “ChatGPT send message w/server” and using a condition “Result of that step’s ‘Error’ is empty/Not-empty”.

I’ve created workflows to continue after the Message step for these Error Yes/No conditions. I haven’t had a naturally occurring error happen yet, so I tried to ‘force’ errors by:

  • Writing a random string for the API key
  • Disconnecting the socket before doing the stream GPT action
  • removing the ‘connection info’ dynamic field from the GPT element

All of these caused errors and stopped it from working, but:

  • gave no error response, popup or show in the bubble debugger
  • did not give “error not empty” from the Send Message action
  • and did not get picked up by bubble workflows such as ‘when an unhandled error occurs’ or ‘when element has error running on workflow’.

This means in my app users are ‘hanging there’ having no idea what’s going on, and it looks to me like errors aren’t being reported by the Message action step. Basically I can’t handle errors at all.

  1. Is this possible to improve? Ideally I’d love it if that “error is not empty” condition is 100% reliable and the Message w/server action always displays some kind of error in that output if ANY possible thing goes wrong, and streaming will not happen.

  2. Is there any issue with running the Send Message w/server action and then immediately in the next action step having the condition ‘previous step error is not empty’ condition? I mean how long does the Send Message action wait before moving onto the next step in the workflow, and should it always pickup if there is an error? (it wont just skip past this but then produce an error later that the workflow “missed”?) - i hope that’s clear sorry!

Thank you for your hard work on this!

1 Like

Thanks @nathanielsmithies! Glad you’re getting value out of it.

First off, thanks very much for the feedback! It’s super helpful and appreciated.

Second, once I"ve got the Safari/Mac/iOS issue ironed out, the next set of improvements will all be error-handling related. Things like clearer error messages, (ideally) much less frequent errors, and more robust error-recovery. You’re definitely not the only one feeling some pain there. Currently the only error explicitly returned is if an invalid API key is provided. That will trigger an event. As part of the coming updates, I’ll include as many explicit error-handling events as possible.

Third, there shouldn’t be any issue with your setup as per step 2, at least in theory. If I understand you correctly, that’s the intended setup, and then the “Fallback” action would be setup to fire when an error does come back. But if error messages aren’t being returned correctly (which, as you’ve correctly pointed out), they’re not always, this setup will break down. So the issue is probably not with your setup, but with how the plugin is handling some error that you or your users might be seeing. That being said, some uncaught errors (uncaught at the level of the plugin’s actions) do still show up in the developer console, so if you’re familiar with this, and are seeing issues, doesn’t hurt to pop that open and take a look. But it’s not always helpful.

Thanks again for the input! I’m hoping to have an update out Friday that will tidy up some of these bits, and will cc you here on release.

Would also be grateful to hear about how the ‘n’ parameter solves long system prompt issues.

OK fair enough! I’ll wait for those upgrades.

Just FYI I tried that explicit invalid API Key error event and then put the wrong API key in, but that event didn’t fire for me.

Good luck with the current upgrades!

So is the solution to put it on the page instead of the wrapper? I’m having the same problem :o

@gulbranson.nils - What I meant by the connection info on the page, is to have it actually be set in the data container when you load the page. You still want to set it in the workflow action. Is that what you’re wondering?

I don’t think I understand what you mean. Do you mean to Set the custom state on the data container? What workflow action are you referring to?

I set the custom state on the page itself instead of the wrapper. I thought that’s what you meant in the above response you gave to me. Maybe I am misunderstanding something

@gulbranson.nils - Oh whoops, I was getting things mixed up. Ignore that last comment :blush:

In the tutorial I meant to put the “Selected Conversation” custom state on the page, but when I was first recording, I put it on the “page wrapper” element. I meant to edit that part out, but may have made a mistake.

Ultimately it doesn’t matter where you put the custom state. You just need to be able to reference a particular state later on when you click the conversation.

You can also look at the chatgpt-demo app, which has the same idea implemented: chatgpt-demo | Bubble Editor

I’ve color coded the relevant workflow purple, for you to reference.

@launchable I still observe situations where the model refuses to connect. However, i realise it doesnt say in the front end that the server is ‘down’. Probably useful to highlight it as part of error handling so that we know it is not due to our workflows.

Also, the rate of server being down is still too high to be reliable. Even when using ensure connected, the server still gives me no sometimes. I’m using Chrome btw.

@nocodejordan - Sorry about this. I’m still working daily on making it more stable.

Just for clarification, the servers are almost never down. But there are a couple of issues related to connecting to them.

One is related to how Safari handles websocket communication, which I’m currently trying to fix (not related to your issue).

Another is related to the Connection Info being set. This is the real culprit for most folks, I think, and I’m trying to figure out why it’s inconsistent. This is only an issue in 5.12 though. I’d recommend sticking with 5.11 until the Connection Info issue is resolved. Once I’ve got that issue resolved, I can stop having nightmares :scream: lol

Once it’s fixed, I’ll release a v 5.13, and v 5.12 will be relegated to the dustbin of history as the Blue Version of Doom. Or something equally melodramatic. :man_supervillain:

Also, re: your point about highlighting that with error messages, that’s absolutely on the docket. I’ll be including those error messages as part of the coming updates. Unfortunately I can’t exactly return it yet as an error, because I don’t fully understand what’s causing it.

In the meantime, you can use “If Data Container A’s Connection Info is empty” as a pseudo-error- check. If the connection info is empty, it will fail, so you could use the fallback action in that scenario.

Thanks for the quick response, I appreciate it. Yes, I sometimes see a lag between the connection info v.s the workflow being activated.

I’m recording a tutorial on setting this up as well so I’ll stick with 5.12 for now and hope it works while i’m recording. Haha.

1 Like

Is there an easy way to allow the previous conversation history to be displayed (even from previous sessions), so the user can feel like this is a long term conversation history?

n operator tells openai how many variants you want to get returned (it is the number of response choices within the returned json)…

Since each time you send a prompt to openai you get charged for both outgoing and incoming tokens, using n instead of sending the same prompt several times saves the number of tokens you consume sending the prompt(s)…

Hmm… This is the debugger when I click on the “test” conversation. It’s empty. Is that why? How would I fill that?

@gulbranson.nils - Are you saving your messages to the database on Message Generation Complete?

If you click on (empty), in the debugger, you will see what DB record the workflow is referencing when it tries to set the Message History. You can double check whether it’s getting the right DB record and messages.

Hi all,

I think I have solved the connectivity/reliability issue, at long last :partying_face:

If I’m right, this will mean a huge improvement in the stability of the plugin.

I am running some final tests this evening, and if all goes well, I will release an update (v 5.12.6) later this evening (around 9pm EST). It is possible there will be a brief disruption of service (a minute or less), but I will try to ensure this doesn’t happen.

If you are currently using 5.12.5, please upgrade tomorrow or at your earliest convenience. No changes are required in your application for the update to be beneficial.

There should be no (new) issues in your application if you don’t upgrade from 5.12.5 to 5.12.6, but I won’t be able to verify this 100% until after the servers are updated. If the server update does break v 5.12, the interruption will be just a few minutes or so, and I will roll back to the previous version.

If something goes wrong with the update (which :crossed_fingers:, nothing will), 5.11.1 will continue to be available and will not interrupted.

If you upgrade (or don’t), please let me know if you find anything amiss, or even better, if things work smoothly :grinning_face_with_smiling_eyes:

2 Likes

Fantastic news :star2::pray:

1 Like

Lovely!

1 Like

Hi again all,

Just published 5.12.6. See above for details.

If you upgrade, please let me know how things go.

The only thing you need to change in your app is, if you’re using the Stop Streaming event, you’ll need to add Data Container’s Connection Info to the action. Otherwise everything should work as before.

Stay tuned!

1 Like