[Updated Plugin] ChatGPT with Real-Time Streaming (No more timeouts!)

@robogpt - I’ll follow up in DMs.

Deployed and it’s working way better thanks so much!

I had a few unhandled errors on the first time using a data container, then it worked.

Question 1) - should I go through my app and re-do all the workflows on my page that has 7 GPT containers, and reduce that to 1 GPT container that, when it triggers the ‘GPT Completed Message’ action, then triggers 7 different custom events depending on the type.

The theory behind this being, if it’s true that each Data Container takes an error or some time to “warm up”, then you would only have it “warm up” once instead of 7 times.

It’s a lot of work, but I will do it if this is indeed the case about “warming up” per container, as if so it would reduce errors for my users. Please advise!

Question/Observation 2)
On a different page I have a ‘Loop’ workflow which uses the same Data Container every time and loops through creating a GPT output for each item on the list until it finishes. It is working, but after a lot of trial and error adding workflow delays in certain places and multiple custom events triggering custom events to force order of operations.

I was getting the ‘start of message output cut off’ issue on loop number 2+, but not on item 1, and I think it is to do with the container’s ‘current message’ not being cleared quick enough before the next “Send message with server” action is run.
I guess this would happen for people with a chat interface and a user sends another message very quickly after the first.

After trying including ‘Stop Streaming’ as part of my ‘end of loop/reset’ actions, I’m not using because it didn’t work at all.

Should I now use Stop Streaming action for my loop purpose? Should this now be the strongest/stablest method to set the container?

Thanks!!!

@nathanielsmithies - I’ll follow up in DMs, as these questions are pretty specific to your app.

1 Like

Thank you for the update!

We have deployed to live as well, will let you know how it goes!

It seems to be an issue with the system text. When the conversation has been going for a while, and the user asks to e.g. summarise the previous message, it summarises the system text instead and respond to that.

@jaos.pcl - ahh, thanks for letting me know. I know why that would be happening. I’ll fix that tomorrow.

1 Like

@launchable Just to let you know I am running your plugin on 2 apps. And both so far seem to be far more reliable. Thank you for preserving with this. It really is a fantastic plugin! I can only rate the app once but I still stand by the 5 stars. Your regular feedback in this forum has been first class.

4 Likes

@sacoetzee - thanks so much for the kind words! :blush: That’s very nice feedback to wake up to.

And I’m glad to hear things are continuing to improve in your apps.

There’s still a bit of a gap to close in terms of reliability, but thanks to everyone’s support and help, we’ve been able to fix most all the issues, and I’m confident we’ll get to 100% solid soon :muscle:

1 Like

Hi all,

I’ve just released another patch v5.15.4. This is meant to help developers who were seeing issues with connections dropping. There is no new functionality in this release, just more stable connections (hopefully :sweat_smile: )

Please let me know if you find any issues, or if you see any improvements in your apps as a result of the update.

Thanks!

(cc @nathanielsmithies @robogpt @project)

2 Likes

Thanks for update, it is on Live test till evening.

1 Like

Nice work! I’ve pushed live, will let you know how it goes after some usage.

1 Like

It seems to be working well! I am super happy thank you so much and well done!

1 Like

new buyer here :slight_smile:

Could you please add the nb of tokens used in the response please? (useful for apps that to calculate nb of words used by ChatGTP)

thanks!!

1 Like

Might be a silly question but if we have everything set up on a previous version of the plugin, what do we have to do (if anything) so that it is updated to the latest version?

1 Like

@minh.iae - Thanks for trying the plugin!

You can get the number of tokens used by checking Last Request Token Usage - Total. (it’s also broken down by Input and Output).

You can set up a workflow to use this by using the plugin event “Message Generation Complete”, and then deduct tokens from a user’s balance, for example. Something like:

  • Make changes to a thing
  • thing to change = Current User
  • field to change = Token Balance
  • New value = “Current user’s token balance - Last Request Token Usage - Total”

Does that help?

@Guillaume.b - It depends on which version of the plugin you were using before. Very little has been changed since 5.12, except the removal of the Ensure Connected action, and the addition of “Connection Info”, which needs to be set on Send Message, to “Data Container A’s Connection Info”.

Several new features / events / errors / etc. have been added though, so you may want to implement those.

Which version are you upgrading from?

Thanks @launchable, did not see the last token!

I have another question. Is it possible to get the streams generated from a Reusable Element (RE) into a page?

My inputs and Button are in a RE but I want to display the streams in the Floating Group on my page. I’ve tried to put the ChatGPT element on the page and pass the element id to the RE so that I can generate the server call inside the RE but the trick did not work.

Do you have maybe another solution to get around this? :slight_smile:

Thanks for your help!

@minh.iae - Good question. I’m not sure. I’ll follow up in DMs.

@launchable - Love your plugin and all the effort you put in :pray:

Regarding the Message Generation Complete event: am I right to assume that this only works with the ‘ChatGPT - Send Message w/ Server’ action but not with the fallback?
Was experimenting with this and realized it would not trigger this event if the fallback was used. Could also be on my end but wanted to check here before I look into how to achieve this if not connected (e.g. custom event with a conditional).

1 Like

@roman.leeb - Glad you’re digging it! :bowing_man:

You’re right, yes. The event isn’t triggered from the fallback. But the fallback action returns a value, so you can use that instead. eg., “If Result of Step 1 Fallback’s Response Message is not empty”, or similar.

1 Like

Hello, I am currently at 5.13.0. Do you recommend updating? If yes, how?