I’ve read a few posts about this but am not sure why my backend workflow was hit with an ‘App too busy’ error, which hasn’t happened before on the exact same workflow. It’s not the smallest workflow (it’s a conversation with Open AI, but each message changes so it isn’t really recursive), but I checked the logs and it never even hit 2% of capacity, so why the timeout? I’ve now started trying to split to Part 2, but I need to know why the workflow stopped.
@emmanuel@josh please let me know what’s happening with this. I’m so close to launching and this is really concerning.
You should really look at optimizing the workflow. Having almost 50 steps is not good design and you might want to consider scheduling some in a backend workflow with some interval so that workflows don’t get clumped together.
You can also submit a support ticket but with such a workflow, it will be hard for them to help you. Do break them down into smaller custom events and save some data in your database as error logs so that you can identify which steps break the workflow.
Thanks for your message, Jordan! I have split it into 2 Parts but it still fails (It got most of the way through Part2). Odd, though, that it was working fine over the past few days??
How do you add in an interval?
Custom events can be triggered in order, right?
Should I split it into more workflows/events?
Currently I make a final change in Part 1, which should only Trigger Part 2 when that change happens.
Use a scheduled workflow instead of a custom event to add in an interval. Also, do you need to create so many aiMessages in the main workflow? Can you schedule some of them to create later?
The main point here is that it is not easy to debug which step is failing just by looking at the error, so you need to continue to reduce the number of steps in the main workflow.
Looks like I do indeed need to split further, failed again. Still annoyed at how it’s a few things being created and workflows done, not thousands of rows, and I’m one single user - how can it not handle that?
Out of interest, you mentioned that each message changes - do they change based on the previous responses from OpenAI in the same workflow? I’m just intrigued to know why you need to call OpenAI so many times in a single workflow.
Usually when I’m using OpenAI I make a single call that contains the whole “conversation” in one - i.e. multiple messages from a mix of roles (assistant/user/system) - but I guess that doesn’t work for your use case?
I’ve probably done this the very long way so I’m keen to hear your approach!
Sounds like you’re also having a conversation (where there’s a chronological exchange of messages)? I have questions that need AI answers, so I ask the question, capture the AI’s answer, then append that to the conversation and add my next question, so the history is built up. It needs to be aware of its previous replies, so not sure how I’d send those in one call?
Have I missed something?
I’m thinking of making one Conversation, and instead of creating multiple discreet messages, that I’d try ‘Make change to Conversation (a text field)’ and then keep that as a running track of the conversation?
Splitting into 3 scheduled calls 5 mins apart has seemed to work, but I think I’ll try the above approach as it cuts down on creation of data, but does involve a lot of change to that Conversation field.
Thanks for the info. It’s hard to know whether your approach is “the long way” without knowing more context, but I can understand if you don’t want to share the details here.
I kind of wonder if you really need to be appending the answer each time, because the assistant knows that stuff already - after all, it’s giving you those answers. But it’s very possible I’ve missed something.
Anyway, in case it’s of any help…
I’m not sure what model you’re using but I’ve generally been using gpt-3.5-turbo, where you can fire it one set of messages - in a single API call - as the “conversation” and it’ll provide a response. Something like this (would obviously be formatted as JSON):
Role: system, Content: You’re a travel operator providing a local guide for the user’s vacation.
Role: assistant, Content: What is your name?
Role: user, Content: My name is <get current user’s name from DB>
Role: assistant, Content: Where are you staying?
Role: user, Content: <current user’s holiday location>
Role: system, Content: Provide a personlized local guide for the user’s holiday including local towns, days out and main sights.
So I need to pass the previous conversation, and ask each question separately as I need to reference them later (and don’t want to do ‘find and replace’.
I’m currently trying to ‘make changes to conversation’ rather than make all new messages, will see how that works.
I’ve thinned out my workflow heavily, as you suggested, so now I only send 2 messages to OpenAI, the second being the big one: 10k characters/1,800 words/2,268 tokens according to the Tokenizer, and well within 8k token limit. All steps done in one workflow of 16 steps.
I did wonder a little, looking at OAI, if the API calls were too close in time? I tried 2 tests; the first failed and the second succeeded. For the first one it looks like, for whatever (workflow?) reason, the calls look to be very close together; for the 2nd one, which worked, OAI showed these as spaced further apart.
I’m actually still using model “gpt-3.5-turbo” (I realise it’s more limited). Might be worth an experiment just to see if it gives more reliable responses?
Hi @mccjon - I was actually working on something quite similar with another user today, namely running backend workflows instead of front-end streaming, and avoiding timeouts. The plugin can do backend workflows, but we haven’t found out yet whether those are immune to gpt-4 timeouts (I’m going to test it, and we’ve come up with a possible solution if the timeouts are still happening).
It’s tricky to say what’s going wrong (as @nocodejordan & @appedge have mentioned). Couple of questions:
are you using GPT-4?
is your second message, the large one, always the one that fails? If so, could be a timeout error, but I would imagine you’d see that in your Bubble server logs. It could instead be a timeout error at the beginning; ie., sometimes the response doesn’t even start, due to server load or something else.
is it working sometimes and not others, or does the 2nd WF always fail with your current setup?
any idea how quickly the failure is thrown on the “could not connect to remote server” error?
Your idea of spacing out the 2 calls is a good one. It should rule out whether you’re getting throttled at the API level, though I doubt this, if you’re still in development/test mode.
You might actually have 2 things that are taking turns causing you grief lol. The “app too busy” errors are probably from the complex workflow, and scheduling BE WFs too closely together. So maybe your refactoring will help.
But the “could not connect to remote server” is trickier. Let me know if you still see the issue after testing with your 5 min interval (you might be able to achieve the same with as little as 10 or 30 seconds I reckon), and we can troubleshoot some more.
I think you might be right as it usually the last (big) message with the entire history that fails. I’ve had one last stab at it, where I’ve broken the question-answer session into 2 independent processes (and had to sacrifice my final ‘summary’, as that is maybe moving beyond MVP and holding me up). This looks to be working so far I’m going to run a few more tests, but if those don’t work then I’ll give your plugin a go!
I’m going to run a few more tests, but if those don’t work then I’ll give your plugin a go!