Extending timeout period for API calls to GPT-4

Hi all,

As of Wednesday April 5th, we have fixed an issue that was causing API calls to GPT-4 to time out, which resulted in an error message of “Could not connect to remote server.”

To address this issue, we have extended the timeout period from 50 seconds to 150 seconds to give API calls a chance to retry before hitting the 5 minute request duration limit. Calls that take longer than 150 seconds will continue to time out.

Plugin API calls will still follow the same retry policy as before (4 retries) but in cases where a request exceeds 150 seconds, subsequent retries have shorter timeouts in order to stay below the 5-minute existing limit.

If you find yourself continuing to run into this issue, please submit a bug report and we will take a look!

19 Likes

Cool that you guys are eyeing GPT integration and ensuring proper functionality. Hope you guys dive head first into GPT and other AI tech.

1 Like

Nice! Any plans to support GPT-4 streaming?

5 Likes

Would be useful to be able to define timeout manually for each API call.
Anyways, that’s a great update! Thanks :partying_face:

We’re building an app that uses agents and other complex loops before a response is recieved which can be almost 5 mins per call. Only 150 seconds mean we’ll have to drop bubble as the base for the app…

1 Like

Rohan, I’ve seen tutorials where Make.com has been used to handle more complex API calls for GPT4 - so you can use that to interface from Bubble - depending on how much of your app is built already or switching cost you have.

1 Like

Hey dude! Would love to chat about ways around this limitation.

Have you considered using an external tool like google cloud functions or xano or [insert other tool here]. Using gcp would be a much much cheaper option

Essentially you’d:

Create a thing in your db with the relevant user prompt and related info.
Send the prompt and it’s ID to a cloud function
Let the function process the api call to OpenAI
When the response returns, the cloud function would call a backend workflow/the data api to update the record accordingly

7 Likes

That’s a good hack!

Yep, and make com seems best for this purpose!

To confirm, this issue has not affected calls to GPT-3.5. Is that correct? (Have not seen any timeouts in our work…). Thanks!

2 Likes

@jared.gibb FWIW that’s EXACTLY how I’d do this though I’ve had no incentive nor time to try it. Since Things and Lists have that sort of default “live query” connection to the database, it should mostly work like magic. I’d probably wrap the Bubble part in a plugin, just because that’s how I roll.

2 Likes

It is also worth keeping in mind, token/response length affects risks of timeouts.
If you can parse your requests into smaller pieces / chunks - the 150s time limit is less of an issue.

Conscious the wording of this post is very specific about API calls to GPT-4. Can you confirm which of the following is correct:

  1. You’ve extended timeout for api calls to GPT4
  2. You’ve extended timeout for all API calls
1 Like

We’ve extended the timeout for all API calls!

3 Likes

We are noticing slow down in our response time using ChatGPT 3.5 Turbo. It used to take up to 2 minutes for a task to be completed, now it’s taking more than 5 minutes for the same task.

The call should not wait for 150 seconds, it should finish quicker if the app gets a response…

Do you have any advice on this?

Hopefully you’ll decide on integrating Chatgpt natively into Bubble. Flutterflow for example allows creating custom components with the help of AI. :+1:

1 Like

My general suggestion from experience is to keep GPT asks short and concise. Break a job your app does into multiple tasks. This is a more complex way to go about it but you most likely will get better results and never time out as you

We do keep it short and concise and we do get amazing results without any timeout. Just after this update the response time has significantly increased. Only when running the call via a Backend workflow.

Could you please submit a bug report for this? This isn’t expected; this change shouldn’t have any effect on response times, just timeout limits.

Is there anything needed to opt in? Calls from the API Connector plugin (I assume that’s what this is affecting?) are still timing out after 30s for me (which I had down as the timeout length, rather than 50s)