Getting a lot of these today, is it me?

{“statusCode”:400,“message”:“Operation timed out – app too busy”}

1 Like

seems to be Bubble-wide

ok thanks for confirming, good (in a way) that its not my pages :). The status page should reflect issues like this, so far its been all green today.

You can see that message if your app has maxed out its capacity and is so far underwater that it’s better to just fail than to keep trying to finish a request, so if you see that on your own app, you may need to upgrade to more capacity.

It looks like this morning we may have been maxing out the capacity of the Bubble homepage, whoops :frowning: . We limit the capacity of our homepage, too, so that a spike of traffic to Bubble only takes down our homepage and the Bubble editor, rather than breaking all user traffic. We generally monitor to make sure we have enough allocated, but there was an issue with our monitoring that we are fixing now.

1 Like

i don’t understand how “what seems to be many peoples apps are all maxed out at the same time” ive justed added extra capacity at a cost. was this an issue or did my app actually max out…Im not even live… just dev and test!!

my stats… could anyone point out where i went over a threshold.

We’re working on a visualisation tool to show how apps consume capacity. It’ll be live soon.

4 Likes

To Bubbleboy’s point, I’m assuming those of us who are paying a monthly fee are on a different system than those on the free versions?

Thanks and that’s good to know but in the meantime could you answer the question. If I am not live and currently only testing and my app is maxing out out then I’m obviously doing something wrong. From the above usage stats could you indicate exactly where i maxed the app?

Ive added extra capacity at a cost. Don’t get me wrong im more than happy to pay if this is necessary and i did infact max out… Call me paranoid but I get the distinct feeling this may not be the case as this happened across what seems to be a number of other members applications

1 Like

I’m not sure about your specific app, but I had some workflows in one of my apps that would cause it to max out until it completed, and that was just with one user (me). I’d guess your issue is probably similar, may be a complicated workflow that hogs all the system resources. Hopefully some of the other visuals the team is working on will be helpful in diagnosing these things.

@potentialthings Andrews thanks for stepping in … It may very well be the case with my app but I assumed the server capacity usage and real-time metrics /reserved capacity which we pay a premium for on a professional plan would indicate this. I would have thought the reserved capacity would be shown and the benefit of additional capacity add-ons in the pro plan could be gauged with an indication showing its maxed

These are my stats for the past 30 days and it shows 179 minutes where the application hit maximum…

The spike on 17 June when I mouse over it states 16% of available CPU usage…

In the back end and I can add additional capacity and it actually states in the logs tab I can see how my application is doing against current capacity… Maybe im blind but I can’t see it

Don’t get me wrong I don’t see this as a negative rather a positive identifying areas where I may not be optimised and need to relook at my application and its logic

The top left hand chart shows periods where the app hit its max capacity!

1 Like

Interesting. Your charts don’t look as I would expect them to. When mine maxed out, I was able to see when it was happening. Because of how I develop, that helped me identify what it was that was causing the system stress. Your charts befuddle me a bit!

@potentialthings yep me too. over to team bubble to do the reveal

We’re working on this, so you’ll be able to see very soon. One thing to keep in mind is that if you run one workflow on a list of things for instance, that is data-intensive, your application can be maxed out over one minute (and be fine the rest of the time).

Sorry but that’s not an answer. If you can manage this from a billing perspective and notify me the app is maxed out why is this not reflected in the report. I’ve paid $20 to add capacity . It’s not the money It’s more about having confidence in what I’m being informed of .

2 Likes

Agreed with Bubbleboy on having a dashboard to see, and help from bubble to optimize configuration. I am worried that the thresholds currently set will be too low to actually run a viable app (mine warned me and its only me on the site), increasing cost/adding capacity might price some apps out of the platform? I hope this is not the case as I have some significant hours (is this metric anywhere?) developing already invested.

Hey, sorry about the confusion. You can be maxed out and still see the top-left chart at low numbers, because the chart is an average over that time window. So, imagine you have a scenario like this:

1:01:10 pm – your app hits 100%
1:01:20 pm – your app is at 20%
1:01:30 pm – your app is at 20%
1:01:40 pm – your app is at 20%

So there was a 10 second window at 1:01:10 pm, where you app is completely maxed out, but over the course of a minute, your average is going to be closer to 20%. And over the course of an hour, if your graph is zoomed out to a longer time interval, the average could be even lower.

So in other words, just looking at the left-hand graph doesn’t tell you if you are maxed out or not. That’s why we also have the upper right hand chart, which shows you an average of the total amount of maxed out time.

You need both charts to understand what’s going on. If the left-hand chart is low, but you see blue on the right hand chart, that generally means you have spiky, expensive operations (like a search that returns thousands of results) . If the left hand chart is high, that generally means the capacity is being driven by continuous app usage… either a lot of users or a lot of scheduled workflows all running.

Hope that helps explain what’s going on

1 Like

Sorry but I still don’t see how I am even close?

1 Like

@jnbridges, the charts are a bit misleading since they’re showing averages over a period of time. So, when you view the data for the last 30 days, each data point seems to be the average over a few hours or something. During that time, it’d be easy to be at 100% for a couple minutes and near 0% for the remainder of the time, which would show up as 1 or 2% average over that hour.

So, the smaller time frame you review, the smaller the units of time are that are averaged.

Additionally, you’ll see that you charts show you system has been at max capacity for 27 mins over the last 30 days. This is the best information about overall app capacity that’s provided as of now, imo.

It sounds like Josh and Emmanuel are actively working on providing more transparency into all of this. So, let’s let them solve the problem through technology (so it solves it for everyone) rather than pushing them on any one specific instance.

1 Like

@sridharan.s lets let bubble clarify this . I’m with @jnbridges. We need to know this stuff . @josh thanks for explaining . The thing I’m looking for here is this… if as a single user if I’m maxing the app on a list of 20 things then as a commercial app I doubt it will scale which is fine if I know know vs expel more time. Also this has never been an issue before so has a threshold changed ?

Hey @Bubbleboy, so if I’m understanding your concern, you’re worried that because you’re seeing brief periods of being maxed out while you’re still just developing your app and it’s just you with a little bit of test data, that means that it will be super-expensive to run your app at scale. Also, you’re wondering why you’re seeing this now and if anything has changed in the system.

I totally understand why this would concern you; you don’t want to invest a bunch of time in something and then find out it is not useable in the real world.

First, to answer your question about what changed: we’ve only been tracking and reporting capacity metrics for a month or two, and we very recently (over the weekend) released an email that notifies you if your app is sometimes hitting its limits. So capacity tracking is much more visible now than it was a while back. If your app is hitting capacity limits for very brief periods of time, you’re likely not going to notice it happening, since it’ll result in slight delays that are indistinguishable from the normal variability of your internet connection, your browser, etc. Now that we’ve released the new tracking features we’ve been getting a lot more questions about this because people notice that something is going on when otherwise they’d miss it.

Anyway, in answer to your concern, yes, if your graph is totally maxed out when it’s just you, you’re probably going to need to make major changes to your app. That’s not what I’m seeing on the graphs you and @jnbridges are posting, though. I’m seeing very brief, temporary periods where it is over.

Keep in mind that it’s not necessary to never hit capacity limits… if you just hit them briefly, things will run a tiny bit slower, but your site won’t totally break. The error messages people were seeing the other day I believe had to do with a couple temporary issues, including us not allocating enough capacity to our internal infrastructure.

The good news is that scaling up is not linear. Let me give you an analogy: imagine you have a house with one shower. Whenever someone uses a shower, your house’s showers are at their max capacity… if a second person wants to shower at the same time, they can’t. Imagine your house has 10 showers, though… even if you have 10 people in the house, you’re very unlikely to be at max capacity, since it is very unlikely all 10 people will want to shower at the exact same time. So just because you are hitting limits with one user and a small amount of capacity doesn’t mean you’ll hit limits with more users and more capacity.

We’ve seen this in practice. One of our users recently had their app featured on national television, and they were able to manage by scaling up to 20 units of capacity. So I know that it’s possible to run Bubble apps that get a ton of traffic.

Finally, as Emmanuel has mentioned in the thread, we’re building tools to help you find parts of your app that are using a lot of capacity, and expect to release them sometime in the next week. There are often multiple ways of accomplishing the same thing in Bubble (as there is in programming), and some of them can be much more efficient than others. So yes, you may need to make changes to your app as it scales, to eliminate things that are using too much capacity. We also may make changes on our end to make things more efficient as we see the data from these tools: we haven’t had much visibility into capacity usage in the past, so we expect to be able to make the Bubble platform faster and more efficient based on what we’re building.

I hope that helps give you the information you need to decide whether it’s worth continuing to invest in Bubble.

10 Likes