New algorithm for managing resources on shared plans

Hi Bubblers –

I want to give you a heads up on some behind-the-scenes changes we are working on.

One thing I know a lot of people care about is how fast bubble-built websites are. There are two components to “fast”: average speed, and volatility (how much speed varies depending on time of day / traffic). We’re taking some steps to reduce volatility on the shared cluster by doing a better job of isolating users’ apps from each other, so that a burst of traffic on one app has less chance of affecting another app.

Currently, system resources are shared on something like a first-come-first-serve model; we’re changing it to give a fixed allotment of resources to each app. We expect this to lead to much more consistent performance; although your app might slow down if you get a ton of your own users, your app should be less likely to slow down if someone else’s app gets a ton of users.

The downside of this is that the best-case performance (ie, if we’re having a very light-traffic day) might go down a bit, but we think it’s better to get consistent speeds, rather than having it be very slow some of the time and fast other times. We will of course continue to work on boosting the average speed across all situations.

These changes are most likely to be noticeable across data-intensive operations (searches that return a lot of results, and workflows that touch a lot of data), since that’s the most sensitive to overall system resource levels. We expect that average page load speeds (for pages that don’t load a ton of data on page load) will improve slightly as the result of the changes, since they will be less likely to be delayed by long data operations.

Another advantage of this change is that it opens up the possibility of letting users pay for more capacity to run their apps without having to switch to dedicated. We’ve gotten the feedback that a lot of people would like to pay a bit more for faster, more reliable performance, but that the pricing jump between shared plans and dedicated plans is too steep. We’d like to be able to offer an intermediate option, and are looking into whether it would be possible to use this new algorithm to offer this.

(None of the above affects users who are currently on dedicated plans – your apps will still share 100% of your dedicated clusters’ capacity).

Anyway, wanted to keep you posted on what we’re thinking and doing here. We’re likely to roll out this new code as a series of incremental changes starting later this week, and then look into potential pricing options for more capacity once it is live and we have some data on how it is performing in the wild.

29 Likes

Thank you for this update! I think this is a great direction. I’d much rather experience more consistent average speeds than bursts of fast and bursts of slow… for myself and for users on various apps. It goes a long way with establishing reliability. Good to hear that the overall average speed and a new pricing option are being considered as well. :+1:

4 Likes

Thank you guys it be nice to see a speed add-on package that can be added to the plans.

But thanks we all appreciate it

2 Likes

Awesome stuff!

It would also be nice to get a guide, or just some guidelines, on which design choices impact speed.

For example, if I have a dozen elements that need to run the same search, is it better to have one hidden element run the search and then have the visible elements reference that hidden element’s value? Is there a difference between one action that changes a dozen fields and a dozen actions that change one field? Does it matter if a long list of changes are hard-coded into one or more workflows vs iterated over by an API workflow?

4 Likes

This!


Thanks for the update, can’t wait to see how these changes affect our apps :slight_smile: Would people on the Team plan still need to pay more if you decide to allow a purchasable performance increase?

Thanks, guys, that’s great! Performance is one of the major things right now…

2 Likes

+1 to this.

@levon, @philip, @blueback09 – kicking off a performance discussion here: Performance Q&A guide

Re: @philip – "Would people on the Team plan still need to pay more if you decide to allow a purchasable performance increase? "

Probably not. We’re still working out how it would work, but we’d likely use a performance increase as a replacement for the current limits on workflow runs, so we’d probably reshuffle the whole plan structure to take it into account.

2 Likes

Sounds really cool!!! +1 from here

Awesome :slight_smile:

I like this idea. Performance is something more convenient than workflows counting. Workflows are something that can be quite different for different types of applications.

As for recourse isolation in a shared environment, I imagine it is something like that, how it works with CloudLinux OS for Shared hoster’s tenants.

1 Like

I have a feeling this change has had a pretty substantial negative impact on our app’s performance.

I get more complaints from users hitting errors and I also see more errors like:

Yes, that happens during some complex workflows, but nothing extreme and also by just a few users (mostly me for the taxing stuff).

Couple of questions:

Are users in Europe more impacted by this change than users in the US (east)?

Is there a difference in resource allocation for the different plans? I think it’s fair that apps on a higher plan get more bandwidth (as those apps probably have more users, workflows etc.)

The dev version as well as the live version of each app are affected in the same way? So if the dev version is very busy, that affects the live version?

I hope you get those speed plans up soon because I’m a little woried about this step. Especially when I read that average page loads will improve slightly while ‘best-case performance’ and data-intensive operations are significantly impacted.

Hey @vincent56, sorry about the negative impact – we’ve definitely seen this come up for apps that are more data-intensive. We are currently working on some changes right now to try to mitigate that impact as much as possible, and you should see some improvements as soon as this weekend.

On the plus side, since we made these changes (about a month ago), we’ve seen a vast increase in overall system stability – on average, people’s apps are going down much less than they were before.

In answer to your questions:

There should not be a difference in how these changes impact European users vs US based users. This is all about consumption of server resources… latency between the web browser and the web server shouldn’t have an effect.

No, there is not a difference in resource allocation between currently existing plans. One of our top priorities is changing our pricing infrastructure to enable buying capacity a la cart, so that low-usage apps don’t need to pay as much as high-usage apps. We’ll grandfather existing users into their current plans, and give them the option of switching to the new system, so that everyone can pick the option that works the best for them.

Yes, dev + live versions of the apps share the same resource pool. We considered breaking them apart, but, like workflow counts today, we decided it would be easier for users to share a common pool (some people need to do intensive stuff on the dev version of their app)

Anyway, the worst is over with – any negative impact this change will have has already happened, and like I mentioned, we’re making changes to try to mitigate it. When we roll out the ability to buy capacity, that won’t affect the performance of current apps.

2 Likes

Good to hear a possible fix will be here already so soon. Also looking forward to the new plans, curious to see how they stack up against the dedicated plan.

Hey @josh

Now this algorithm is live and everyone has its own capacity will guys be considering giving us an option to preload data on the page? Also controlling speed rate of upload and repeating group loading for vertical scrolling and ext vertical scrolling?

It should make sense now that everyone has a Max capacity and it will improve sales of capacity addon sales.

1 Like

@josh Lately our app has been performing terrible, things almost completely freezing at times. Especially on views with many auto-binding elements.
Are you collecting any statistics on request response time / processing time? It is not a good feeling that the performance of what we pay for has decreased.

I have noticed that too, there is a significant drop in performance and page loading time for the last two weeks or more.

I have noticed that performance varies quite a bit depending on time of day too.
For example:
After 2am US east coast workflow takes less than 1 second
Afternoon/evening US east coast and workflow takes more than 15 seconds.

1 Like

@josh Perhaps you can clarify something for me. I have 5 allowed apps on the Pro plan. Is each of those apps allowed 20% of my total server capacity or is my total server capacity divided by the number of active apps that I have?
For instance, I’m using 3 of my 5 allowed apps. Is 20% of my capacity allowed for each app or 33%?
Alternatively, is my server capacity split depending on the demand from each one of those apps?
The reason that I ask is that two of my apps are basically just dormant old versions of my site that I can refer back to if I really screw something up with my current design.
However, if having those 2 extra apps means that my primary active app only gets 33% of my allowed server capacity that could potentially be a big contributor to the performance issues that I’m seeing.

If that is the case, Is there anyway for me to identify certain apps as “inactive” so that they aren’t allocated any server capacity or do I just need to delete them?

I’ve been struggling to optimize all of my workflows, but if the above is true, I would probably see much larger gains by just deleting the two old apps.

Thanks for any clarifications that you can provide.

1 Like