Workload Usage/Costs Problem - Solved (i think)

[nick.lockley]

So we have been having insanely high workload usage spikes over recent days whereby our daily workload usage went from around 2-4k units a day to 45k units per day & we simply couldn’t work out how to fix it or what was causing it as many of the units were used by data searches on site.

So we booked an hour with bubble coach @boston85719 (Matt) today and it’s the best thing we could have possibly done to address the issue.

Matt clearly has an in depth knowledge and experience of how Bubble workload usage is calculated and various methods to not only reduce the usage but to completely remove it from our usage/workload costs completely in many areas.

He went through our metrics in detail with us, helped us identify exactly what was causing the issue, then went through the site with us to identify what was causing the spikes. Immediately found several usage heavy/duplicated search expressions, some of which that had additional data searches within the consttraints which was clearly causing heavy usage.

He also showed us various methods to replace our previous expressions with ones that are less workload-intensive and I have gone from worrying that my app is not even scalable, to looking forward to things again.

Hat’s off to Matt on this one! Absolute legend.

Note, I will update this post with the new usage results in a day or so alongside the old results when we have tested the difference/outcome but already confident that it will be even less than it was prior to the recent spikes.

UPDATE: Here is a snapshot of the before and after results. We launched the updates at 18:10 and there’s a clear difference since. We also know exactly whats causing the usage since and thanks to matt also know how to remove that from the workload usage. Absolute RESULT

Kindest Regards,
Nick & The BetterM2M Team

[itsdoleey]

Can you share some of those tips that helped? been scouring for ways to cut down on the workload for my app too.

[nick.lockley]

Hi,

I would guess it differs greatly for different use cases and circumstances so your probably better off speaking to Matt. But nn my case it was data searches consuming most of my workload. In that scenario, you can find out which elements are running up the usage by going to settings in bubble - click “metrics” on the first page right above where your workload usage/% is shown.

From there filter it to the last 24 hours (or whatever period) and itl show you a pie chart of the breakdown of the usage. In my case most of the pie was data searches. So click on that slice of the pie and it then shows a new pie showing the exact breakdown of the data searches. The colour key on the right of the screen shows the highest % usage at the top then goes down to the least.

From there you can usually click on it again to go straight to that element in your app.

Methods that fixed my problem

Data searches within data searches and repeated searches for the same data/IDs.:

I had many data searches as my app shows a lot of dynamic content on the website. Many were doing searches and also had constraints in those searches that did additional searches (finding an ID for the constraint). Solution to this issue was to simply have a hidden floating group behind the page, this group does a single data search to find my “user/partner/dataset”, then instead of doing additional searches to find the ID I simply refer to that floating group to access the data it’s already retrieved. So now we only do that search once, not every time we need to use the data againn

Switching to option sets where possible:

Bubble doesn’t seem to charge for searching option sets. So things like my reviews and faq’s etc which before searched the database now simply pull from an option set which completely eliminated the workload cost/usage. I can also set attributes against my option sets to filter it if need be.

Search Inputs:

I have a search input which searches a database. we switched this to an option set and replaced the search input with a repeating group/focus group under an input as we think it might be less workload heavy.

Cutting down unnecessary repeated searches by utilising parent groups.:

We had some elements, like buttons etc that showed prices each doing their own data search for the price. We fixed it by simply making the search from the parent group then referring to the returned values of the parent group. (Noob error on my part)

More technical methods to reduce data search workload usage:

There’s a plugin called JSONator alligator or something that can store records in JSON or something then refer to that. We didn’t have time to cover that in the session sadly but it may be our route to completely removing even the most necessary and data heavy data searches.

Again…Matt understands this far better than I do & we simply didn’t have enough time to get into the more technical methods but I would say reach out to him directly he’s incredibly helpful and approachable.

Thanks,

PS, Heres a later test of our workload usage for today. It was 45k per day before we contacted Matt and fixed it. It’s now 1140 per day so super happy with the result.

[itsdoleey]

Wow, reducing 45K to 1K+ is incredibly optimized!

Mine is currently around 9-11K per day. It used to be 4-5K about two weeks ago, but an influx of traffic increased workload consumption, peaking at 15K per day. I’ve been optimizing everything I can and managed to bring it down to 8-10K.

The floating group concept has inspired me to try a few new strategies to reduce the number of calls. Thanks!

[Jici]

@nick.lockley

There’s a plugin called JSONator alligator or something that can store records in JSON or something then refer to that. We didn’t have time to cover that in the session sadly but it may be our route to completely removing even the most necessary and data heavy data searches.

This is kind of stuff that make nocode… no more no code… and this is a big issue from my point of view that Bubble need to care. In their own initial session about WU and how to “save” WU, this was the kind of example they give (and also to use external service like segment). What the point to use Bubble in this case?

[Jici]

By the way, thanks to share! this is very helpfull for all Bubblers. And a big thanks again for @boston85719

[nick.lockley]

No worries! This community has been so helpful over the last year. And yeah, honestly, the guys worth every penny and has been fundamentally useful to the growth of our platform. Had several sessions over the last 8 months and every one of them has fixed a problem we couldn’t fix. Recommend

[boston85719]

@itsdoleey

The floating group concept has inspired me to try a few new strategies to reduce the number of calls.

This floating group approach is not really based on floating group. So the use of the floating group is to be able to get the elements it will hold out of sight of the user and out of the way so as to not affect responsiveness of the page. This is why the floating group is used as the container of the elements inside and is set to float beneath the page.

What is inside of the floating group, as I use it in all of my apps are plugin elements and other elements like repeating groups or groups that are not to be displayed but are used to store values that will be repeatedly referenced by other elements on the page that are displayed to the user.

So, for example, if there is a search that is needed multiple times on the page because there are multiple elements that need to display a value from the search, and the search is for a single item, create a group that is placed into the floating group. That group will have a data source that is the search. Then for each element on the page displayed to a user that needs a value from the data entry, the dynamic expression will reference the groups ‘thing’ rather than having to perform another search.

It is unclear via Bubble documentation whether or not we are getting charged WUs for each time the search is listed on the page, so it is not completely clear how much WU savings there are to be had with that approach. However, having spoken to Bubble support prior to the WU pricing introduction, I was told the same search used in multiple elements on the same page is actually only performed once, and those discussions were focused on page performance and speed, so it could be the case that Bubble is charging WUs for each time the same search is on the page, even though they may just be performing the search once. The reason I would believe it is possible we are charged for each time the same search is on a page, is that we are charged WUs for each entry in a backend workflow parameter that is a list of things when running recursive backend workflows, which seems wrong since we are already charged WUs to retrieve those items to populate that parameter when scheduling the backend workflow the first time.

@nick.lockley

PS, Heres a later test of our workload usage for today. It was 45k per day before we contacted Matt and fixed it. It’s now 1140 per day so super happy with the result.

I was expecting to have saved you some WUs with the changes we made, but reducing your WU consumption by over 97% is a surprise to me. I hope those numbers are correct.

@itsdoleey

Can you share some of those tips that helped? been scouring for ways to cut down on the workload for my app too.

Best and easiest thing to tackle first are the searches. Checking in the editor using the WU tools currently available, any searches that are consuming a lot of WUs, look at ways to eliminate them. If they are in a workflow action but the data is already on the page, reference the data from the page instead of performing the search in the workflow action. If there is a search being used as a constraint in a search, find ways to eliminate the need for the search as a constraint, which may be as simple as adding a data field to the data type of the main search. Any data types that are searched for that do not need to be dynamic from the standpoint of allowing users to create/modify those values and they are for the most part relatively static, use an option set instead, even if that means biting the bullet and spending an hour to populate an option set with 100 options. An example of that might be if you have a data type for countries, and you are searching for those countries to display to users, change that to an option set.

Other things with searches, are the number of fields returned versus needed. So, if you have a search results page for products and the product cards only display 5 fields of 30 that are on the product data type and the filter feature only references 5 or 10 fields, consider having another data type created that is used only for the search page to cut down on the number of characters returned. Since Bubble charges per character returned from the server, you may be overpaying greatly based on the number of characters returned for the search.

[Jici]

@boston85719

I was told the same search used in multiple elements on the same page is actually only performed once, and those discussions were focused on page performance and speed, so it could be the case that Bubble is charging WUs for each time the same search is on the page, even though they may just be performing the search once. The reason I would believe it is possible we are charged for each time the same search is on a page, is that we are charged WUs for each entry in a backend workflow parameter that is a list of things when running recursive backend workflows, which seems wrong since we are already charged WUs to retrieve those items to populate that parameter when scheduling the backend workflow the first time.

Another reason why Bubble is not ready to use WU as pricing scheme.

[IljaIljitsj]

Great tips, super useful

[joaquintorroba]

This resonates a lot with me, thanks for sharing!