Why is this costing so much Workload Usage?

I’ve attached two images to show you the usage breakdown, and the workflows. I would love help to figure out why this workflow is using so much workload units.

Can anyone tell me why this is costing so much? I’m always over 600k each month, and this is the main reason I’ve not switched to the new pricing plans as it will cost me a fortune.

I can’t figure out why this is costing so much when there are other processes I would have thought would be more intensive. This is simply creating a data entry for my notification system. Yes it might be sending to my full list of users(1,000), but the other processes like sending to PostApp to send an email also do this.

Is there anything I can do to optimize this?

How often does it run? Is it recursive?

My theory is you’re sending that user list as a param, bubble charges like 0.02 to pull 1 record. When sending lists the entire list is loaded when it’s sent. I’m assuming you’re looping it so every time so it’ll cost roughly 20 Wu the first loop, 19.98 the second to pull 999 users, etc etc

Opposed to the email portion which I’m assuming you send an individual recipient for

We have different events, and I’m able to add 2 announcements(sent to all eligible people minus anyone who is already registered), and reminders that are sent to only the people who are registerd.

When an announcement is triggered I run it through several workflows to determine the final list. Some events have restrictions based on things like if it’s member only, location specific, if they already registered for the event, etc. Once I have that final list it gets sent the the attached workflow.

It runs throught these steps in a loop for each item until the list is empty. Step one terminating if the list is empty, and the last step looping removing the person it already ran the workflow steps on.

Both the notifications and emails are being sent to 1 person at a time using the same process of taking the list, sorting, and taking the first item. I’m thinking maybe I could optimize it by having the second step just finding the user, and then all subsequent steps can just use results of Step 2. That would remove all the sorting each time. Anything else you think might help?

The sorting I don’t believe is your core issue, rather the sending of the full list. Could you schedule API workflow on a list rather than passing the list every time and it needed to pull every item?

I origionally did it that way, but smarter people then me told me to do it this way instead as it was better for some reason. Less errors or something like that.

They are right, schedule on a list gets buggy on larger lists, not sure why but past a couple hundred/thousand it skips items.

Ideally though you’re not going to load that full list every time though.

I was having issues with it not sending to the full list, so that’s why they told me to do it this way instead. Any suggestions on making this better, but still being reliable?

There’s multiple ways of handing it but it depends on what you want your UX to be.

I’m just speculating here but it appears that you have an event management platform? maybe? If not you can use similar logic but here’s an example for that use case.

When the event is created by the event creator you also have them setup the notification schedule for anyone who subscribes to the event. When a new user subscribes to the event you schedule the API workflow for the notification time that was setup by the event creator (only for the user who just subscribed). (with a “notification” data type with statuses for sent or not & expected send date)

This would then result in rather than a list that gets looped through and loading the full list on each loop causing compounding WU charges for data loading you’d only be sending the notification with a single recipient.

The looped list and “each result returned from database” is what I’m assuming is racking up your large charges due to how lists load in bubble.

Just dealt with similar myself

It’s an animation group, and we have guests and members. Events is just a part of what we do. We have a lot of events, and this is how we notify our members about upcoming events. So it’s more marketing to people who have not registered/subscribed to the event yet.

Once a list is created are you not just passing that list? I know when I worked with one of the Bubble people they mentioned I don’t need to sort the list multiple times, because once the list is sorted and you pass it it does not change, so you don’t need to keep sorting it. Why does it cost a lot to pass a static list?

Also, if it was the list passing would the one showing the high usage be the one actually passing the list(The last step)? This is actually just the one sending to the Send-notification Workflow. On the middle pie chart(yellow) that workflow does not seem to be costing much, but just the one that’s sending to that workflow.

It’s due to how bubble handles lists of things. Let’s say you have a “list of users” stored in a parent data type called events.

When event is loaded bubble loads the entirety of the list on load. This is why when you have nested lists it’s pretty stable for <100 items but once you’re passing that it gets slow and once you move into multi thousands it gets REALLY slow, because the full list has to load first.

the same logic works when you pass the list of recipient in your flow.

So what you’re doing on that backend workflow is looping a full list pull over and over.
1000 users (20 wu to pull)
999 users (19.98 wu to pull)
down to
100 users (2 wu)

The back end workflows don’t cache data that list is being pulled over each time it runs. The sort of the data is also redundant, I’d pass it sorted. If I’m not mistaken operators execute individually. So even though that list is pulled 1 time in the backend and it’ll not need to repull data for each param (ex it doesn’t pull the list for recipient param, then again in email-option param) it only pulls it once the sort operator will reexecute for each field I believe.

“schedule on a list” is different from recursive because it sends “this item” meaning no need to pull that list every time like you are in the parameters.

I’m not a big fan of schedule on a list for large lists but what I’ve done in the past is something similar to this - schedule a “flow check saftey net” on the execution flow for +5 min. When this executes it says “if notifications for that notification batch “sent” = no is not 0” then it reschedules the execution flow to start again on that batch of notifications where sent = no. This way nothing gets missed & you avoid the redundancy in your loop pulling that list.

there a lot of ways to handle this.

I may be wrong in thinking the list is a list of objects not a list of static texts? If it’s objects bubble pulls it.

I would be curious to see if you pass “list of recipients unique ID’s” as static texts so it doesn’t pull data but the 1st time on the execution flow then you search for the first items unique ID if it would stop the return data costs bc it’s not actually pulling all objects from the database.

Rather your list is static text of ID’s and you use a search action to pull just item #1 of that list.

this kind of workflow i no longer recommend bubble for 1000 users/events/notifications.

use xano backend.

its a joke cos 1000 people is nothing but unfortunately the markup on search /crud operations from bubble is very high.

@TipLister Can you clarify which workflow is no longer recommended?

something which triggers other wflows.
eg new event to new event added to 100 people, to notification to 1000 people.
or match all users with same answer to a question, 10 new questions per day.

no problem if your app charges per person but if it is freemium eg eventbrite, you couldnt build that on bubble anymore and make a profit.

I reached out to Bubble support to get an answer to this issue…below is their response, and based on it, I think your assumption will hold true that it would be better to send a list of unique IDs as text and then for each loop perform a search for the single UID that is required for that loop.

Based on your message, I understand that, in this case, we’re discussing running a recursive workflow on a list of items – the list of items we’re recursing over was initially defined on the client as the displayed list within the Repeating Group.

You’re right – on the client, the search is done once when the page is loaded and the Repeating Group is displayed.

However, on the server, the items must be pulled with every call.

Here’s a timeline of what happens:

  1. The search for the Repeating Group runs, and it’s populated using a search. Workload units are charged to fetch these items.
  2. For these items, a call to the server is initiated, and the list parameter now needs to be populated for use in the recursive workflow.
  3. The server call uses instructions from the initial client-side call but not the data itself.
  4. Since we have instructions on what data to pull but not the actual data, we have to call the database to pull the actual data from the server.
  5. We receive instructions for the next recursive call on the server for the next items to pull.
  6. We make another database call using these instructions, consuming workload.
  7. We repeat steps 4 through 6 until the recursion ends.

In short, we use the list of users as a reference to determine what data to fetch from the server – but the list is simply a reference. The actual data will still need to be pulled, since everything happens on the server for backend workflows.

You can use the data sources for Repeating Groups as data sources for other elements on the page without accruing additional workload units. However, this isn’t true for server-side actions.

We cover this in our manual entry here:

Each action in an API workflow will also spend resources. Again, we need to keep in mind that on the front-end (the page), we can sometimes leverage client-side actions to move some of the processing away from the server, but in an API workflow everything happens server-side.

3 Likes

Beautiful🙌 thanks for getting a confirmed answer on that!

1 Like