Reduce WU for Scheduled Workflow to send email

eric10 · 2024-09-25T21:26:11.575Z

I understand sending out emails to everyone in my database is going to take a fair amount of WU, but I feel like it should not be as many as it is. I recently switched to the new payment plans and my monthly average WU is 300k-500k, and this will get expensive.

I was previously told to set up this Backend workflow up as seen in the attachment to avoid errors. I would have it fail on me halfway through when I used the “Schedule an API Workflow on a list” way, and I was not sending that many(less then 1,000). From my understanding, this list method works better now, but my question is will it take considerably fewer Work Units and it be worth my time to switch these over? Also will it be reliable?

Screenshot 2024-09-25 at 2.09.26 PM1448×1342 125 KB

As you can see in the attachment, it checks if the list is empty if not it runs it through the process. Since there was multiple steps and I did not want to have to keep doing the sorting each time I thought it would be more efficient to do one sort to find the user I will be sending to this round, and then all following steps only need to use the results of Step 2. In the below chart, you can see this is the largest WU problem I have in this workflow.

Since I use these to send out emails about our upcoming events I can’t reduce or not send them, so I would appreciate a more efficient workflow that still sends them all out but more efficiently.

Thank you for your help.

Screenshot 2024-09-25 at 2.09.49 PM2046×928 99.9 KB

georgecollier · 2024-09-25T22:05:06.828Z

eric10:

From my understanding, this list method works better now, but my question is will it take considerably fewer Work Units and it be worth my time to switch these over? Also will it be reliable?

Works reliably up to 100,000 Things. You have to upgrade to the workload based plans before you can do this, which is daft, as it means you have to upgrade, then change, then test. You can’t change it to optimise it, and then upgrade.

eric10 · 2024-09-25T23:27:06.441Z

Thank you @georgecollier, I’ve already upgraded to a new plan with a Add-on Workload.

My question is if switching to the List instead of the way I’m doing it if anyone knows if it would be more efficient. I have to believe there are smart Bubble people out there who has experience with these types of things who know what the faster option is without me having to test it.

Jici · 2024-09-25T23:42:17.338Z

My experience actually show often a reduction of WU arpund 50 to 85%. When you will move the the SAWOL function, you will also review your workflow that could lead to make your workflow simpler and take less action than before. So each workflow are different and can lead to different results for WU.

boston85719 · 2024-09-26T04:32:41.237Z

eric10:

My question is if switching to the List instead of the way I’m doing it if anyone knows if it would be more efficient. I have to believe there are smart Bubble people out there who has experience with these types of things who know what the faster option is without me having to test it.

You can check out from the words of likely the most knowledgeable person on the subject with a use case example of an email campaign

[Feature Enhancement] Improved Scalability of Schedule API Workflow on a List

Example: Running an email campaign

Let’s say you have 20K users, and you want to send them all a personalized email. You may want to log this outreach by updating a field on each User to indicate the last time they were contacted. You can create an API workflow that sends a personalized email and updates this field for a given User (in the screenshot below, we’ve named the API workflow “monthly-email-send,”) and then use Schedule API Workflow on a list to run this workflow on your entire user base.

Generally speaking, I would deduce that simply removing the need to have a ‘schedule backend workflow’ action for a recursive workflow will be a WU savings when switching to a backend workflow on a list because we are charged WUs for a schedule backend workflow action to do the scheduling, so just removing that from the equation would likely amount to a savings in regards to WUs.

Additionally, the way in which Bubble explained the change and improvement to the workflow on a list feature is improved speed/efficiency.

Looking at your screen shots with lots of schedule API workflows, I’d imagine there may be a benefit to bringing those actions within the different backend workflows into the same backend workflow to avoid those scheduling charges will equate to even greater savings.

Also, there doesn’t seem to be any need for the step 2 make changes to user because as far as I can see that is just fetching which user to utilize for the flow, and without seeing the actual backend workflow parameters, I assume the ‘user-list-app’ is actually just a parameter for the backend workflow and is not a ‘do a search for user’, which means, you can get costs savings in regards to WUs by simply removing step 2 and opt to send into the parameter list of users already sorted as you want them by created date, so the parameter list is already in the correct sort order and just reference the parameter list:first item in all the actions that need that user.

However, when you populate a parameter as a list of things for a recursive backend workflow, every single ‘loop’ of that recursive flow, you are charged for fetching, each time, the list of things that is the parameter. So, lets say you have 1,000 users in the list that is a parameter, on run 1, you get charged for fetching 1,000 Users, then on run 2 you get charged for fetching 999 users, then run 3 charged for 998 users and so on. That is again an area of cost savings in terms of WUs for using backend workflow on a list instead of a recursive flow.

BUT, the workaround I found for avoiding these charges of fetching the entire list of things for each ‘loop’ of the recursive flow, is to instead of sending in a list of things, you send in a list of text which is the unique IDs of the ‘things’ then you have a step, like step 2, where you search for that thing via the unique ID (coming from your parameter of type text) so as to only be charged for fetching a single things for each loop of your recursive backend workflow.

eric10 · 2024-09-27T21:00:19.699Z

@Jici What is SAWOL? To clarify are you saying you’re seeing a 50-85% reduction when you switch from a Recursive to on a List?

eric10 · 2024-09-27T21:23:49.469Z

boston85719:

I would deduce that simply removing the need to have a ‘schedule backend workflow’ action for a recursive workflow will be a WU savings when switching to a backend workflow on a list because we are charged WUs for a schedule backend workflow action to do the scheduling, so just removing that from the equation would likely amount to a savings in regards to WUs.

In order to get the list I have several backend workflows. It goes through each step removing people based on the event’s details. So for example it might start with if they are a member or not, then if they are in the correct location, then if the event is private or public, all this to get to the final list of who the emails will be sent to. Then I push the list to the recursive workflow to actually send the emails. Are you saying its best to not do that on the backend workflow so I’m only sending a list to the backend once? I guess I’m not following what costs more WUs.

boston85719:

Looking at your screen shots with lots of schedule API workflows, I’d imagine there may be a benefit to bringing those actions within the different backend workflows into the same backend workflow to avoid those scheduling charges will equate to even greater savings.

It sounds like scheduling workflows takes a lot more WU than just doing it in the same workflow? Sometimes those other workflows are pretty big, so I’m not sure how I would be able to do that.

boston85719:

Also, there doesn’t seem to be any need for the step 2 make changes to user because as far as I can see that is just fetching which user to utilize for the flow, and without seeing the actual backend workflow parameters, I assume the ‘user-list-app’ is actually just a parameter for the backend workflow and is not a ‘do a search for user’, which means, you can get costs savings in regards to WUs by simply removing step 2 and opt to send into the parameter list of users already sorted as you want them by created date, so the parameter list is already in the correct sort order and just reference the parameter list: first item in all the actions that need that user.

Does sorting take more WU than doing a search each time? I was thinking of doing only 1 search for the list, and sorting it once was better than for each step to have to do a search and using first item. Also, if I sort a list, and then send it on to another workflow does it keep the sort, which means I would not need to keep sorting it? If that’s true that might save a lot of WU just alone.

boston85719:

BUT, the workaround I found for avoiding these charges of fetching the entire list of things for each ‘loop’ of the recursive flow, is to instead of sending in a list of things, you send in a list of text which is the unique IDs of the ‘things’ then you have a step, like step 2, where you search for that thing via the unique ID (coming from your parameter of type text) so as to only be charged for fetching a single things for each loop of your recursive backend workflow.

Would this only be if I decided to keep the recursive? Would this approach you suggested be better or worse than using the list function?

Thank you

Jici · 2024-09-28T04:24:07.303Z

eric10:

SAWOL

Schedule api workflow on a list. Yes they are results we got from moving to recursive to sawol.