API Scheduling Contributing to 40% of Backend Workflow

jtrudeau1530 · 2024-08-26T11:20:18.834Z

Hi guys,

I need some assistance in catching some inefficiencies in my backend workflow. It seems as though just scheduling the workflow is contributing to ~40% of my WU, where I am only looping through ~800 items and using ~13,800 WU

I am attempting so save on WU by getting a search of objects when initiating the workflow, then carrying that list and adding/subtracting on its next iteration. I am new when it comes to server-sided workflows and I am unsure if this is normal usage for the task, or wildly inefficient. This runs daily and uses on average ~18,000 WUs to process ~700-1000 items in a list.

See photos below

image1016×363 19.5 KB

image1581×513 25 KB

image1582×537 26.1 KB

boston85719 · 2024-08-26T11:32:08.136Z

jtrudeau1530:

then carrying that list and adding/subtracting on its next iteration.

This is the problem…every iteration, the entire list of things is ‘fetched’ again by Bubble, which means, if you start off at 800 items, you’ve got 800 items fetched for the scheduling of first run, then on second run, you’ve got 799 items fetched, then on 3rd run you’ve got 798 items fetched and so on.

Better to on scheduling fetch the items, but have the backend workflow parameter be of type text as a list and not the ‘things’, and when you send in the fetched list of things to that text parameter, send in the Unique IDs…then add to your backend workflow a Make changes to thing and the thing to change will be ‘do a search for’ with constraint of unique ID = list item (this is the text based list parameter that is the list of unique IDs).

With that setup you do not get charged for fetching the list of things each loop and the ‘do a search for’ with constraint of unique ID is not an aggregate search and is just a single item search.

Another approach could be to use the schedule api workflow on a list…I suggest trying the API workflow on a list first to see how that affects WU consumption, and then test with first suggested approach to compare the two.

jtrudeau1530 · 2024-08-26T11:36:39.195Z

Thanks for your input! I always thought that anytime you “Do a search for…” would perform a search on all lines and just return the single item. Are you saying that you are only charged WU for the list returned? If this is the case I have a lot of changes to make

boston85719 · 2024-08-26T11:38:52.519Z

jtrudeau1530:

Are you saying that you are only charged WU for the list returned?

You are charged for the search 0.3 WU and then charged for the list returned (items returned and each character of data returned)…there used to be a list that showed the different activities but seems it has been removed from the internet…that list showed a clear difference between charges for an aggregate search and a single item search (single item search when use the unique id as constraint or when using current page thing or when using get data from URL and set the type to a thing)

jtrudeau1530 · 2024-08-26T11:39:47.818Z

Thank you so much for your help

georgecollier · 2024-08-26T11:51:00.935Z

boston85719:

when you send in the fetched list of things to that text parameter, send in the Unique IDs…then add to your backend workflow a Make changes to thing and the thing to change will be ‘do a search for’ with constraint of unique ID = list item (this is the text based list parameter that is the list of unique IDs)

This is a great workaround.

It’s crazy that an optimisation hasn’t been made here on Bubble’s end. If you take a List of Things as a parameter and schedule the next workflow with List of Things:minus first item, it does a search, returns all the data and then passes the unique IDs to the workflow. A List of Things parameter is, behind the scenes, a list of unique IDs, so I wish it’d just manipulate that list instead of retrieving them all from DB again…

boston85719 · 2024-08-26T11:53:08.516Z

georgecollier:

It’s crazy that an optimisation hasn’t been made here on Bubble’s end. If you take a List of Things as a parameter and schedule the next workflow with List of Things:minus first item, it does a search, returns all the data and then passes the unique IDs to the workflow. A List of Things parameter is, behind the scenes, a list of unique IDs, so I wish it’d just manipulate that list instead of retrieving them all from DB again…

I suggested it when I found out that this was true. @chris.williamson1996 was the first person to post about this. I contacted support and they confirmed this is what they do. I suggested they fix that since it is charging us unnecessarily to refetch a list that has already been fetched.

Zeroic · 2024-08-26T13:29:29.839Z

Another item on Bubble Boosts please in addition to the Search optimization that @brenton.strine was talking about

I think Bubble should spend some time on making users aware of this before going in for the next boost since only a subset of devs are aware of these shortcomings. In case this is picked up (and even if there are only a few things you want to fix), there won’t be any noticeable UI updates as such, but nevertheless, would be super important, that people will only appreciate once they’re made aware of it

jtrudeau1530 · 2024-08-26T14:16:53.533Z

Huge help!
Already seeing a 50% reduction in WU.

Thanks much, cheers

image1361×817 46.6 KB

system · 2024-11-04T11:20:46.952Z

This topic was automatically closed after 70 days. New replies are no longer allowed.