I need to do some maintenance work, i.e. modify/delete large amounts of data every month. I can do it (a) by scheduling a recursive WF or (b) i can manually use the “Bulk” feature in the Bubble app editor (Data > App Data > “Bulk” button). Both actions work fine. The benefit of (a) is that I can do this maintenance work without any manual action on my end by scheduling the recursive WF on a monthly basis. But i noticed that (b) costs a lot less WUs than (a), about 90% less.
Is there a way to schedule (b), i.e. a manual “bulk” update in the DB? Right now, the only option I see is to press the “Bulk” button manually in the app editor. To clarify, I’m referring to this button in the screenshot:
That’s because you don’t pay for rescheduling recursive backend WF.
Another option you can look at - making an API call to modify your DB entries. There are some threads here and some videos on YT about it.
1 Like
At least in terms of WU cost, using the Bulk action from the data tab is the same as scheduling a workflow on a list (although it’s significantly faster for some reason).
Scheduling a workflow on a list will always cost less WU than running a recursive workflow on the same list, but I certainly can’t see why there should be anything like a 90% difference…
But you can try it for yourself and see.
Then, if there is a significant difference, you can just schedule a monthly workflow that schedules a workflow on a list (rather than using a recursive workflow).
But… as I said, there really shouldn’t be much difference in WU cost regardless (although recursive workflows are the most costly)… so if I were you I’d investigate further to see why there’s such a big difference (perhaps you’re running duplicate instances of the workflows, or don’t have your exit conditions set correctly?)
1 Like
I will investigate my recursive WF further but I think the WU cost is due to the huge amount of things (10,000+) that need to be edited.
Quick question @adamhholmes : My understanding was that running a WF on a list of things when that list is huge (10,000+) is either not effective or nor working at all, is that right?
In theory it should work just fine (especially if you space things out enough).
But in practice, not so much… it will likely time out or hit capacity, and there’s no simple way to know when/where/why/how… so yeah, it’s probably not a good idea to run workflows on such large lists that way.
But if it works with the Bulk operation in the data tab then there’s no reason it shouldn’t be possible for it to work just as well.
I’ve got no idea why the Bulk operations from the data tab are so much faster and more reliable than running a workflow on a list (in theory they are the same thing).
I just ran a test on 500 things… to update a single field…
With the Bulk operation from the data tab it took around 12 seconds.
Running the same workflow (on the same list) using ‘Schedule API Workflow on A List’ took more than 5 minutes to get through the first 100 things, at which point I gave up. (the WU cost in both cases was identical, as expected).
Bubble say they are working on improvements on this… but who knows when we’ll see those come to light.
3 Likes
Thanks @adamhholmes
In Bubble’s manual, they actually say that “Bulk operations run client-side, so they depend on your browser window staying open and active. In other words, you need to keep the popup open for them to continue. If you are looking to modify a large amount of data, the most effective and reliable solution is to use recursive scheduled workflows instead”
So I would assume using the “make changes to a list of things” and using the “bulk” button in the app editor might actually are 2 different things?
Yeah, I’ve read that in the manual before… I’m not sure what that’s supposed to mean?
How can a backend workflow run client-side? And if it does (which it obviously doesn’t) why do we get charged for server usage?
I guess it probably just means that the scheduling of the workflows is done client-side?.. which would explain why it costs less WU than a recursive workflow… where all the scheduling actions are done on the server, and so each incur the cost of a server action (which is the single most WU expensive thing that exists in Bubble)
Anyway…
Well that’s almost certainly true… it’s definitely more reliable… but also significantly slower and more costly in WU.
Yes, those are two completely different things…
In many cases (although it depends on the complexity of what you’re doing and the amount of data involved), especially since the new pricing model, making changes to a list of things is both the fastest and least WU costly way to modify (fairly) large lists of things… (in my experience, up to around 3000 reliably).
But I doubt it would work reliably with 10,000 things, currently (for me it always maxes out around 3000-5000)
2 Likes