As of today, which option is the best to edit thousands of records in the database: recursive workflow, schedule API workflow on a list, make changes to a list of things?
I’ve seen lots of post about this topic, but since this post I have not seen much more information and wanted to know which is the current best practice.
As I have read, historically, recursive workflows were more reliable, then bubble made some improvements and now schedule API workflow on a list is the official better option.
On top of that, I have read that make changes to a list of things can be faster and cheaper, but it timeouts after 5 minutes (which can cause some records to be missed).
This makes me conclude schedule API workflow on a list of things is the best option for editing thousands of records.
This depend on how many items you need to update. If you can use make change to a list of things, this is a good option for short/medium list. For larger list, better to use schedule api on a list.
Use make a change to a list of things action. IF you have more than a few thousand use a recurring WF to loop thru the above make change to a list of things action.
If a filter can determine which records haven’t been updated then for each loop simply update a few thousand records that meet such filters and reloop if there are additional records that havent been updated.
If a filter can’t filter out the updated records then Sort all records by a unique sortable field and pass the unique sortable field of last record updated in the recurring WF so that the next loop knows which record to start from. (However unfortunately you cant rely on the records last modified date).
I would go for a recursive workflow since you need to edit thousands of records in the database and also you needn’t worry in this case about race conditions.
I wasn’t saying recursive is always the right answer. I was pushing back on the idea that schedule API on a list is needed here.
A simple recursive loop that runs “make changes to a list” in batches is likely the easiest method to make such a batch update (depending on the situation of course and assuming that a single make changes to a list is insufficient). I’d also assume that it would be cheaper than WFOL but haven’t thought about WU specifics in quite awhile and even if a loop isn’t cheaper, it wouldn’t be significantly more WUs…
At the end it comes to a case by case decision and understanding the pros and cons of each method. Also, having options is good if some refactor is needed.
