I feel like this may have been answered already, but can’t find it. Is there a limit to the number of items you can send with the Schedule an API Workflow on a List action?
5 Likes
I’m 60% sure that you can load as much as you want through it but that it has a pagination like stopping point at 50 or 100 entries so you have to submit larger lists in multiple tranches. Sure hope that’s not the case, but I believe it is. Not particularly confident either way though as I haven’t run into this yet.
1 Like
Yeah, I’ve been triggering a workflow for lists that can easily exceed 100… Sometimes it’s 1k+, and I am positive it times out and/or just can’t support that many at one time. @emmanuel or @josh, is there a hard number here?
2 Likes
There’s a hard limit of 20,000 list items retrieved in a search (which we can raise on dedicated installations.) There’s also soft limits after 7 - 10 minutes for any given workflow run, which you’ll probably hit before the 20,000 list limit (we can’t raise this one on dedicated right now; we’re working on various things to get rid of this limit, including speeding things up, and breaking up long-running workflows into chunks so we can run them persistently).
Running a “Schedule API workflow on a list” action on a search starts scheduling items one-by-one; if you hit a limit, it’ll be in the scheduling phase. (Once a workflow is scheduled, it’s pretty much guaranteed to run).
My general recommendation is if you need reliable operations on large lists of things, use search conditions to break them into batches of 100 or so. Over the next 6 months we’d like to get to the point where this just happens automatically behind the scenes and you don’t need to think about it, but there’s a lot of technical challenges around doing that.
8 Likes
How do you break up the results of a search into a list of shorter lists?
Like, I’m trying to delete +100K records and of course the action “delete a list of things” fails. So maybe I can schedule an API workflow on a list, where each item in the list is a list of 500 records? Then the API workflow would delete each shorter list of records.
1 Like
Could you do it by date created or are created dates all more or less the same?
I’m not sure how we’d get a list of lists in either case. Can you describe how it might work?
Looking at the list expression options, there isn’t even a way to get item-X-to-item-Y. The only options are
- first item
- last item
- a random item
- a specific item
- first item to item-X
- item-X to last item
So I can’t even do something like “500 random items”, let alone “item 300 to item 400”. If I could do repeats I could take the first 100 items, then delete the first 100 items, then repeat, but I can’t do that. If I could reference the other cells of a repeating group I could have one cell take the first 100 items, then the next cell could run off a list that filters out the first 100 items, taking the first 100 items of that list, and so on, but I can’t do that either.
2 Likes
When you select the ‘list to delete’ in the API workflow, do you do a search for a Thing? As a constraint, the date is there. I have them setup where I set created date < current date changed in X and date > current date changed in Y.
What I dont know for sure is if you need to create new endpoints for each list or if you can have them as a new workflow action.
Okay, but how do you do that more than once? I see how you could get one list of records that fall between two date/times. How do you get a list of lists?
So you want to delete a list of Things that is part of another Thing?
Could you do do a search for Things Thing:filtered > advanced where This Things creation date etc etc? Seems possible but havent tried though 
Is it true that the list to run a work flow on cannot have only one item in it? I am scheduling a workflow on a list that varies in size depending on when its called (its an API) and it seems when I call it and there is only one item returned in the API then the scheduled workflow (which saves what is returned in the API as things) does not work. When i change the parameters of the API to include more than one item, then the workflow does run.
Did you find a solution?
1 Like
No. As far as I know the “schedule API workflow on a list” function is still limited and unreliable.
I went and got a whole different server to do analysis and processing on large batches of data and I might replace all of the “schedule API workflow on a list” actions with calls to the other server just because I don’t fully trust this function to not make any mistakes.
What server are you using now for the analysis?
hostm.com
They’re awesome. I don’t have much experience picking hosts, but their customer service has always been impressive and I like their straight forward one-price-no-limits deal.
I’m using the API to pull data to the hostm server where I can do whatever I want with it. I looked at turnkey databases and dashboards but they all wanted a direct connection to the bubble database, which I can’t get because I’m not on a dedicated hosting plan.
I haven’t used logic on the hostm server to manage actions in bubble yet, but it should be do able.
How would you break it into sets of 100?
2 Likes
Hi, coming into this train late. I know the Bubble team is currently working at it (Bubble’s Roadmap).
Is there a good work-around for this? What is a good way to do batches of 100s?
I stubbornly wasted quite some time trying to get around this instead of asking for help. When I finally asked for help the solution was clear within minutes.
I’m writing this as a:
- note-to-self (I tend to forget),
- way of helping someone else in the future; and,
- getting some feedback from the Bubble hive-mind.
Here is the solution I came up.
tl;dr learn how to use recursive workflows.
Some context: I need to:
- create a list of Apartments within Buildings (100-300 apartments per building, up to 30 buildings), and
- add those created Apartments to the Building’s List of Apartments.
It’s a poor-man’s graph database, I guess. The problem was that I would schedule an API workflow on a list and, because of well-documented limitations of Bubble’s database, these items would either not be created at all or they would not be added to the Building’s List of Apartments.
The main problem was that it was trying to add multiple Apartments to one Building’s List of Apartments. That lead to data inconsistencies.
The solution I came up with - with the tutelage of the all-knowing 
@romanmg 
- was to create a recursive workflow for each Building. So that only 1 Apartment is added to the Building per iteration.
When the user clicks a button, it schedules an API workflow on a list; running on a list of Buildings and I pass a list with texts that will be used to create the Apartments.
- that API Workflow creates an Apartment
- then adds it to the Building
-
schedules itself again, but this time it takes that
list of texts:minus list of texts:first item(which was the item just used). - it will stop scheduling itself when that
list of texts:count is 0
This is probably child’s play for you pros, but for me it was like the first time I discovered VLOOKUP in Excel.
I hope this helps some poor soul in the future.
Thanks! 
20 Likes
I couldn’t edit the previous post, but I wanted to improve it:
list of texts:minus list of texts:item #1. Using first item could potentially remove various items.
I learned that from @keith on this post: "minus item: first item..." expected behavior?
2 Likes