Anyone seen this recently?
Looks like you can’t pass lists around anymore ? Or is that just something that is in the Log?
1 Like
pretty sure this is just a log thing
1 Like
Hi @NigelG, this just happened to me as well when trying to delete a bunch of records using a backend workflow:
It seems like the deletion process was stopped.
Any ideas on how to bulk delete around 1500 records successfully?
1 Like
That message is just saying “we ain’t gonna log 1172 items, so here’s the first 100.”
See how the world would be better if I wrote everything?
3 Likes
jajaajjajajaja awesome explanation dude. So it doesn’t means that the app timed out or anything like that, right?
I don’t believe so, but there are documented limits for the number of items you can delete in one fell swoop. Check the reference for that.
1 Like
I have a long email chain with Malcom from Bubble about this subject. Will dig it out for what he suggests.
In my very painful experience … the easiest way to “delete” a whole table is to copy a blank one from Live.
However … there is certainly an issue passing lists around and then trying to recalculate them when things are deleted.
e.g.
List of numbers from 1 - 1000.
Pass in a list of the “first” 100 and delete them…
Now another backed process runs a query to get the NEXT 100. You would hope that would start at 101, but it won’t.
Bubble looks like it has a two-stage process…
- Delete the things
- Remove the pointers/index to the things
When you run a query to get the next 100, it just looks at the “index” which is delayed in being updated. So it finds the initial 100 again, but then can’t delete them.
This is exacerbated but passing in a list of 1000 and then assuming that the 100 deleted will disappear from the list (because they don’t exist). Again, this does not happen, but this time the list that you are passing around ALWAYS stays at 1000 long, and will contain a list of things that don’t exist. What you will see is that the Unique ID is empty, but the data is there.
A few bugs around thing that I reported have been fixed, but it is still a possibility that data may be there some time after you have deleted it.
What you need to do is pass around a pointer to the current delete point. Just a number.
Note this is the same thing with database consistency (and probably why we won’t ever have a consistent database function).
Process A : Insert thing called “thisthing”.
Process B : Check if “thisthing” exists and insert it if not.
At large enough numbers you will get duplicates. Because Bubble’s locking is abstracted into the platform away from the real database. “Real” databases have ways to stop this kind of thing, or at least stop Process B until something else has stopped messing.
2 Likes
I vote for this 
1 Like
Sounds like more “big data” issues w/ Bubble. Appreciate you posting this!
I found a pretty simple workaround to delete massive amounts of things using any constraints that you’d like through Backend Workflows. This might work also if you’d like to bulk modify lists.
Here it is:
-
Create Backend workflow A.
-
In Backend workflow A, Step 1 will be a delete action for the list of items you want to delete until item #99 (apparently trouble happens when you set workflows on lists of +100 items).
- Create a new backend workflow that is a Custom Event. This custom event will re-schedule Backend Workflow A every time that it is triggered.
- Put this custom event as the Step 2 in Backend workflow A. Set this custom event to trigger Only When the count of items in the list that you want to delete is not 0 (it will trigger as a cycle for every 99 items until there is nothing else to delete in your list).
This worked for me. I hope it works for anyone else who needs it!
This is similar to what I was suggesting with List Popper. Note that with List Popper this is a little more controllable AND ALSO, your initial list could be many more items than 1000 (I don’t know if I wrote this down in any of the docs, but in my testing I’ve found that SSAs like List Popper can be sent lists of several thousand - perhaps on the order of 10K items without running out of memory).
All it does is fetch the List, which is then turned into the array of Thing objects (no values are fetched - it’s just the Thing objects and their methods). The first Thing is removed from the array and presented at the item output. The remaining array of Things is published to the remaining list output. And you just recursively schedule whatever workflow you need until the remaining items output has zero length.
(Passing the remaining items to the List Popper in your recursive workflow.)
I tried the list popper before trying this new solution but it did not worked for me.
It stopped deleting when it had deleted around 100 items, just as if I wouldn’t use the list popper.
What I did was:
Step 1 - List Popper, Search for items that I wanted to delete.
Step 2 - Delete the list of items of Step 1
Maybe I did something wrong, but if not, I think that without a custom event that creates a loop for the process of deletion, the workflow will always break when reaching around 100 items processed.
Yeah, you’re doing it wrong. List Popper is for building recursive workflows.
Oh, my bad then. I looked for “recursive workflows” but I didn’t find something like that it in the editor, so I thought that they re-named them as “backend workflows”
Could you please show me where can I find “Recursive workflows” in Bubble’s editor?
I can provide a shorter example, but in the meantime, see the video in List Popper and Friends: New SSA Plugin for Your Backend Workflow (API Workflow) Needs
1 Like
Never heard of that … what “trouble” ?
I have tried this method on very large lists. We calculated that it would take c. 3 weeks to process even on Pro plan and boosts.
Also, you end up not deleting 99 every time, as the search (as I explained above) will pick up some items that are still being marked as deleted.
Works well for smaller lists. Not large lists.
The trouble is, in my case, that it stops deleting when it reaches item #100.
If you see the “Copy” action in Bubble it says it will only work for max 100 items. They might have some limitation when handling processes of more than 100 items for other actions as well.
This worked for me, it doesn’t matter if it takes items that are still marked as deleted because it will just keep deleting the first 99 items on your search results on a loop, until there is nothing else on your search results.
The speed of deleting, in my case, was around 100 records deleted per second.