Recursive Workflows Stopping Unexpectedly

[matthew.s.levine1]

Hi All –

I’m running a recursive workflow that moves through a list, and if certain conditions are met, creates a new thing. I currently have a 3 second delay between each iteration.

I’ve noticed that ~90% of the time, the workflow behaves “as-expected”, and processes all of the items in the list. However, the remaining ~10% of the time it will stop running at a seemingly random position in the list. I’ve checked the logs, and they don’t indicate anything at all (such as an error message or indication that a condition was not met). Instead, the records just stop.

What is the likely explanation as to why this is happening? Based on a forum search I’ve surmised there is some likelihood that my app is hitting max capacity, but I’m not so sure, because (A) my capacity graph never goes above 20%, and (B) the Bubble documentation seems to indicate that approaching / hitting max capacity would only slow your app down, not stop it dead in its tracks.

I appreciate any help in advance – this is causing me a bunch of headaches!

[richard10]

I can tell you from my own experience that 3 seconds is a bit fast, especially if you’re doing some heavy lifting . Try knocking it on a few more seconds and see if you get more reliability.

Are you able to identify all your records in one do a search for (plus merges) that you’d need to add?

If you can do one search, less often, and pass your results list to another creator cursor, it will be more reliable.

[cmarchan]

Hello @matthew.s.levine1 welcome to the community!

@richard10 suggestions are right on the money. Try to avoid doing searches when scheduling and have them ready as lists when sending them to the backend. Inside the workflow avoid searches in conditions as much as possible. Break into parts long flows of 15+ actions. Use recursive where possible. Slower but more reliable than scheduling a list.

[matthew.s.levine1]

@richard10 @cmarchan thanks for the tips! One more question on how the capacity and delay play into this… after I ran the workflow twice at once, and it failed the second time I tried a test to see if it was due to capacity issues. To run the test, I triggered it 10x in rapid succession (expecting multiple failures). Instead, all 10 ran to completion, and no slower than they normally would despite my capacity spiking to 100%. What should I make of this?

[richard10]

It’s blind luck, honestly. You got unlucky first time, and lucky the second. To do that much heavy lifting in one (cursor) flow and get 20-30% of capacity, you must be searching a lot.

I structure my cursors like this

Parameters minimum as needed plus a final optional parameter for the list of things you work on.

First action if the list parameter counts zero, then call self with list of results of my one big search

Second to nth actions to work on mylist:first item

Final action if list length >=2 call self, list:minus item:first item.

This way it comes in, searches exactly once, then just churns through like a meat slicer slicing one slice off the top of the list until it finishes.

I have never personally avoided searching in conditions, but if @cmarchan says avoid it, then avoid. What’s your experience of this, @cmarchan ?

Richard

[cmarchan]

I try to have searches pre loaded as much as possible

On the page and when sending them to backend flows as well (when possible)

[richard10]

Makes a lot of sense if the lifting has been done once already on the page! Most of mine hasn’t, there’s a lot of asynchronous data processing and a big do a search in every iteration is the fastest way to crash everything.

We have a large number of records and if you want the page to look performant you’ve just got to fire the actions off quickly and crunch the numbers later or the users get all antsy.

[cmarchan]

Place RGs in a hidden popup

Label each in a way that you can use them as data like …

data - tasks
data - projects
data - milestones
etc etc

Pop ups are loaded no matter what to the page. Those searches are being performed as the user is looking at the page, moving his mouse, scrolling up and down, … and your lists are ready!

[leonhardthibald]

My recursive workflow just stops at 100 iterations. Anyone know why this happens?

Would appreciate any tips.