Forum Academy Marketplace Showcase Pricing Features

[New Feature] Scheduling API workflows can now be done recursively

Hi all,

We had previously restricted scheduled API workflows from scheduling themselves (or scheduling another workflow that then scheduled itself). This restriction is now removed.

Recursive scheduled workflows can be useful in situations where you are processing an unknown quantity of data that might be quite large. For instance, the following workflow will make sure every Thing in your database has been processed within 7 days:

Processing data this way is generally a better approach for large quantities of data than using the “Schedule API Workflow on a list” action. A brief comparison:

Scheduling on a list Scheduling recursively
Workflows may overlap Workflows run one at a time
No way of knowing when all workflows are done Workflow can check if there is more data to process, and do something else if there isn’t
Limited by the time / capacity it takes to search for the list and schedule all the items (max 5 minutes) Can continue indefinitely
Faster and simpler for short lists (< 50 - 100 items) More reliable for long lists
Can burn through a lot of capacity if workflows run in parallel Capacity usage is limited to one workflow at a time

A couple notes:

  • In order to avoid maxing out your app’s capacity, I recommend starting with a 5 second gap before scheduling the next item in the list. You can then shorten this gap once you see what the capacity impact is.
  • If you make a mistake and create an infinite loop that does not look like it will ever end, you can end it by manually deleting the next scheduled run in the Logs -> Scheduler tag. You can also modify or delete the API workflow itself: deleting or changing the workflow won’t remove a scheduled item from the list of scheduled tasks, but when it comes time to run that item, Bubble will use the updated workflow or do nothing if it has been deleted.
  • If things get really bad, you can pause and resume all scheduled workflows for your app using the new “Pause tasks” button:

83 Likes
A fair and honest chat about Performance
Schedule API workflow on a list - does the list have a limit?
Recurring event to create thing return from external API
My API Workflow on a list skips steps. What to do?
Data API Bulk Update?
Schedule API workflow optimization
Need help looping through stuff
How to mix two Lists in an API Workflow
Pyramide scheme with bubble
Is there any way to run a Bulk workflow but in the background from the UI?
Loop and create
Prepopulate sms: to multiple number from rg
How do I send recurring weekly emails to users?
Incomplete Schedule Workflow on a List
How to know when a workflow ends?
Assign values to a list in the API workflow
[Unsolved] API Workflows Not Consistant
API Connector - Dealing with API Rate Limits
Different prices
How to delete after x amount of time
Iterating list items using javascript to bubble
Do something automatically every 1 day
Daily check / event
Increment Date & Time by Value w/ Scheduled Workflow
Disable Workflow Timeout
Is Bubble consistently unreliable?
How To Setup This List Filter?
How can I force an action to finish before the next one starts?
Make Loops in Bubble
Schedule a workflow after "Schedule workflow on a list" has finished
Question about iteration / looping
Tracking page views
Run workflow in sequence continually until condition is met
Scheduling API Workflows Recursively for Emails
Best solution of sending mail to User list?
Spacing out scheduled workflows
Scheduling API Workflow recursively to update data on bubble app
Automatic recurring background actions on database
Log show 3 but db have 2 :/
Need Experienced Freelancer on recursive Workflow to send sms
Send email individually to list of users
Scheduling API Workflow on a List Alternatives
How to achieve Unique, Sequential Numbers Reliably?

Thank you! This is huge.

1 Like

great article! Thanks for the new options…

1 Like

@josh thank you for enabling this. Really excited to see it!

As well, I appreciate that you went into detail - especially the tips for mitigating issues and the comparison between scheduling on a list versus scheduling recursively.

4 Likes

Thanks! This is a fantastic development which will help for many use cases.

1 Like

Do I :eyes: LOOPS!?

Nice…Thanks team Bubble.

Thank you so much for this feature and the comparison! :slight_smile:

1 Like

Oh, this is fantastic! :raised_hands:

5 Likes

Great work!

So I get this right, and I am sure there are still a few of us who need to see this in a flow chart :slight_smile: - anyone chime in - can some one please give a real world practical example?

3 Likes

This is the best day of my life!!!

2 Likes

So much awesome :slight_smile:

1 Like

So Thing A can call Thing A. That’s about it. It’s helpful.

2 Likes

This is great stuff, thank you Josh + team!

It is a “Do … While” loop. Where the “While” is able to be dynamic i.e. you don’t need to know the answer before you start.

Example : Update all the users who have not been active for 7 days …

Previously you would have to generate the “list of users” in advance.

Now you can loop the API workflow while there are still users to process. Although it may be that this is still better done by “schedule on a list”.

Example : Create a thing for each month in a year.

Previously you would have to generate a list of 12 numbers or months, and do a workflow on this list.

Now you can “Do … while there are less than 12 months created”.

A more complex use case is one where you are doing some processing within the loop that might be difficult to work out in advance.

So say you want to add up invoice items until the total is > £500 and then create something.

Now you can loop while the total is < £500.

Obviously this allows you to … “Do … While black = white” so loop forever.

10 Likes

Thanks @NigelG just needed an old school explanation :smiley: Now I know what it is this is awesome!

Thank you very much for that awesome update!

Hi @josh, nice explanation.

Can you clarify, does the scheduled workflow wait for all remaining steps of the current workflow to complete, as well as its scheduled date? Is this only for recursive workflows?

Definitely one of the best updates recently!

1 Like

This is great – thanks!