How do I send data via an API in batches

My application uses two different databases for different purposes. I’m trying to send data from one to another. When I send it all at once using “Schedule API Workflow on a List”, it usually times out and is more prone to errors. So my manual workaround has been manually create a batch by creating the list with “items until 50”. After that finishes, I literally click the button again to trigger the workflow with the next batch, With over 1,000 records to transfer, I’d rather not. Is there a more efficient way to automate this?

The manual batches allows me to reset the connection with my other database so it doesnt timeout. I thought about setting up a recursive workflow on multiple lists, but my assumption is that would just add on the next list to the schedule instead of closing the API connection and starting a new one. Is anyone able to confirm whether or not a recursive workflow on a list would keep the API connection open? If it does, any other ideas on how I can create batches to send automatically?

1 Like

Instead of scheduling it why not try "Add a “Do Every X Seconds” Action
I mean
Drag and drop the “Do Every 5 Seconds” action onto the workflow canvas.
Set the time interval (e.g., every 3000 seconds) for the workflow to run
Here you can add your action what you are doing manually currently.

Not sure if this will solve your problem, but you can try.

thanks

I dont even see that action…but regardless I dont think that would solve my problem because I don’t know how long it will take the finish the action. I could time the first one and use that…but it doesnt feel like the right move.