Tricks for queuing API workflows?

Anyone got any tips for creating a queue of API workflows that run in sequence and not simultaneously? My current use case is using AI to generate sections of a document. The generation uses the content of the previous sections to prompt the generation of the current section.

Users can generate each section one by one, and they should be able to generate multiple at a time (so if they click generate on sections A, B, and C, section A will be generated, then section B, then section C, rather than just running the API workflow as soon as it’s scheduled.

A recursive workflow only really works when the user knows how many sections they want to generate when they kick it off - they can’t come back two minutes later and add more section generations to the queue.

Interested to hear anyone’s workarounds or solutions. I guess I’m looking for a way to put an API workflow queue in a funnel so that only one can run at a time and the next one in the queue will start when the previous one finishes. I’m thinking of something like a ‘Queue’ data type that is searched for and, if found, scheduled to run when the current API workflow is complete?

Had this use-case before.

I implemented a Queue data type with workflow ID and status. Haven’t come with a better solution yet.

2 Likes

Does Bubble provide both the workflow ID and status of the workflow as a value or did you need to just capture the workflow ID and create your own status? I’m wondering if Bubble is providing some kind of status value similar to a progress percentage.

@georgecollier you can do something similar to what you are thinking and what @redvivi said. I often use the term Processor for my data type that I use to track backend workflows. I would say for this use case you can have a data field on your Queue that is a list field of the backend workflows that are ‘to run’ and another field that is ‘completed’ plus a 3rd for ‘total workflows’, which the total workflows are all workflows triggered, the ‘to run’ has only those that are remaining, and the ‘completed’ has all that are finished, so that you can compare against the ‘total workflows’ to ensure all were ‘completed’, but the ‘to run’ will allow a user to continuously add more as they wish.

2 Likes

Custom status.

1 Like

That’s not entirely true. While the recursive workflows you’re thinking of are more WU efficient because the iteration count is statically defined, you can have recursive workflows dynamically determine whether a next iteration is needed or not. For small lists, the WU difference is low and can be outweighed by the benefit of smarter recursion. This is exactly what you are talking about here:

The specific setup for dynamic recursion depends on your schema but mainly on how much you are allowing users to go berserk on queue management.
If you only want to let users add to a simple queue a few minutes later, then you don’t need anything too fancy, you don’t even need a new datatype; you could just iterate through a list in the datathing.
This would allow users to add to queue (add to the list) while the recursive workflow is iterating, as long as you also implement error-handling which reschedules the same iteration (or deletes queue) if something goes wrong. You also need a way to determine if a recursion is active on a particular datathing, so you know whether to initiate the recursive workflow or just add it to queue.

Lists are ordered and simple, but if you use them, the API workflow itself should not update the list, or you risk running into race conditions with the backend and frontend modifying the list simultaneosly. I would personally save the total iteration number on the datathing, and avoid removing completed iterations from the queue. You can also know use iteration = queue:count to determine whether the recursion has been completed. You could always reset both of these at a later date when the chances of the user being online and active are very low.

If you want a more complex queue, with status reporting/multi dependencies etc, then using a new datatype with individual entries for each queue item would be the best solution.

Edit: I was assuming you’re saving the response of the APIs to the database. But I now realise you might not be. In this case you could use Local storage to save which sections have been sent to be generated and determine from there whether a section can be sent for generation, or whether it has to wait for dependencies’ response.

I am :slight_smile: Thanks for the ideas guys, good to know I’m down the right track and there’s lots of good points I can consider when implementing. Will report back with any pros and cons once done.

2 Likes