Return Data From Recursive Workflow

harry7 · 2024-12-09T13:07:27.429Z

Hi everyone,
This is quite a similar issue to the one in this post here:

How to return data from last step of a recursive workflow APIs

This is another one of those Bubble dilemmas that seem so simple, but after hours of trial and error, I come up with nothing. I am trying to find gaps in sequential numbers in a list of things by running a recursive workflow. (This workflow is just a step in another API, which is called from a third API. This 3rd API needs to ultimately return the result from this recursive workflow. This adds extra layers of complexity to my dilemma.) THE OBJECT In a nutshell–and greatly simplified–what I am…

However, I need to have the workflow on the backend so the solution suggested here doesn’t work for me.

I have a recursive API workflow on the backend with a ‘Return data from API’ action to see if there’s an error when completing the workflow. During this API workflow, it calls multiple other API workflows and custom events so it is quite lengthy.

My current issue is returning the data from the recursive workflow. It currently only works if there are 2 recursions (the workflow runs twice). Any more than that and I can see with the debugger that the return fields are empty. I have been creating a db record just before the return data action and can see the same fields are getting populated correctly no matter how many times the workflow runs.

I then reference the returned data on the front end to show a warning if an error occurs during the backend workflow. To me the issue either seems to be that the step to show the warning is happening after all the recursions of the workflow can be completed (but still after the initial run has completed), OR the return data is not being populated at all if there are too many recursions.

Any help in solving this issue would be really appreciated

NOTE: I am using the APP connector to run the workflow from within the same app.

mitchbaylis · 2024-12-10T08:34:06.414Z

you have to add the return data action but also set the api workflow to return data (two places)

BUT if you are triggering other api workflows and using recursion the issue becomes much more complex since the nested workflows will process separately to the main api workflow.

you can force workflows to process in sequence using custom events but custom events have a built in anti-repeat logic so you have to break the repeat cycle with an api workflow

the best way to process a list of things is to hold it as a list throughout the steps or write it to the database (but that incurs database workload usage).

often I’ll hold the lists as I process them (process one item at a time and remove from list then repeat until empty, then run finish event). I’ll usually have a progress on the main data to update the user as to the process moving forward or erroring. if you needed super granular progress you would hold the lists or counts on the data and then count them vs the completed count.

harry7 · 2024-12-11T22:55:44.903Z

Hi @mitchbaylis, appreciate the reply. I’m not sure what you mean by having the return data in two places but I assume I have done that if it works sometimes

You are correct in assuming I’ve got other custom events that are nested within the API workflow. The actual error data which I am trying to return comes from a custom event in a custom event in a custom event (3 levels) which is triggered from the API workflow.

Here is a ss which better illustrates my recursive workflow:

image1530×710 41 KB

mitchbaylis:

if you needed super granular progress you would hold the lists or counts on the data and then count them vs the completed count.

The way I’ve currently got it set up is I send the current_position as 1, then add 1 for each loop, and a total_item_count as the total recursions that should happen. I send a list of things and then use

this thing: item #current_position

to get the correct one.

mitchbaylis · 2024-12-12T03:47:46.583Z

with the api workflow you can set the actual workflow to return data to the thing that called it (rather than the standard 200 response). If you’re doing everything in bubble then you likely don’t need to return data from the API that you’ve called.

how you have it setup sounds correct.

I’d normally send the list of things to process and then minus the items from the list each time and do the count separately (I don’t count the items left incase they don’t minus off for some reason). Counting up is the most fail-safe method so I use that.

What’s important to note about custom events is that they all wait for the entire of the custom event to fully process before the “event” is completed and the next actions process in the workflow.

So if you have custom events nested within each other then level 1 will wait for level 2, level 2 waits for level 3, level 3 waits for 4… therefore level one waits for 1,2,3,4 (all the nested custom events).

The return data from API step that you have as the last step gets evaluated each api run (each loop). So you need to make sure that if you’re returning data there that you are holding that data through all the loops so you can return it in the final step.

ie if you’re creating temp calc and need to return 10 of the created temp calcs then you need to have a list of them on the loop and add to the list each loop, then reference that list in the api return step.

you’ll also need to be mindful of how all the custom events and other data triggers/api workflows interact with the data during this process - I’ve often had unexpected issues because I’ve had data triggered workflows that are triggered during a loop that then affect the final output unexpectedly.

normally what I do to debug this is create a record for all the steps so I can precisely see the inputs/outputs in a simple text format (no need to try and sift through the bubble logs). then I can narrow down what is working and what isn’t, check for looping issues and timing issues (since the debug records are created directly after each step that runs I know when each step ran and can see if it is in the order I intended)

for instance in your screenshot - if you held the list of temp calcs on the api workflow step and that step has a condition to not run on the last step then the list would be empty for the return data to api (if you referenced it from the api step that didn’t run).

to get around that I’d nest the api step in a custom event and have the custom event return the list and just condition the api call within it. the two steps that run on the last recursion should also be in a custom event - 1 you only evaluate the condition once (less WU units) and 2 they then evaluate in sequence rather than potentially at the same time. The api workflow step of create and repeat I’d also put in a custom event.

essentially the api workflow “calculate tiers 1” would have 2 custom events in it. the complete event needs to be BEFORE the loop event (so the condition is evaluated before the loop changes it again).

in general most bubblers need to use far more custom events than they do.

harry7 · 2024-12-14T05:29:51.271Z

You are correct in that I only trigger the API workflow from within my app, however I use the app connector (not API connector), rather than scheduling the custom event.
I have tried adding a debug record step just before the return data action, which shows everything working correctly. However, the data is still not returned to the front end.
In my use case I’m not actually returning the temp calcs to the front end. I am just using them to get a final sum which is also added to the thing in the backend. In my workflow I trigger an API to try and send this data to an external app. The data I am returning is whether this API had an error or not. If yes, then there is an error air alert that shows on the front end.

mitchbaylis:

the two steps that run on the last recursion should also be in a custom event

I’m curious how this would work, as the return data from API action cant be within the custom event as far as I know?

mitchbaylis · 2024-12-14T05:37:57.402Z

you could pass the data back out of the custom event to then return it in the api workflow

I wanted to do a similar thing for syncing data to xero (creating an invoice). I needed to show if the sync had errored to the user. What I ended up doing was holding the error in a field on the relevant data and then showing an inline alert if error wasn’t empty.

This was actually a better solution than the alert toast since users could navigate away from the page but still needed to see there was an error later on. On success I just cleared the error field to close the loop.

harry7 · 2024-12-14T06:39:36.010Z

Funnily enough that is my exact situation I’m trying to create an invoice in Xero. Are u saying to have a field on the invoice in your app which then displays a warning if populated on the front end?

I currently have a Boolean ‘created in Xero’ field which also sets the status of the invoice to error. However, the users do not necessarily see the invoice once it has been created - might be something I have to rework in my app specifically.

mitchbaylis · 2024-12-14T06:50:06.985Z

yes, I added a field to the invoice thing in bubble to communicate and hold the error message. I only map to the field if the call errors, and I write “Success” if it is successful (gives the user assurance).

Since it is an internal business app I was working on I gave the user functionality to debug the invoice - error message to tell them what was wrong (client id invalid etc) and a way to retrigger the call. Also a way to see the errored invoices that needed to be manually reviewed by admin.

I also integrated it via make.com for better clarity on the api calls and logging. It’s well worth the small cost. I always start API integrations this way and leave them there during testing and MVP, then move them to bubble API connector once stable and debugged.

harry7 · 2024-12-14T08:05:41.155Z

Thanks for this mate, although it doesnt solve my initial problem I think I will just use this alternate approach as it still has the same outcome for the user.

I’ll take a look at make as well - appreciate the suggestion!