I have a question to ask - I built a backend workflow containing a custom event that flows into a recursive workflow to pull in data from an API and save it to bubble’s DB.
I did a test run and found that out of 170 records, 10 records were not created. I tried looking in logs but it doesnt seem like bubble shows everything. Mentions that the “return is truncated to 100 items”.
Now I set my index to 1 and then my final condition to stop the workflow is when index <= count.
If I set this up incorrectly, it would still only account for 1 missing record, but what happened to the other 9? Is there any best practice steps I could implement in my backend workflow to ensure it pulls in everything?
Also another useful piece of information, is it seems to be skipping the data at the end of the of the loop.
I have found the steps of a backend workflow don’t always respect the ‘Only after (Step 1)’ and complete prior to the specified step, some sort of race condition. I’ve gotten around this by setting up a separate workflow that doesnt trigger unless the count of ite.s created by the first workflow matches that of those scheduled. Could this work for your use-case?
You’re right - so here is some added context. I need to pull in clients from an external source system but the only way to do that is buy using 3 API calls:
I trigger Schedule API Workflow to pass off parameters to a backend workflow that contains 2 API calls. First one is to pull Active Work Orders, the second is to pull Invoices. I push the startDate and endDate as parameters to both to get distinct clients from both the invoices and work orders and then merge both lists together to get 1 distinct list of clients.
Then I feed the merged list to a custom event. This custom event then takes the first clientID from the list and then passes it through another backend workflow which contains the third API call which then takes the client ID and creates the client. This API call needs to be looped because the only way to get client information is to feed it the client ID. It then pushes the remaining list back to the custom event to repeat the loop until index <= count.
