I am working on a Bubble.io project where I use a scheduled API workflow on a list to iterate through items. For each item:
A new entry is created in the database if it doesn’t already exist.
If an entry exists, it is updated instead.
After the API workflow completes iterating through the entire list, I want to display the processed data in a table. To achieve this, I plan to set a custom state to true once the workflow is fully completed.
My question is: How can I detect when the scheduled API workflow on a list has fully completed?
I’ve noticed that scheduled workflows don’t provide direct completion callbacks, so I’m not sure how to determine when all iterations have finished. Any suggestions or best practices would be greatly appreciated!
you can search this topic and find various threads on the subject. If you search with key name and a user to tag, you find all threads with that keyword the tagged user was part…if you tag me in this search you’ll find a few threads on how to accomplish this. It basically involves running an extra step on the last item in the list.
@jenil.kukadiya You could create a database table for “Workflow Runs” or something similar. When you start the workflow, you create a new “Workflow Run” of status “Working”. You could also have a “count” field on this datatype, and add 1 to this count every time the workflow runs. When the count is equal to the total times the workflow is scheduled to run, you will know its completed
I have implemented the following approach to track the progress of a scheduled API workflow on a list:
Created a table called ProcessingTracker with two fields:
ProcessedCount
TotalCount
Before calling the scheduled API workflow, I initialize ProcessedCount to 0 and set TotalCount to the number of items in the list.
In the backend workflow for each item in the list:
I insert or update entries in the target table.
I increment the ProcessedCount field in the ProcessingTracker table by 1.
However, I’m facing an issue where the entries in the target table are being inserted/updated faster than the ProcessedCount is incremented in the ProcessingTracker table. As a result, the ProcessedCount does not accurately reflect the actual progress during the workflow.
How can I address this discrepancy and ensure that the progress tracking is accurate? Any advice would be greatly appreciated!
you get that issue because bubble is processing records concurrently (multiple at a time) so when it writes +1 it writes it to quickly like you mentioned. ie it writes +1 three times to the value 1 and gets 2 instead of 1+1+1+1=4
a few ways to solve
add a delay into the api workflow to slow it down - not ideal but it’s an option
don’t use api on list but instead just api workflow and pass the list to the workflow then process the first item and then remove it - this forces bubble to process items one after the other
update a field on each item and then use a search to get the count (may still be buggy and need #1 set to 0.1 seconds or so to work correctly)
#2 is usually what I use. Just make sure you add stop logic into it incase things are missing - ie only reschedule if there are items left in the list… (because even though bubble says data is required it will still run the workflow if that required data is missing).
You also likely don’t need to store the progress on a separate table since 2 number fields are very lightweight.
You shouldn’t use a data type you need to write to on each iteration, it’s unnecessary to achieve your goal and costs 0.6 WUs for each iteration.
Better methods exist , like using Unique IDs as text to pass in the last item in list unique id and setting conditions in backend flow to verify if the current items unique id matches last item unique id parameter.
In my backend workflow, I can’t pass a list because it comes from an API response with a plugin-defined type (e.g., “Billing Run Attendance Records”). This type is visible in normal workflows but not in backend workflows. Is this because plugin types aren’t supported in backend workflows? How can I work around this to use the API response in my backend?
So, it seems I cannot use a recursive workflow because I need to pass the whole list, but in my backend workflow, the data type is not matching. Therefore, I must use “Schedule API Workflow on a List” instead.
Thank you for your response! I appreciate your help. However, I didn’t fully understand your explanation. Could you please provide more details or an example to help me better understand? I’d really appreciate it!
it’s likely a limitation in the plugin, since the plugin doesn’t exist in the backend the backend doesn’t know the data structure.
you can setup a backend workflow to receive unknown data structures using the “detect data type” option. select this option then send the data structure to it and then the backend can process the data.
you’d need to send the data as json via the api connector plugin. to do that you’d need to extract the data structure from the plugin.
an easy way to do this:
put a text on the page and the content is set to the plugins data
copy the texts json
set the backend workflow to initialize
run the api connector to the backend workflow with the copied json
add the api connector step to the plugins workflow
While the URL was in API connector with trailing /initialize I was able to initialize the backend workflow, but no data was detected it just showed as detected data {} I was able to manually modify the detected data after initialization.
But anytime that I am triggering the backend workflow via the API connector the values are empty, but the backend workflow is triggered.
Is there anything that jumps out to you that I’m doing wrong as to why the values are not being sent with the API call and received by the backend workflow?
If I setup like in the screen shot below with Parameters, the values are sent, but I really want to use JSON Body to send in the values but they don’t send.
