On initial onboarding of a user, I’ve got to fetch a batch of data from an external API. Based on some forum recommendations, since I don’t know how many of the items they have, it grabs the data in few steps, stores in a placeholder thing (result of the GET is saved to a field as “add items to a list”) and then runs a workflow to break apart the data to create the individual things I want. This is a bit of a long-running process on the front end (maybe 2 mins or so to process ~200 items), so I wanted to move it to the back-end and have the user notified when it’s complete.
The workflow worked just fine on the front-end other than being slow, but moving the workflows to the back-end now is running even more slowly and not making it through all the items. The initial “grab” action works, but the “schedule an API workflow on a list” seems to be taking forever and seems to be hanging up and not completing. The same process that took 2 mins on the front end never finishes and ~60 of the 200+ items is getting created after waiting for 30+ minutes.
So to recap, what I had before:
- Step 1 (frontend)- Create Thing (2 fields, 1 for the user, and 2nd is a “list” field which has the results of API GET)
- Step 2 (backend) - Schedule API Workflow on a List (running on the result of Step 1)
- Step 3 (frontend) - Step 1 again with different settings if 200 items created out of Step 2.
- Step 1 (frontend) - Schedule API Workflow
- Step 2 (backend) - Create Thing
- Step 3 (backend) - Schedule API Workflow on a List (result of step 2)
- Step 4 (backend) - Step 2 repeat if 200 items on Step 3…
Is there a better way to handle this? I do need individual things in order to do some of the operations needed, so can’t leave them in the initially created item.
Any help would be appreciated!