I am running a process which does this:
- Get Paginated list of items from Shopify .
- Save this list of items in a local list in my database (pictured below)
- When the pagination is complete, a recursive workflow is called. This workflow has one job only: Traverse the list, for every item in the list, create a new “Product” and push this item into a data field of that “Product” (as pictured below).
- When an item in the list has been processed, it is removed from the list. (pictured below)
- The workflow then calls itself again, unless it has finished processing the list.
Now the problem:
The workflow is simply too slow. It’s quite fast when the list is small (25-40 items), it makes products quickly. About 3 products in a second. However, when the list is large (100 items) , it’s extremely slow.
Some guidance on what I’m doing wrong would be extremely appreciated. I have already experimented with having a pause before a recursive call, but the problem persists.