Struggle with recursive backend workflow & api-call: missing data & too many scheduled worklflows

Need help Database
1

Hi there,

I want to create a dashboard which displays news (techblogs, science publications and funding events) and refreshes them every day.
I retrieve the news (data) from an external API (JSON-Format) and save them to my database via a backend worklflow.
I make 3 different API-calls to display the 3 different types of news.
In my API-Call, I preset the row count (which corresponds to the amount of articles I want to retrieve from this specific type of news).
It looks like this:

image
image909×94 7.32 KB

I’ve set this property for every API-Call, but when start my backend workflow, it only works for one of my three news types.
The other two newstypes only produce 4 instead of 7 articles or no article at all. So how can it be that this works so unreliably?

Also, in my scheduler, I noticed that it produces way too many workflows, although I’ve hitted the inital button to start the workflow just once?

image
image1792×391 58 KB

I start my recursive backend workflow via hitting a button in my frontend:

image
image988×316 8.96 KB

Then I proceed in my backend with the 3 different workflows. They all delete the old data, create a new entry and recall themselves 24 hours later.

image
image1092×332 15 KB

Any help is really appreciated! :slight_smile:

2

It looks like you’re running a recursive workflow on a list… which is almost certainly why you’re seeing too many workflows…

Without knowing more about what you’re doing, it’s difficult to say what’s going on here… but why are you rerunning the workflow recursively on a list? And what list are you running it on? (Scheduling a workflow to run on a list means the workflow will run multiple times, once for each item in the list)…

3

My data from every API-Call looks something like this (with more entries, of course):

{“TotalHits”:9048,“Documents”:[{“Title”:“Example Title”, “Date”:“2022-09-26T18:55:08.8350000Z”}, {“Title”:“Example Title2”, “Date”:“2022-08-26T18:55:08.8350000Z”}]}

I need the Title and Date for every Document (Article) of the JSON-Data.
I make 3 API-Calls so I get 3 different JSON-Files formatted like this.

How can I

  • save data like this efficiently to my database (without using the run on a list feature?)
  • redo the same workflow every 24 hours
  • be able to use the saved data seperated by news type (seperated by JSON-File) in repeating groups in my frontend ?

Thank you for your help!