Only add new data from API (without creating duplicate data))

[cj11]

Hi!
I have set up a workflow where with a press of a button I am calling data from an external API and (using background workflow) recording the results in my database. I originally kept getting a duplicate of every entry, so each transaction was recorded twice. I was able to stop this by adding the only when “Search for transactions: count is 0” to the background workflow.

However, each time I click the button to call the API it gives me a full dump of the data, meaning it adds a bunch of duplicates since I already have those entries in my database. I have been searching around the forum and looking at multiple Youtube tutorials covering this, but everyone seems to reference the solution I already have in place.

Does anyone have any idea what might be wrong? Thanks

[oliviercoolen]

I’m not completely sure I understand your usecase well. However what you could do is have 3 lists in your database. (Only do this if the data is not heavy)

For example you have a data type called “API Data”. This type would have 3 fields as lists. List 1 = mostRecentData, List 2 = oldData & list 3 = newData (the data you want to work with).

What you do when you call the API is first change List 2 (the oldData), to the data of List 1. Then you change List 1 to the results of your API response. Then list 3 becomes the result of step 2:filtered. And the filter would be :advanced where for example ‘this text’ is not in the result of step 1.

In other words, you compare the new response of the API call, to the previous response of the API call. Then you keep all the data of the new response, without the data that was already given with the previous response.

Hope this helps.

[cj11]

Sorry I’m surely not explaining it perfectly, and thanks for taking the time to respond!

Unfortunately I am still quite confused as to how to make that work. Basically I just want there to be no new data added if all the data is already there (in my case if there are no new transactions registered at the API provider). But the trigger I have set up now just adds a complete copy, creating duplicate data in the database.

Your suggestion sounds logical, to compare the existing data with the new data fetched through the API, find the difference (the new data) and add that to the database. But I am still lost in how to do this in Bubble. Where in the workflow would you do that?

Simple description of current setup:

1. Click button to get transactions
2. Schedule API Workflow (see screenshot)
   
   
3. Backend workflow creates a new entry (see screenshot)
   
   

[lee1]

Does the data thats coming from the API have it’s own unique reference number at all?

If it does, create a new field in bubble and save it to the item when you pull the data from the API and create a new item in bubble.

Once you’ve done this, in your back end workflow at the bottom

Do something like this.

Do a search for transactions where each items API ID doesn’t contain API ID (on bubble where you’ve saved it to check against).

Hope this makes sense.

[simon7]

Are the data duplicated already in the response from the external api? What can you see from the logs?

[cj11]

Dang! That worked, seems like “doesn’t contain” was the operator that did the trick. When I delete a recent transaction now and trigger the API with the button only that entry is re-recorded. So it looks like it works as I wanted it now.

Big thanks! Love this community

[christina.sandberg]

Thanks! This is what I have been looking for for many, many hours.