Parse json to database (unique ID in the JSON record)

zzsnowballzz · 2024-10-22T06:55:59.019Z

Continuing the discussion from Parse json to database:

Hi, @ed19
Thank you for your help. Basically, you are right, but I am receiving feedback from the website via API. The problem is that they are being added continuously. Therefore, I can only rely on their unique ID. The hitch is that I can’t figure out how to skip those IDs that are already recorded in the Bubble database. Do you know how this can be implemented?

image656×987 33 KB

ed19 · 2024-10-22T08:50:48.692Z

This would normally be something you’d handle at the point of making the call (though admittedly some REST APIs don’t allow even this basic level of filtration).

Scan the docs for a parameter(s) that allows for the exclusion of entries. For example, with Shopify you can use “since_id” or “created_at_max” to only receive results after your last recorded entry’s created_at or id.

Failing that, you can use a filter on the resulting payload in bubble before creating your entries, to the tune of: “id is not in: do a search for entries on X date: each item’s id”.

Couple of things I’d point out however…

You may be able to achieve the same desired result by changing your call type from action to data, then reading data from each user live as opposed to saving entries into your db (depends on volume and availability of parameters).
Edit: strike this, I spliced this response with another query I was just answering. I was pre-coffee, please forgive the transgression
Secondly, if there is a GraphQL API available, 100% use that, since filtration becomes incredibly simple at that stage.

zzsnowballzz · 2024-10-22T19:32:24.288Z

Hi @ed19

ed19:

Failing that, you can use a filter on the resulting payload in bubble before creating your entries, to the tune of: “id is not in: do a search for entries on X date: each item’s id”.

Thank you so much. You have helped me a lot. Your method works perfectly. It’s exactly what I needed. Unfortunately, I still have little knowledge in filtering and conditions, but now I understand the principle and will use it often.

Thanks again!

PS: In case it helps someone, this is how I did it.

2024-10-22_22-28-50427×377 8.42 KB

2024-10-22_22-28-44475×403 8.93 KB

ed19 · 2024-10-22T21:06:31.401Z

No problem at all.

Just to say, you’ll want to keep an eye on that conditional if your entries start piling up, that kind of search can eat into wf units if you’re not careful.

zzsnowballzz · 2024-10-23T11:05:42.116Z

Thank you, you are absolutely right. I will keep an eye on the WF units.

system · 2024-12-31T06:56:57.212Z

This topic was automatically closed after 70 days. New replies are no longer allowed.