Parse json to database (unique ID in the JSON record)

Continuing the discussion from Parse json to database:

Hi, @ed19
Thank you for your help. Basically, you are right, but I am receiving feedback from the website via API. The problem is that they are being added continuously. Therefore, I can only rely on their unique ID. The hitch is that I can’t figure out how to skip those IDs that are already recorded in the Bubble database. Do you know how this can be implemented?

This would normally be something you’d handle at the point of making the call (though admittedly some REST APIs don’t allow even this basic level of filtration).

Scan the docs for a parameter(s) that allows for the exclusion of entries. For example, with Shopify you can use “since_id” or “created_at_max” to only receive results after your last recorded entry’s created_at or id.

Failing that, you can use a filter on the resulting payload in bubble before creating your entries, to the tune of: “id is not in: do a search for entries on X date: each item’s id”.

Couple of things I’d point out however…

  • You may be able to achieve the same desired result by changing your call type from action to data, then reading data from each user live as opposed to saving entries into your db (depends on volume and availability of parameters).
    Edit: strike this, I spliced this response with another query I was just answering. I was pre-coffee, please forgive the transgression :slight_smile:

  • Secondly, if there is a GraphQL API available, 100% use that, since filtration becomes incredibly simple at that stage.

Hi @ed19

Thank you so much. You have helped me a lot. Your method works perfectly. It’s exactly what I needed. Unfortunately, I still have little knowledge in filtering and conditions, but now I understand the principle and will use it often.

Thanks again!

PS: In case it helps someone, this is how I did it.


No problem at all.

Just to say, you’ll want to keep an eye on that conditional if your entries start piling up, that kind of search can eat into wf units if you’re not careful.

Thank you, you are absolutely right. I will keep an eye on the WF units.

1 Like