I’m building a website that allows users to connect their health devices and access their data. The site includes a subscription service that automatically updates this data through a backend workflow. However, I want to optimize this by only updating records that have actually changed.
I have a data type that stores run data, including a “Last Modified” date. When a user first connects their account, all run data is downloaded and saved. While most runs remain unchanged, some may be edited, which updates their “Last Modified” date as reflected in the API.
I’m looking to set up a system that runs every X hours to check for updates and ONLY download the data where the “Last Modified” date differs from what’s stored in the Bubble database. This would help save on workload units and avoid re-downloading the entire list each time.
Any advice on how to set this up effectively? Would this be done using the filtered parameter?
I will also add the ability for users to click an “Update” button on their profile so that it can check for any changes between the last modified dates in the API call and what is saved in the bubble database, and then just only make changes to the existing runs (which is broken down into location, tags, distance, etc.).
Is there a simple way to compare the API data with the data in my bubble database and only download the ones that have changed?
This means that you will make the changes in the Bubble database first and then send those values via API to the API providers database?
Does the API return a data field of modified date that is corresponding to when data values in the API providers database has changed because that is connected to the users device?
Depends on how things are setup and what type of data is being returned from the API provider and where the changes are being made first, as in the API provider database or the Bubble database.
I will need to call the API and then see the dates of the last modified runs. If the last modified date is different than what I have saved, then this field needs to be updated in the bubble database.
Does the API return a data field of modified date that is corresponding to when data values in the API providers database has changed because that is connected to the users device?
I am not sure what you are asking, sorry.
Depends on how things are setup and what type of data is being returned from the API provider and where the changes are being made first, as in the API provider database or the Bubble database.
The changes will be seen on the API first and then I want to update this on the Bubble database.
Is it possible for webhooks?
No there is no support for webhooks as far as I can see. Is there any other way?
As you mention the “Last Modified” date differing from what is in the Bubble database, I asked if the API data actually has a field of “Last Modified” as a date…if so, just compare the date you have in Bubble versus what the date is from the API, if they are different, then it has changed in some way and likely you should redownload the data to Bubble.
Is there any reason you need to save the data in Bubble? If all the activity is coming from the device via the API, why does that data need to be saved in Bubble? You could just use the API data and show that to the user of your app.
This is exactly what I want to do but I am not sure where to start.
Is there any reason you need to save the data in Bubble? If all the activity is coming from the device via the API, why does that data need to be saved in Bubble? You could just use the API data and show that to the user of your app.
There are API limits in place so if I load the API as “data” within a repeating group, constant page refreshes and more users would mean using up my API credits with the service I am using.
There are actually multiple APIs that I connect to in order to download specific run information, sleep times, locations, etc. which I also display within graphs and charts so I need to download all the data, cross reference them together and then display it.
I have set up restrictions on the amount of data that is downloaded to Bubble depending on their subscription plan to ensure workload units are not being used unnecessairly, but all I need to do now is run the check of the Bubble database “last modified” times against the API and download those ones only. That way after the initial bulk data download, the past runs/sleeps can be updated if it has been modified by user since they last synced their data.
