Weird/Awesome: custom state delivering real-time DB updates?

matt71 · 2024-09-25T16:24:07.534Z

Tbh I’m not sure whether this is a bug or a tip, but… I’ve been populating RGs with either live “Search for” results, or a custom state equal to those same “Search for” results. i.e., to find the right balance between real-time updates and WU.

As part of my testing I noticed the following:

On login, I set my allFoobars custom state to be “Search For” all Foobars in the DB
I set the data source of the RG to be the allFoobars custom state
I then went and changed a value for one of the Foobar items in the DB

Expected behavior → the RG would not reflect the DB update until I had reset the custom state. Observed behavior → the RG updated in real-time, and when I looked at the WU chart, there was a .04 WU Fetching data request.

This suggests that Bubble is sending incremental DB updates to the custom state as they happen (which would be awesome). But… this isn’t behavior that I expected, or that I’ve read about elsewhere. Am I hallucinating? Are others seeing this too? Feedback welcome.

@boston85719 and @ihsanzainal84 who’ve been active on similar topics.

matt71 · 2024-09-25T19:40:31.615Z

Following up with more details, in case it’s helpful / interesting:

It continues to be true that the custom state delivers real-time, incremental updates for items in the original Search For set
If a brand new item is added, then you do need to refresh the custom state in order for it to appear in the RG
Otherwise, incremental updates are delivered in real-time for existing items

In my case, I believe I’m going to pursue this custom state approach for ongoing updates (daily for existing items), and trigger a refresh if/when the server-side DB changes in size (~once a month).

boston85719 · 2024-09-26T04:05:08.476Z

matt71:

It continues to be true that the custom state delivers real-time, incremental updates for items in the original Search For set

This means then that if an existing entry is modified to say change the name, that name change is reflected in real time? And when this occurs you are seeing a 0.04 WU charge associated with only updating the search results stored in your custom state and not associated with the actual modify thing action?

matt71:

If a brand new item is added, then you do need to refresh the custom state in order for it to appear in the RG

This means that any new data entries created are not updated in real time?

matt71:

In my case, I believe I’m going to pursue this custom state approach for ongoing updates (daily for existing items), and trigger a refresh if/when the server-side DB changes in size (~once a month).

Just keep in mind your page is refreshed, and the custom state is reloaded every time a user closes the page and then opens the page again, so everyday, if a user logs in, logs out and closes browser every day, the custom state list of entries will be updated to reflect what is in the database at that time. You as a developer would not need to do anything for this to happen.

matt71 · 2024-09-26T04:29:24.259Z

@boston85719

boston85719:

This means then that if an existing entry is modified to say change the name, that name change is reflected in real time? And when this occurs you are seeing a 0.04 WU charge associated with only updating the search results stored in your custom state and not associated with the actual modify thing action?

Yup, that’s what I’m seeing. Using the name change example → name change of existing entry is updated in real-time, and 0.04 WU charge is specific to updating the search results in the custom state, not the modify thing action.

boston85719:

This means that any new data entries created are not updated in real time?

Yup, confirmed. Anything outside the scope of the original custom state assignment needs the state to be refreshed. I believe this is okay, and possibly ideal in my case because:

I have a fairly steady e-commerce inventory, where the size of DB (ie number of unique items) probably only changes about once every 1-2 months.
Meanwhile, I have a steady stream of small updates to existing items (e.g., inventory sales, holiday discounts, etc.)
My app is available as a PWA, so afaict users’ login expectations are longer vs shorter sessions. (I’m likewise checking “keep the user logged in.”)
Loading the app with “Search for…” as the RG data source is 4X more expensive in terms of WU vs custom states – for the same exact page, even with all the tricks explained elsewhere on the forum.
Anecdotally, the app is also just snappier using this custom state approach

Anyway… I’m still experimenting and exploring, but I’m trending towards custom states at the moment – especially given these new real-time update features.

boston85719 · 2024-09-26T04:37:37.994Z

matt71:

I have a fairly steady e-commerce inventory, where the size of DB (ie number of unique items) probably only changes about once every 1-2 months

Depending on the size of that list, and the amount of data you want to display in the initial search results, you may consider using an on page static JSON version of it…this is what I have been exploring for a couple of months now in order to satisfy my need to avoid being charged for WUs to retrieve lots of data fields I do not need (we are charged for each character returned). It is not an approach that is easily applied to all situations, but for me, I have a list of 2,000 static products with some 40 fields, but the search results only need 5 fields displayed. Currently the apps WU metric consumption is 95% attributed to the search of that product list. Putting together the JSON approach, I’m fairly certain at this point will remove all WU charges except the page load.

I have not finalized the approach as of yet, but from my investigation/play so far, it is going to be the way moving forward to reduce considerably WUs for fetching data in certain situations.

bonjour_17 · 2024-11-04T08:32:06.028Z

I’ve been using the “datastore in a floating group” technique for some time now, but recently I needed to have that datastore inside a reusable element so it’s accessible from different reusable elements.

What I’ve found :

create a custom state (list) inside the reusable element, and set this custom state from inside the element with my list of stuff → when accessing this CS from outside the RE it will NOT be updated real time.
create a custom workflow inside the reusable element, this workflow RETURNS a list of stuff → then trigger this workflow from outside the RE and populate a variable with the result → this WILL be updated real time.

For practical reasons I would like to rely on a custom state on this RE but I need that to be updated real time.
Any idea how to workaround this?

Thanks

boston85719 · 2024-11-04T09:37:24.225Z

I use properties for this. I can pass property values from page to reusable element and onto nested reusable elements. I have never tested to ensure the data is updated in real time, as I’ve never seen it not.

bonjour_17 · 2024-11-04T09:41:49.113Z

Great idea, cascade the data from the page to the “datastore” RE.

I fought to have that datastore autonomous but your way might be the way to go.

boston85719 · 2024-11-04T09:48:51.546Z

bonjour_17:

I fought to have that datastore autonomous but your way might be the way to go

Just check if there is a WU charge for passing it from one RE to another, as that may happen similar to how we are charged WUs when running a recursive backend wokflow that passes the parameter value as a list of things. I personally have not tested against that, but if it is charging WUs, then the same work around for recursive backend workflows would likely work, which is pass in the unique IDs as a list.

bonjour_17 · 2024-11-04T10:33:17.508Z

I just found out that if set to optional, a RE property can have a default value, and that value can be set from within the RE.

So that’s what I am doing right now.

image356×341 6.51 KB

boston85719 · 2024-11-04T10:49:52.362Z

I would test to see if each of the RE that need that value set are going to be running their own search (ie: get charged for each) or if it will function in the same way when we perform the same search on a page (ie: multiple RGs on page with same search) which only charges for one search. I’d imagine Bubble would not charge for each search in each RE if they are all on the same page, but still something to check out before developing the entire feature set around the use of the Search for Tasks.

Also, if you are needing to filter the tasks from some other element, you may just want to set that search on the RE when placed on the page, or wherever the filters are. When you are building functions in the RE with a property, the default value does not need to be set in the RE itself, and your features/functions would still have access to the ‘tasks’ data fields as the property type of ‘task’.