Advice on custom states vs. DB searches wrt WUs and performance?

Hey folks. I have a database of about ~300 items, and ~10 fields in each. On my home page, I show 15 different RGs using this database. Think: e-commerce store, where each RG is a category.

Afaict I have two options: 1) do individual “Search for” DB queries for each RG; or 2) load the full DB into a single custom state on page load, then just filter in each RG. #1 seems to be more responsive in terms of loading latency. However, I believe #2 will be more efficient in terms of workload units.

Any suggestions, or additional considerations that I should keep in mind?

Just keep in mind that by using #2 and storing it in a Custom State, the data will not be dynamic, since the state stores some “static” version of the data at the time you set the state. So any changes after that will not be reflected unless you update the state.

Another way to do this and avoid multiple searches is to use a RG to pre-fetch the search and then reference this search elsewhere, applying the necessary filters for each case.

Thanks @Newed. Can you say more about using an RG to pre-fetch the search? e.g., I’m not sure how/where you’d store the search results + reference those later – unless you’re still talking about custom states?

@Newed Actually, I think I figured it out! Basically, have an RG on the page that fetches the entire set of results (e.g., All Results RG). Then, in another RG on the page, set the data source to be the All Results RG, and filter those as needed.

In your view, would this approach be the best of both worlds? e.g., not having to do a bunch of different searches, while still keeping the data dynamic?

Exactly that. Some guys use a floating group (behind the page) to store things like that, others use groups on the page, anyway, it will depend on your use case and layout, but that’s basically it.

You know, you can also use custom states, there is no problem with that, as long as you are sure that you don’t need to use real-time updated data. It all depends on your use case.

I would load the data in a RG on your page and from there push it in a custom state and work from the custom state in your application. Should you for scaling purposes or pricing purposes want to offload the data to an external backend it is easy. In that case you only need to populate the RG with the external db and everything works again since the structure and data type of your custom states remain the same

Just make sure you automate the refresh and fill of the custom state.

Thanks @sem. Any recommendations on how best to automate the refresh of the custom state in your scenario? In my case, I expect I’ll want to update the underlying DB of 300 items about ~once a day.

Workflow on page load with a condition if needed

I can not really think of any reason there would be an advantage of any type to doing this. Just causes unnecessary complexity.

This is the best approach as the elements are on the page, but out of sight of user and do not impact layout of the page at all.

This has no real impact on the UX in terms of whether or not the search performs a real time update.

When it comes to WUs the real time updates does cost WUs, so if you are thinking that for a User to not see changes made to the database entries while they are viewing the page, you can use the repeating group in the floating group method, and add the operator :make static ( I have not confirmed this via testing, or reaching out to Bubble support regarding it, but my understanding is that this makes it such that the search as the datasource of the RG is not going to be incurring real time update WU costs, and in my opinion is a better approach than using custom states for that purpose)…if you want users to see real time updates then do not use the :make static operator, but be aware that Bubble seems to be charging more than they publicly showcase for real time updates.

In the post above, I showcase that even though the database entries for the search were not changed at all, Bubble charged 0.10 WUs, which is 2X more than the pricing table states it would cost, but also 100% more than what I should have incurred as those WUs should only be applied if the database entries are actually changed while a user is on the page which has the search.

If you change the api call that populates the RG you have another datatype and therefore need to change throughout your app everything to this datatype

That is not true. If you add a new parameter to an existing, already initialized API Call, the data type of the API call does not change. The only difference is that there would be a new ‘field’ available, and if you were to need that field value somewhere in the app, you would need to modify those places to use that field. However, if you originally initialized an API call without having checked the boxes for capture errors, and then you do check that box and reinitialized the API call, then yes, the API call data type becomes different because of how Bubble appends ‘body’ to every data type and field associated with the call.

I could only assume that your approach of using a custom state to hold onto the values of an API call that you first ran in a repeating group is that you have a custom State that is of type Text and your Repeating Group is also of type text for which you would be populating from the custom state, because otherwise, if you were using a custom state that was of the type of data that is the API call, well, then, if you were to change the API call to allow for capture errors, or maybe the opposite, removing the ability to capture errors, then it would change the data type and you would then need to change the data type of the custom state and the repeating group.

No matter what, there is no real reason, and I highly advise nobody to use the approach of loading the data in a RG on the page and from there push it in a custom state and work from the custom state value for populating other RGs. It is adding extra work, complexity and doesn’t serve any good purpose.

@matt71 Pls pls follow @boston85719 method (esp the highly advised part)

1 Like

Thanks for confirming what I tried to say. Will be more verbose next time. When you move from API call A to API call B you have to change all sorts of things.

Structuring and pushing data is custom states works great for this. If you have another setup please share.

Does it happen often that you change from API A to API B? Not for small hobby of mvp projects. Does happen more often for large scale projects with lots of external calls/services and db.

What do you mean by that?

Are you asking if it is often that I stop using API A and then begin to use API B, where API A is from a different provider than API B?

Or are you asking is it often that I change API A as the source and need to switch to API B as a source while the user is in their session?

Or are you asking is it often that I change API A in such a way within the API connector that it essential becomes different, like checking the box to capture errors, which bubble then appends Body to the data type and fields which materially changes the data type of the API call?

In either of those 3 possible scenarios you may be asking about, No, it is not often, and in reality, I’ve never done it.

For case 1, I would not setup an app with an API I was not 100% sure I was going to use.

For case 2, I would not use the same RG, and I’d show the RG for the appropriate source, which means I’d have an RG for API A and and RG for API B.

For case 3, I would already know that this happens, and so I focus while setting up the API call and I check the box to capture errors prior to initialization.

Thanks @code-escapee. For now I’m using the “invisible floating group approach,” whereby I load global RG on page load, then reference it elsewhere using filters.

While I only expect to update my DB of 300 items ~once per day, the cost of being out of sync is high (e.g., user paying $10 for something when the new price if $7). Thus, I’m currently optimizing for real-time updates.

Per @boston85719 's thoughts on WU accounting, I’m admittedly nervous about cost with this approach. But I guess that’ll be a champagne problem – when my app traffic takes off, and I need to be more obsessive about WU unit economics.

This has been a great conversation, so definitely welcome any more thoughts!

If it is a Single Page App make sure to put conditions on that RG to only be visible when necessary (ie: when the user is on the view that requires that data) and also, ensure that in that same conditional is where you put the datasource and leave the main datasource empty…you don’t want to incur WU costs when unnecessary.

Ok well in that case I understand your idea that custom states are not interested. As soon as more changes and more external db getting used it is a pain when not setup custom states.

The more you keep in Bubble and the less extensive you make your app the less need there is for setting data in custom states.

So it depends. As most things obviously

It is a single-page app, so @boston85719 can you say more about your feedback? I have it set up such that the global RG is only visible when users are on that relevant view. However, I’m not sure what you mean by this:

“ensure that in the same conditional is where you put the data source and leave the main data source empty.”

Right now I have the “children” RGs referencing the “parent” global RG via filters. How does that track with leaving the main data source empty? Thanks!

A’ight. I believe I’ve figured it out. I’m now making the data source conditional on the user being on the relevant view (in addition to the RG visibility).

My only remaining question, @boston85719 is whether I should also make the “children” RGs visibility and data source conditional as well? Or, since they’re just filters, it doesn’t impact any WU calcs?