I was also wondering if I could find a way to make the data loading faster in my Bubble.io app. I watched a tutorial from George where he explained how workload units work in Bubble.io. After watching it, I got an idea about how I can improve performance and reduce unnecessary workload usage.
In most dashboard type websites, we have an overview screen where we show numbers like total users, verified users, leads, and other data. Every Bubble.io SaaS app has its own metrics. But the problem is that many of us use âDo a search forâ again and again, even when we are fetching data from the same table.
This is not a good approach, especially if you care about Bubble.io performance optimization.
What we can do instead is create a hidden popup on the page. Inside that popup, use a repeating group and load all the data using one âDo a search forâ. For example, in my case, I am searching Leads. They all come from the same table, only the status is different.
So instead of using âDo a search forâ multiple times, I load all the leads once inside the repeating group in the popup.
Now, since all the leads are already loaded from the database, I simply reference that repeating groupâs list in other elements and use :filtered to filter the data based on status like âWonâ, âLostâ, etc.
This way, we reduce multiple searches, improve performance, and optimize workload units. This is a small but powerful trick every Bubble.io developer and no code developer should know.
But also iâm pretty sure that if a search is identical to another and the search has been completed at least once, that data should be on the client.
This idea gets thrown around a lot⌠and what youâve described above is certainly useful from a development point of view⌠(I do it myself often, and generally recommend it).
But if youâre talking about saving WU then itâs not nearly as black and white as this idea implies, and in many case this actually increases the WU (or at least, it doesnât decrease it).
Keep in mind that data is ONLY loaded by Bubble when the element that contains it is visible OR a visible element refers to it. And also that data is cached, so searches are NOT repeated if they are for the same data.
So in real terms, using a Popup to store data as lists will ofter result in MORE searches (actual database requests) - not less.
As a simple (and easily reproducible) example for testingâŚ
Say you have 30 leads in your database, with 3 different statuses (say, New, Won, Lost).
If you have a RG with ALL Leads in a Popup - NO Data will be loaded until a visible element refers to it..
And, even then, ONLY the data thatâs referred to gets loaded - not all of it.
So if you have a hidden RG in a popup searching for ALL Leads, as I said, no data will actually be loaded on page load if nothing visible refers to it.
Then you have another, visible, RG that loads the original RGs Leads, filtered by Status = New..
Well.. only the Leads with Status New will be loaded.
Then you change the Filter Status to Won - and the Leads with Won status will then be loaded.
And then you change the filter status to Lost, and then the Lost Leads will be loaded.
So there were 3 separate searches made (costing 0.3 WU each plus the data)
Which is NO less than if you just put the search, including the Status constraint, directly on the main RG.
So there was NO saving in WU, nor in data efficiency by having a hidden RG in a popup with all the Leads, compared to having the Search directly on the visible RG.
If you really want to save WU then the RG you load ALL the Leads into MUST be visible on page load - it cannot be in a âhiddenâ popup, as it will NOT be visible, and therefore the data will NOT be loaded until the filters are changed.
But if you make the RG visible (and set to âload all items immediatelyâ) then only a single search will be performed (costing 0.3 WU instead of 0.9, plus the same data cost as before - so perhaps 0.4 WU overall instead of 1.0 WU - so less than half).
But of course, thatâs not applicable anyway with large amounts of data.
So generally, using a hidden popup to hold data, whilst it definitely makes sense from a dev perspective (to keep things organised, and prevent using the same expressions over and over again) it very rarely makes things more efficient (often the reverse is true, as more searches will usually be done).
If youâre trying to save on WU by reducing the number of database requests then the RGs you use to store the full data set must be visible.
But then, that will only increase WU efficiency in cases where ALL the data gets viewed by the User (which is probably only a fraction of the time).
So in most cases, itâs not beneficial anyway, from a WU perspective.
The main advantage of doing that is to improve perceived performance for the User (which comes at the cost of more WU).
Itâs a common method to improve user experience by pre-loading data, does not save you WUâŚ
Quick tip: If you donât need real-time updates to a Search result, store the results in a state. Any Searches kept as a data-source will incur âRealtime Searchâ WU Costs.
This is pretty much the average ratio of Fetching Data I have as a beseline in the apps I build. If you donât need real-time updates to a search, store the Search in a state instead of as a data source.
When I use this kind of prefetching, I use an Expression element with Publish data = ON placed on the page that holds all the reusables, e.g. /app.
Then, when the expression publishes the data, I have a workflow that passes it to custom states in the reusables, for example to /app/leads. There, I can work wwith it as needed e.g. filter it by status, for instance.
If your goal is to make data load faster, there is only one true way and it is a case by case dependent situation as to how you compare âfaster loadâ.
If you use a Hybrid Data Structure approach and incorporate API Objects into your database structure, you can achieve faster data loading. This is true on an apples to apples comparison of something like
âif I need to load at once 100 things, loading 100 custom data type entries is slower (and more WU consumed) than loading a single custom data type entry with a list of 100 api objectsâ. â This is faster loading and it is drastically less WU for the same data to get onto the device. The only caveat is that you are loading the 100 things.
âif I am using pagination for custom data types, loading the first page of custom data type entries is faster and less WUs than loading a single custom data type entry with a list of api objects, but as soon as I paginate past the first page of custom data type entries, it is faster to load and less WUs to have just loaded one custom data type entry with all api objects in a listâ â you can see video demonstration of this Paginate Data Jedi | Loom
Do not, use a floating group set to float beneath the pageâŚit would be great if people stop spreading bad practice of use of a popup since they are not visible and do not always load stuff, but the floating group float beneath the page is visible, but just not seen by user, and so does load stuff.
You do reduce multiple searches, that is true, however, you do not necessarily improve performance as you are filtering client side compared to server side, so dependent on size of list, performance may be slower, and you do not optimize for workload units necessarily, and that is completely dependent on the size of the lists fetched, the number of times fetched.
Only true way to get faster data load and lower WU consumption, is through a Hybrid Data Structure approach using api objects.
This is why a floating group set to float beneath the page should be used if the approach is to have a benefit. Hidden popups do cause the issues explained in the above quote, but if that same RG is in a floating group set to float beneath the page, the RG data is loaded because the floating group is visible, but just not seen by user.
@shabbirfaisal632 if you really want to explore how to achieve faster data load and lower WU consumption, explore a Hybrid Data Structure approach that uses API ObjectsâŚit actually doesnât just lower WUs for searches, it also lowers it for creation/modification of data when dealing in bulk (ie: 2 or more things). Any other approaches out there do not provide both faster data loading and lower workload unit consumption.
If itâs in a popup, and not visible, then it doesnât âpre-loadâ anything (data isnât actually loaded until a visible element needs it).
As I pointed out, if you want to âpre-loadâ data to improve user-experience (which you often should), then the RG needs to be on the page (and visible), NOT in a popup.
Or use a custom state set on page load, as you suggest.
You do NOT reduce the number of searches by doing this (as I explained, in many cases you increase them).
It also does NOT result in client-side filtering (that was the point I was making) - using data in a hidden popup is often no different from using individual datasource on the related Elements from the perspective of when the data is loaded (i.e. in multiple database requests, as and when itâs needed - NOT all loaded in a single database request and then filtered client-side - that only happens IF the RG is VISIBLE and set to Load ALL Items Immediately).
Also, in many reasonable use-cases, client-side filtering is BETTER for performance than making repeated database requests and, as I also pointed out in my previous post, is the only real advantage of âpre-loadingâ data by, for example, loading it into a RG thatâs visible on the page (i.e. in a floating group).
Yeah⌠a floating group works great for this, but a regular group works just as well (with 0 dimensions).
To me, the main benefit of using a floating group is just where it appears in the editor, on the element tree (i.e in the Overlay section) which makes it easier to find and keeps it separate from the main page content.
I still think Bubble should have a separate section on the Element Tree - or better still, a completely separate tab in the editor for page data/variables (including custom objects/arrays) so they can all be managed in one place.
And just to clarify - Iâm not saying thereâs anything wrong with using a Popup to store data variables (there isnât, and I do it regularly)âŚ
Itâs just that doing so does NOT do what many of the people who do it think that it does (the same can probably be said of using a Floating Group or regular group for the same purpose in most cases).
I wrote from a perspective that a user would use the appropriate container type for this approach to have benefitsâŚthe difference between a hidden a popup and a floating group set beneath the page, is the difference between the RG contained being visible or not, so whether or not the data is loaded or not. If the RG is in the floating group, data is loaded and then any references use that instead of searching DB.
Between floating group and a group, yes, maybe that is main benefit, but between floating group and popup, the benefit is getting the desired behavior for searches and client side filtering.
Are you implying that putting an RG placed into a floating group that is set to float beneath the page, will function the same exact way as an RG placed into a hidden popup?
That drawing misses a point of whether or not the element is visible or not, which is the difference between using a hidden popup or a floating group set to float beneath the page.
This affects the decision of whether any data needs to be retrieved at all. My flow diagram shows how Bubble executes the decision to return the data once it knows that data for a given constraint needs to be retrieved
Iâm saying it will only improve performance (i.e. perceived UX speed) when itâs set to âShow All Items Immediately. And that is only something you can viably do with small(ish) data sets. (a few hundred items max).
AndâŚ
It will only cut down WU when the User ends up viewing more than the data thatâs initially displayed (which, depending on the app, and its users, will probably rarely be the case). The majority of users will probably never change the filters, or whatever it is that loads the subset of the stored data - so it actually probably ends up costing more WU on most cases.
So, in most cases, it probably doesnât do either of the things that people would assume are the benefits of doing it (of course that depends on why and how youâre doing it - in some use cases BOTh of the above can be true, or certainly at least one of them).
The same is true for RG Infinite scroll with automatic batch loading⌠it costs MORE in WU to load ALL of the data in small Batches than it would to load it all at once (or even in batches of 400, which is what Bubble does with larger data request), as there are more searches performed - but in most cases users will NOT scroll through ALL the data, so itâs a net gain. (it costs more to do it, but it will happen less often).
So thereâs more to the equation than just the cost of loading data in a particular way.
Okay, so I think we are both saying the same thing about the concept of it than, which is the use of a hidden popup doesnât actually allow the concept to work, but a floating group set beneath the page or a group set to 0 by 0 do. Iâm not really trying to look so much at the theory of why somebody would or wouldnât want to use the concept. There are times a user may have more than one RG on the page referencing the same data type but different filters and so the concept would be applicable for WU savings only if the concept is applied properly with a floating group or perhaps group.
By point, I meant the flaw in the concept as expressed by the OP based on a video they watched that gave them the idea to use a hidden popup instead of a floating group set to float beneath the page. The concept of WU savings doesnât work if the the reference RG is in a hidden popup.
So the diagram, as related to the point of the thread was missing the piece that Bubble doesnât know that a data from an RG in a hidden popup is needed to be retrieved until the popup is made visible or the elements visible on the page reference it, and as pointed out by @adamhholmes that would actually result in the searches being performed, causing the concept to fail at itâs intended purpose of saving WUs.
