Just wanted to share a fix or a workaround for complex data structures when using @ZeroqodeSupport Fuzzy Search plugin. Thank you Zeroqode for your help.
If your Fuzzy is consuming a lot of workload units, or it is not working with complex searches, try this. I’ll share my example.
Summary: instead of “Do a Search for… Things”, create a custom state “Pre-Loaded… Things” and use it as data source.
Step by step:
Create a custom state “Pre-Loaded Golf Drills”, in my case this was for data type “Golf Drills”
In workflows, on page load, set state to “Do a search for Golf Drills”
Then in your repeating group data source, don’t use “Do a search for Golf Drills”, instead, use custom state “Pre-Loaded Golf Drills”
Then, in Fuzzy’s data source, also don’t use “Do a search for Golf Drills”, instead, use the same custom state “Pre-Loaded Golf Drills”
This saves a whole lot of workload units because “Golf Drills” are loaded only once on page load, and each time the Fuzzy search is triggered it is searching for the pre-loaded custom state and not the data in the servers. In other words, by doing so, Fuzzy’s consumption technically consumes 0 (zero) workload units.
First screenshot - set custom state to search for all drills
Not sure this is accurate - its my understanding that Fuzzy Search will load all the data on initialization, and then essentially store that in memory, so that any search utilizing Fuzzy Search will filter based on that data.
In both cases, if you reload the page, it will perform the search again to populate either the fuzzy search temporary memory, or your custom state.
So I don’t think this will save WU’s (unless you can show me otherwise via network tab in terms of whats been fetched).
Since it’s now referencing a state instead of a Search, the plugin won’t make any calls. It will work with what is in the state.
This is just the nature of how data sources work in Bubble. Any and all Search set as element data sources will run their queries when either the element is visible or the element’s data source is referenced. Hence why privacy rules are important as they prevent unnecessary values from being returned client side. Breaking datatypes into parts also help tone down the WU costs.
Use Search in data sources when you explicitly need realtime updates to a Search result. Using WFs to store Search results in states is actually the recommended practice. For some reason a lot of new users don’t know this…so they end up complaining about WU costs.
Note that while Search can be controlled, realtime updates to individual records (A Thing) are not.
But won’t storing full results in a custom state be really expensive and may even break the browser if the number of elements is high? Or is it like Bubble will apply some intelligence and get only some elements?
I don’t know about the inner workings of the mentioned plugin, but one thing is for sure: if you store an unconstrained search in a custom state and the plugin works with that in the client-side you will be sure about the total WU spent, and it will be the one of search itself.
There are drawbacks in using this strategy of course. You are indeed downloading ALL records, which can consume a lot of WU, potentially even more than those spent by the plugin working without this strategy, bloating the browser in the process.
@karaulal maybe you can share your experience on what happens if large set of data is used as the source. Maybe you can benchmark on 1K, 10K, 100K, 1M, 10M items?
Large data sizes will kill performance in any browser-based app. It is even more problematic with Bubble since we cannot save data into local storage directly, it still gets loaded into browser memory first, even when using local storage Bubble plugins. Unfortunately there is no “Bubble Intelligence™️” to stream large chunks of incoming data.
This is why the Fuzzy Search plugin is never a good match with large datasets. It’s better to run a search WF then to deal with WU costs and performance degradation trying to integrate fuzzy search with any Bubble plugin in this scenario.
This is true, that the WU consumed by Fuzzy search performing it’s matching is zero, but there is still the need to download all the items from the database by performing an actual search and then adding it to a custom state.
If that is what you want, there are other methods, besides adding it to a custom state, that will achieve the same result for the ability of searching the previously fetched data entries within the fuzzy search element via it’s matching function.
One thing to keep in mind as others have pointed out that there may be some slow down of the application for the user if you are loading lots of data. If a user is on desktop, you’d like to keep the per tab data under 2GB, for laptops under 500MB and smartphones under 300MB. What that might translate to if you have a data type with some 15 text fields, you could assume each data entry is around 2KB, so could load around 150,000 entries on an iphone browser tab to make it sluggish or crash, so to be more conservative, just assume you have your Golf Drills with some images and text fields etc., so maybe total per entry of 10KB of data, and you want to keep an iphone browser tab snappy, that would mean do not load more than a total of 100MB at once, so no more than 10,000 entries at once.
If you really care about saving WUs you should check out my paid Plugin Data Jedi as it makes using what I call a Hybrid Data Structuring approach super simple. What you could do with that approach is load 10,000 entries (or more if you prefer, and less than that obviously) for the WU cost of just one single item returned (remember bubble charges WUs for fetching data per data entry returned at a rate of 0.015WU per item)…so this saves 95% or more on WU consumption for data fetching.
The plugin has multiple actions/elements that make building scalable bubble apps easier and better, including the searchbox element which lets you search and match across multiple field values at the same time…I might add an element for the fuzzy search (it is just about capturing misspellings) plus the multi select element as well.
What I do is drop a hidden popup onto the page and place a repeating group inside it. The data source of that RG is whatever search I want to run. Since the popup never shows, size doesn’t matter I usually just make the RG something like 5×5.
Then anywhere else on the page, instead of running the search again, I just reference RG Search’s List and filter it when you need to. That way the query only runs once and the whole page pulls from the same cached result.
No, it doesn’t. Custom states are static lists of data you set until you set it again. This is why others have pointed out it’s the same as using workflow to display data, both methods create static lists…simplest way to get static list is do a search followed by operator of :static or I think it’s :make static.
I didn’t have a question. Was just correcting the statement
that was made in response to
as
is not really the same as updating live. Updating live means that the user doesn’t need to do anything in order to have the most up to date data set in the browser (or the fuzzy search plugin), since another user of the application on another device could save or make changes to the data of the application while one other user is on another device using the fuzzy search with a custom state list as it’s data source.
The issue here is using Fuzzy Search plugins that tap into your Bubble app’s DB.