Daisy Chain Filtering and Search Constraints Best Practices

Questions
21

Anytime you use the :filtered operator, all the filter constraints within that are done client side, not server side

As far as my understanding is based on what I’ve been taught by Bubble is that using the :filtered operator, regardless of the type of filter, is happening client side, not server side.

I believe you are referring to the checkbox element from Bubble

Screen Shot 2024-01-20 at 11.28.22 PM

Nobody should be using those in my opinion, nor should anybody be using radio buttons. I started using Bubble March 2018, I think that was also the last time I used either of them. In all of my apps, I build my own checkboxes for filters by using a repeating group with the filter options as the datasource, then add into the RG cell an icon (checkbox, or radio whichever the design dictates). As I can remember, the main reason I did this was it was much easier to manage the selections of the user.

You do not actually, but maybe if you are using the checkbox element type, I’m not sure. But, in my apps, the only time I use a conditional data source for the most part on my RG is when I am including a function to allow a user to select from a searchbox element a value (usually user) and then the RG will display only the selected value, so my conditional datasource is the searchbox value…other than that, I rarely require a conditional datasource for the purposes of filtering.

That things needs an overhaul…hopefully the AI Bubble is building will fix it :wink:

Yes, true. What I’ve done in the past to overcome that is to set somewhere all the values, so that it would be similar to all filter choices pre-selected, which was overkill and very convoluted. I’ve recently begun using the 'intersect' count is 'selected values count'which makes it so that when the selected value count is 0 the advanced filter constraint is effectively ignored because it is empty…this is not requiring a conditional, it is in the main advanced filter as part of the dynamic expression for the constraint itself.

Just watched the video, and no offense to Andrew or Gaby who pioneered this approach in 2017, but my stomach turned over when I saw 37 custom workflow triggers. It is not so much the number of 37 that, but the use of custom workflow triggers itself. Custom workflow triggers are not copied and pasted over when you copy and paste with workflows elements…but if you copy and paste a RG with a search that has what I believe is the proper approach to filtering (ie: constraints on the Do a Search with Ignore empty constraints), then that gets pasted over. Plus, how easy is it to get in there and Remove a filter? Seems like from the video that would be a pretty daunting task to find and manage properly, because if you are like me and naming conventions are important, you’d need to rename all of the subsequent custom workflow triggers that follow the single filter you wish to remove, plus the time it takes to find it - of course you don’t need to update anymore than one custom workflow trigger to keep the chain of events occurring, but yikes, what a mess that is.

@artemzheg I understand you are not advocating for it’s use and are playing devils advocate with some of the hypothetical examples you provided, which is much appreciated, because we still have yet to hear from anybody who actually uses it to describe the benefits of daisy chain over just using search constraints.

I just hope this approach is not picked up by too many newbies, as it just seems to get people off to a really bad start in terms of understanding how to interact with data within Bubble.

1 Like
22

Only advanced filters or when used on option sets. Bubble treats :filtered like a search whenever possible. I thought this a while ago too but I was corrected

1 Like
23

I’ve used to take on faith that as soon as you apply :filter operator - everything is filtered client-side. And this was the reason:

An underlying principle is that if a filter (or sort) can be done “on the database”, it will be faster than a filter (or sort) that Bubble has to do after retrieving an initial set of data from the database. Which filters are done on the database vs. not? Filters which show up in the Search palette (the additional sidebar which slides out when you click “Do a search for”) are done on the database and are thus are generally fast. Filters which are applied with :filter are generally “advanced” filters that are generally slower.
[bubble manual]

But while playing with WU consumption in one of my apps I’ve noticed zero difference between using basic [=fields of the data type] constraints in Do a search expression and in :filtered operator. I’ve dived into Chrome DevTools and discovered that :filtered operator returns filtered items from the DB (again, if I was using basic constraints).

Not wanna copy the whole thread here, so you can check some info I got from Bubble support about client/server filtering here:

May be one day I’ll have enough time and enthusiasm to sit down and write a detailed summary topic (based on the results of communication with Bubble support and my experience) about :filtered being executed server/client side with examples…

p.s.

That’s true. At least they should understand what they are doing and how it works under the hood instead of following some advices blindly.

24

Were you corrected by Bubble support or another user?

When I was in Bubble training to be a Bubble Trainer for the Bootcamps they offered in the past, Bubble stated during the training that the user of the :filtered operator is client side, since as stated in the manual, those done in the ‘do a search’ are server side.

Screen Shot 2024-01-21 at 12.07.09 PM
Screen Shot 2024-01-21 at 12.07.09 PM1139×236 46.5 KB

The above from the manual is in a section specific to advanced filters and not the :filtered operator generally, which is why it states that those performed via the :filtered operator are generally “advanced” filters since most users would not use the :filtered operator for purposes other than advanced filters since most every other type of filter can be done via the ‘do a search’ constraints.

You would not see a difference in WU consumption since the filters are performed client side, they would not be part of WU consumption, however, if the search results are returned unfiltered from the server, based on the fact Bubble charges a small amount per character of data retrieved, there should be a difference in WU given a large enough dataset.

I wonder if this is in @petter book on performance. He has worked closely with Bubble engineering to get the behind the scenes run down on these types of things in preparing for writing his book.

25

Yep, that’s what I meant.

Petter knows about it for sure:

If you have the book - you can check these sections (I’m referring the latests 3rd edition):

  • Client-side vs server-side operators
  • How the :filtered (and other) operators works

But still that’s confusing even for experienced Bubblers. I’ve pointed this to Bubble support and they’ve promised to look into it. May be @petter can update it :slight_smile:

26

@georgecollier @ed727 @artemzheg @adamhholmes

This is what Bubble support provided as an answer from engineering

At its core, “:filtered” is just an operation to add new constraints on top of a list. Before ever executing a query, we simplify it as much as we can, and therefore, :filtered will either become “just a new set of constraints on top of the parent search” (very fast) or be a manual filter on top of each of the rows of the parent list (potentially very slow).

Here’s a quick guideline:

  1. If :filtered contains an advanced filter, filtering will be done “row-by-row”
  2. If :filtered contains one or more constraints, and the previous part can be condensed into a simple search with constraints, then we consolidate the constraints and do it as a simple database query
  3. If :filtered contains one or more constraints and the previous part is already too complicated to do as a simple database query, then we will do the filtering row-by-row.

In terms of references to client or server actions, the actual distinction is technically closer to “Bubble-logic” or “database queries.” Database queries are typically fast and efficient, but when it’s not possible, Bubble (usually on the client but sometimes on the server depending) will be forced to pull data directly from the database and perform filtering and such row-by-row as mentioned, which is why it tends to be slower.

We’ve of course worked over time to optimize both of these processes, and will certainly continue to do so, but in general with the filtered operator, whether or not it involves server actions depends on whether it can “compile” a database constraint; what makes :filtered compilable or not depends on whether its “parent” search is database-compatible.

  • Something that starts with a non-db-compatible list is almost never going to end up db-compatible (ex. list custom states, API call results, list fields)
  • Something that starts with a db-compatible list is very likely to stay db-compatible as long as you don’t use list operators (ex. :each item’s Thing) or advanced filters.

In short, usually if you don’t operate on already-loaded data, the process will need to iterate through whole database tables, making it slower.

9 Likes
27

Darn cool stuff @boston85719 !

Thanks for sharing :fire:

1 Like
28

Finally we have a publicly available info from Bubble support on :filtered operator.
Thanks for sharing :slight_smile:

29

SO nice to share the response with us ! Thanks :wink:

30

Imagine if all the hundreds of detailed engineering responses to support questions were compiled somewhere.

31

Just chipping in to this as my app was heavily built with many daisy chained filters.

As Boston85719 said:

This is exactly correct. Daisy chaining filtering was necessary for me as I could not complete all the filtering in the initial search step. It required multiple steps due to the way i’d designed the database fields/structure. Some very capable bubblers confirmed this (ie that i was doing it the only feasible way for my use case). I actually watched the video in question and it was a good solution at the time, and worked fast and well.

However, in the end, I re-designed the whole database structure, and made it possible to do everything from the initial search. To my surprise i noticed a BIG drop in WU usage! I wasn’t expecting that -in fact i thought that WU would increase (because I assumed the current method was only doing ONE main search then the rest filtered on front end without WU (wrong).

So yes, my opinion is, while daisy-chaining has a use case, it’s much better to figure out how to structure the databse so they are not even needed in the first place :slight_smile:

Best

32

Do you have a before and after of your db? I would love to get your insights and learnings.

33

Actually, when I looked at your app, it was just running one search. It was not running a search for each step. In terms of WU, it wasn’t much worse than normal searches - it was just really unmaintainable.

34

Yes, like I mentioned i was surprised with the big WU drop as didn’t expect that. The search was always the thing draining the WUs and after the re-design they pretty much halved. I don’t understand whats happening behind the scenes, just know that i was happy about that :slight_smile:

35

Me? unfortunatly no, but if you DM me i’ll answer anything want to ask that I can from memory. I had about 20 daisy chained filters before :sweat_smile:.