Daisy Chain Filtering and Search Constraints Best Practices

[boston85719]

@artemzheg

Usually “client-side filtering” term implies the fact that Bubble returns all items row-by-row (relevant and not relevant) from the DB until there is enough relevant (corresponding filters) items to be displayed. The most well known example of client-side filtering is using Advanced: option.

So in my example above both filters seems to be applied server-side (Bubble fetches relevant items only).

Anytime you use the :filtered operator, all the filter constraints within that are done client side, not server side

@ed727

A single filter containing all the constraints

@ed727

Not sure about #2, since Josh said Bubble might execute a filter server side, or might not, depending on the filter.

As far as my understanding is based on what I’ve been taught by Bubble is that using the :filtered operator, regardless of the type of filter, is happening client side, not server side.

@artemzheg

if you are using default checkbox elements a lot.

I believe you are referring to the checkbox element from Bubble

Nobody should be using those in my opinion, nor should anybody be using radio buttons. I started using Bubble March 2018, I think that was also the last time I used either of them. In all of my apps, I build my own checkboxes for filters by using a repeating group with the filter options as the datasource, then add into the RG cell an icon (checkbox, or radio whichever the design dictates). As I can remember, the main reason I did this was it was much easier to manage the selections of the user.

@artemzheg

Going with standard way filtering - you’ll have to manage a lot of conditional data sources.

You do not actually, but maybe if you are using the checkbox element type, I’m not sure. But, in my apps, the only time I use a conditional data source for the most part on my RG is when I am including a function to allow a user to select from a searchbox element a value (usually user) and then the RG will display only the selected value, so my conditional datasource is the searchbox value…other than that, I rarely require a conditional datasource for the purposes of filtering.

@artemzheg

small element inspector window

That things needs an overhaul…hopefully the AI Bubble is building will fix it

@artemzheg

Also “ignore empty constraints” is ignored by advanced filtering, so we need to add some excessive expression like ...or this multidropdown's value:count > 0

Yes, true. What I’ve done in the past to overcome that is to set somewhere all the values, so that it would be similar to all filter choices pre-selected, which was overkill and very convoluted. I’ve recently begun using the 'intersect' count is 'selected values count'which makes it so that when the selected value count is 0 the advanced filter constraint is effectively ignored because it is empty…this is not requiring a conditional, it is in the main advanced filter as part of the dynamic expression for the constraint itself.

@artemzheg

P.S. also there is a twitter thread where AndrewV (developer from Bubble team) shares how and why he is using DCF in his project:

Just watched the video, and no offense to Andrew or Gaby who pioneered this approach in 2017, but my stomach turned over when I saw 37 custom workflow triggers. It is not so much the number of 37 that, but the use of custom workflow triggers itself. Custom workflow triggers are not copied and pasted over when you copy and paste with workflows elements…but if you copy and paste a RG with a search that has what I believe is the proper approach to filtering (ie: constraints on the Do a Search with Ignore empty constraints), then that gets pasted over. Plus, how easy is it to get in there and Remove a filter? Seems like from the video that would be a pretty daunting task to find and manage properly, because if you are like me and naming conventions are important, you’d need to rename all of the subsequent custom workflow triggers that follow the single filter you wish to remove, plus the time it takes to find it - of course you don’t need to update anymore than one custom workflow trigger to keep the chain of events occurring, but yikes, what a mess that is.

@artemzheg I understand you are not advocating for it’s use and are playing devils advocate with some of the hypothetical examples you provided, which is much appreciated, because we still have yet to hear from anybody who actually uses it to describe the benefits of daisy chain over just using search constraints.

I just hope this approach is not picked up by too many newbies, as it just seems to get people off to a really bad start in terms of understanding how to interact with data within Bubble.

[georgecollier]

@boston85719

As far as my understanding is based on what I’ve been taught by Bubble is that using the :filtered operator, regardless of the type of filter, is happening client side, not server side.

Only advanced filters or when used on option sets. Bubble treats :filtered like a search whenever possible. I thought this a while ago too but I was corrected

[artemzheg]

@boston85719

Anytime you use the :filtered operator, all the filter constraints within that are done client side, not server side

@boston85719

As far as my understanding is based on what I’ve been taught by Bubble is that using the :filtered operator, regardless of the type of filter, is happening client side, not server side.

@georgecollier

Only advanced filters or when used on option sets. Bubble treats :filtered like a search whenever possible. I thought this a while ago too but I was corrected

I’ve used to take on faith that as soon as you apply :filter operator - everything is filtered client-side. And this was the reason:

An underlying principle is that if a filter (or sort) can be done “on the database”, it will be faster than a filter (or sort) that Bubble has to do after retrieving an initial set of data from the database. Which filters are done on the database vs. not? Filters which show up in the Search palette (the additional sidebar which slides out when you click “Do a search for”) are done on the database and are thus are generally fast. Filters which are applied with :filter are generally “advanced” filters that are generally slower.
[bubble manual]

But while playing with WU consumption in one of my apps I’ve noticed zero difference between using basic [=fields of the data type] constraints in Do a search expression and in :filtered operator. I’ve dived into Chrome DevTools and discovered that :filtered operator returns filtered items from the DB (again, if I was using basic constraints).

Not wanna copy the whole thread here, so you can check some info I got from Bubble support about client/server filtering here:

Search & Filter Performance

Hi Bubblers, Looking into ways to optimise our app and our uses of search and filtering. I understand search happens on the serverside while filtering is done locally. In a scenario where we have a large list of Things that need to be filtered based on the tab a user is currently on, is it better to apply a different search with a constraint for each tab, or to first load the data into a ‘master’ search without the constraint and then apply a filter on the master list for each tab ? I also o…

May be one day I’ll have enough time and enthusiasm to sit down and write a detailed summary topic (based on the results of communication with Bubble support and my experience) about :filtered being executed server/client side with examples…

p.s.

@boston85719

I just hope this approach is not picked up by too many newbies, as it just seems to get people off to a really bad start in terms of understanding how to interact with data within Bubble.

That’s true. At least they should understand what they are doing and how it works under the hood instead of following some advices blindly.

[boston85719]

@georgecollier

Only advanced filters or when used on option sets. Bubble treats :filtered like a search whenever possible. I thought this a while ago too but I was corrected

Were you corrected by Bubble support or another user?

When I was in Bubble training to be a Bubble Trainer for the Bootcamps they offered in the past, Bubble stated during the training that the user of the :filtered operator is client side, since as stated in the manual, those done in the ‘do a search’ are server side.

The above from the manual is in a section specific to advanced filters and not the :filtered operator generally, which is why it states that those performed via the :filtered operator are generally “advanced” filters since most users would not use the :filtered operator for purposes other than advanced filters since most every other type of filter can be done via the ‘do a search’ constraints.

@artemzheg

But while playing with WU consumption in one of my apps I’ve noticed zero difference between using basic [=fields of the data type] constraints in Do a search expression and in :filtered operator. I’ve dived into Chrome DevTools and discovered that :filtered operator returns filtered items from the DB (again, if I was using basic constraints).

You would not see a difference in WU consumption since the filters are performed client side, they would not be part of WU consumption, however, if the search results are returned unfiltered from the server, based on the fact Bubble charges a small amount per character of data retrieved, there should be a difference in WU given a large enough dataset.

@artemzheg

May be one day I’ll have enough time and enthusiasm to sit down and write a detailed summary topic (based on the results of communication with Bubble support and my experience) about :filtered being executed server/client side with examples…

I wonder if this is in @petter book on performance. He has worked closely with Bubble engineering to get the behind the scenes run down on these types of things in preparing for writing his book.

[artemzheg]

@boston85719

You would not see a difference in WU consumption since the filters are performed client side, they would not be part of WU consumption, however, if the search results are returned unfiltered from the server, based on the fact Bubble charges a small amount per character of data retrieved, there should be a difference in WU given a large enough dataset.

Yep, that’s what I meant.

@boston85719

I wonder if this is in @petter book on performance. He has worked closely with Bubble engineering to get the behind the scenes run down on these types of things in preparing for writing his book.

Petter knows about it for sure:

The Ultimate Guide to Bubble Performance - the new edition is out (now 210 pages!)

Hey @greg18 , Thanks for posting! The difference that you bring up about :filters being performed server-side/client-side was one of the major revisions of the second edition of the book, and if that’s the version you’re referring to it may not be clear enough still (I can’t check the book right now, but thanks for including page numbers). In the first edition, filters were described as being performed client-side more or less without exception, but it should have been made clear that this wil…

If you have the book - you can check these sections (I’m referring the latests 3rd edition):

• Client-side vs server-side operators
• How the :filtered (and other) operators works

@boston85719

The above from the manual is in a section specific to advanced filters and not the :filtered operator generally, which is why it states that those performed via the :filtered operator are generally “advanced” filters since most users would not use the :filtered operator for purposes other than advanced filters since most every other type of filter can be done via the ‘do a search’ constraints.

But still that’s confusing even for experienced Bubblers. I’ve pointed this to Bubble support and they’ve promised to look into it. May be @petter can update it

[boston85719]

@georgecollier @ed727 @artemzheg @adamhholmes

This is what Bubble support provided as an answer from engineering

At its core, “:filtered” is just an operation to add new constraints on top of a list. Before ever executing a query, we simplify it as much as we can, and therefore, :filtered will either become “just a new set of constraints on top of the parent search” (very fast) or be a manual filter on top of each of the rows of the parent list (potentially very slow).

Here’s a quick guideline:

1. If :filtered contains an advanced filter, filtering will be done “row-by-row”
2. If :filtered contains one or more constraints, and the previous part can be condensed into a simple search with constraints, then we consolidate the constraints and do it as a simple database query
3. If :filtered contains one or more constraints and the previous part is already too complicated to do as a simple database query, then we will do the filtering row-by-row.

In terms of references to client or server actions, the actual distinction is technically closer to “Bubble-logic” or “database queries.” Database queries are typically fast and efficient, but when it’s not possible, Bubble (usually on the client but sometimes on the server depending) will be forced to pull data directly from the database and perform filtering and such row-by-row as mentioned, which is why it tends to be slower.

We’ve of course worked over time to optimize both of these processes, and will certainly continue to do so, but in general with the filtered operator, whether or not it involves server actions depends on whether it can “compile” a database constraint; what makes :filtered compilable or not depends on whether its “parent” search is database-compatible.

• Something that starts with a non-db-compatible list is almost never going to end up db-compatible (ex. list custom states, API call results, list fields)
• Something that starts with a db-compatible list is very likely to stay db-compatible as long as you don’t use list operators (ex. :each item’s Thing) or advanced filters.

In short, usually if you don’t operate on already-loaded data, the process will need to iterate through whole database tables, making it slower.

[cmarchan]

Darn cool stuff @boston85719 !

Thanks for sharing

[artemzheg]

Finally we have a publicly available info from Bubble support on :filtered operator.
Thanks for sharing