Thanks for the info Fabrice. Regarding the bytes for large data types, ok, I was not very aware of that, I’m always thinking in terms of records and not considering very relevant the number of fields/characters. I will have this in mind for DB planning when searching large data.
Regarding nested objects, yes it seems that as deeper is the level of related objects the loading time grows kinda exponentially.
- However, for the Bubble side I found a way that (for some reasons beyond my knowledge) speeds up the loading, are combinations of “:make static:filtered (with sort)” “:make static:sorted”, “:make static:unique elements:filtered/sorted”.
I’ve been doing further tests with all this, I go to detail here the two key points I see:
Note: If something is wrong, I apologize, I’m not a programmer and I can only test by changing a variable at a time to understand the source of the problem and make progress.
A) Indexing process?
For non-nested objects, for 15k records, I can get the total count of things and start loading the records in the RG almost instantly, however if the data source comes from the plugin it takes more than a minute. This data type I would say is not very complex, just around 15 fields, some related objects, but not very populated.
But, as I said in the previous comment, I guess that this is just because native Bubble can know the total count and start showing a few records without really having processed all the 15k records. While the plugin, needs to have completed the process of all the records to be able to make the search later. Kind of an indexing matter.
-
Preload in custom state list : I tried to preload the data in a custom state and then set plugin to get the data from the custom state. But it does not help, it takes the same time. This is what makes me think something like the plugin needs to process/translate the data to its own schema
-
Preload in a secondary plugin search element : If instead of a custom state, I preload the data on a search plugin element, and then set the main search element to get the data from this secondary one, there is no further delay once the data are preloaded. Again, it seems that once it is in its own scheme it works super fast. So this might be a workaround for large data.
B) No update when data source is zero
When the data source has not records, it seems to me that the plugin does not update the list.
I think this causes further issues, like preventing to use filter/sort.
-
If I use the filters before the search (in the plugin search data source or with conditionals that change the data source), if the outcome of the filters is empty, then it seems the plugin get stick with the previous list, and never updates the list to zero, giving a wrong list of records for the user.
-
If I use the filters/sort after the search, it cannot work if the highlight feature of the search is being used, because it changes the list after the search (the number and sort of the records) and therefore the search matched fields and the later list does not match obviously.
I tried it as a workaround to use the filters
I checked that is not only that visually the list is not updated, even the count of things remains with the previous number and never updates to zero.
I’m positive with this and I guess that this is just something that can be fixed if the plugin updates the list when data source is empty.
Loading status:
- It would be very nice to have a “loading status” for the search plugin data source, as it is for native bubble. It would help to set conditionals for hiding, changing text, etc. while the data is being loaded. I tried the plugin event for when data is loaded, but it only works the first time, so it is not really fitting with the use of conditionals in the data source.
Just a very little last issue: it seems that the highlight color does not work well. The color ser in the backend does not relate with the color show in the frontend. I think you say something about there is a difficulty in how Bubble handle color codes…
Thanks for your work and help!