Need for Speed: Load only the last 15 items from a list with thousands of items. Items until # doesn't cut it

alejandrowunderlich · 2020-09-03T23:45:51.423Z

I need to fetch only the last 15 items from a very large list of things. However, it seems using :items until # is still very slow to complete it’s process. I’ve also tried :filtered to only load items created after X date to try and reduce the list further.

It seems that Bubble is first processing the entire list of thousands of items, then showing me the last 15 items.

I have no constraints on the list of items and no filters that could be slowing the search down. The only thing I have is to sort in reverse order by creation date.

Anyone have any idea if there’s a way to only fetch the last 15 items and “Stop” the search once it has 15 items? @keith Could list shifter come and save the day here?

deadpoetnsp · 2020-09-04T00:20:25.059Z

How about create a hidden 15 cell RG, set it to Do a search for, and then get the RG’s Display List?

ed727 · 2020-09-04T00:50:26.234Z

How big is your list?

Tried it on my end (abt 8,000 items), using a sort in the “Search for” box and adding “:items until 15”. Was just as fast as if I didn’t establish any count limitation.

alejandrowunderlich · 2020-09-04T15:38:25.708Z

deadpoetnsp:

How about create a hidden 15 cell RG, set it to Do a search for, and then get the RG’s Display List?

Interesting hack, will test the speed of this.

ed727:

Was just as fast as if I didn’t establish any count limitation.

This is exactly what the problem is I think. It seems that having a count limitation doesn’t increase the loading speed. My DB is about 7.5k items right now, and growing fast! If we could somehow just load the last 15 items, then it speed wouldn’t increase even if we have 100x items in our db.

@emmanuel would love to get your perspective here. What’s the fastest way to load the last 15 items on a very large dataset?

ed727 · 2020-09-04T15:49:40.944Z

Hi, good luck with it. I meant to say it’s fast on my side because it’s all executed server side and it’s a simple search (no constraints, and sort by created date). Bubble loads data into the page and “:items until” makes it stop loading at #15.

I’ve experienced slower loads when there’s an advanced filter (which is run client side) or when there is a nested search involved. But direct searches run server side have been very fast.

sudsy · 2020-09-04T16:39:05.995Z

How slow is “very slow”? How many fields are in the data type you’re fetching, and what type of fields are they?

I suspect the amount of data being retrieved for each Thing will have a greater impact on performance than the number of items in the list (all else being equal).

One optimization technique for frequently accessed data types that contain many fields (especially if some of those fields are large-ish lists) is to break that data type into two separate data types - a Thing data type and a Thing Detail data type.

The former would be used to quickly present lists to the end user, and the latter would be used to fetch the detail for a specific Thing that was selected from the list.

mstfmomin · 2020-11-14T11:43:58.448Z

HI,
I am facing the same problem, but I don’t find a solution here. One of the replies said doing a search and sort, but its not clear. Please give me a straight answer on how to solve it.

aj11 · 2021-05-09T01:21:14.039Z

Did you ever come up with a solution for this?

lottemint.md · 2021-05-09T01:32:22.778Z

This one may help you:

Millions of rows can bubble search features work on that size of a dataset? Database

Hey @court, It should work okay without using the Algolia. I had the same question before, so I decided to test it. There are 1 868 420 of records: [image] The following preview page shows the total and the first four records immediately: https://jus4tests.bubbleapps.io/version-test/urls [image] Note: The app is on the free plan. The most important thing is to structure the database in the right way. For instance, for a Product table you need to create an additional one Product …