Hello,
I’m facing a major issue with a simple query on a table that contains 140k items by default.
I’m doing a basic “Do a search for” with a constraint that returns 102k results.
Unfortunately, I can’t display or retrieve any data (not even the unique_id) from items beyond the 50k mark.
I tried querying using “item # from”, and even directly using “item #50001”… no results.
I tested “item # from 49999” and I do get 2 results.
Whether it’s on the backend or frontend: Bubble doesn’t seem to allow access to items beyond #50000.
What am I missing here?
I’ve tried many things to “bypass” what clearly looks like a hard limit on Bubble’s side… but nothing works — it just won’t go past 50000.
And just for context: there’s no sorting applied on my query, which I know normally limits to 50000 items.
Unfortunately it’s a limitation of bubble, I went through this in a database migration we did, to solve it, we took the records by season, I don’t know if for your case it would be like this, but you can do something similar, we put records for example from January to February, then March to April, thus fitting into the limit of 50 thousand items.
Thank you for your answer… unfortunately i can’t do this… i need to request all the item from 50k+1 until 50k (event 25k / 25k i don’t care)…
This is very very problematic…
Really problematic, did you manage to do anything about it?
Nop.
Every new project, i prefer to use supabase now …
Sort them all by created date. When you reach 50,000, take the last item’s created date, and add that as a constraint and look for items created after that creation date. Then your cursor is effectively reset to zero.
Regardless, trying to return >50,000 results in one query on any tech stack will begin to cause problems, so it’s not a surprise it fails when you try to do this with Bubble.
Sorry it’s not an acceptable behavior. if I want to return item#50001 ( just one item ) . I can’t . No matter the constraint .
Do you mean that using any backend when you query for > 50,000 results in one query, this will begin to cause problems ? Did I read this correctly ?
What do you mean?
If you constrain a search so that there are less than 50,000 possible results, then you can access each result using cursor.
Suppose we have a Thing for each number, going up to 150,000.
Do a search for Numbers could have 150,000 results, and you won’t be able to access numbers past 50,000.
But, you can Do a search for Numbers >= 50,000 and < 100,000, and will be able to access all 50,000 of those using the item # or cursor.
I don’t really understand you’re use case. You can’t possibly be trying to return 50,000 results in one search. Are you trying to export data? If so, you should probably use the Data API. If you give more details about what you’re trying to do, we might be able to help.
It’s a generalisation but basically all production grade APIs will recommend pagination over returning 50,000 results, because the request payload will be unreasonably large, cause too much memory pressure, DB load… of course like anything, there will be exceptions. But yes, trying to return 50,000 database rows in one query is virtually always a bad idea.
My use case started with an export data yes .
I needed to format data in a specific way with specific format as text for some fields . Export data with csv would be very complicated after to “reformat” data. …
Data api wont solve anything , same problem with 50k item ( support answer).
Your exemple supposed that a number is stored on each record…. Which is not the case .
And , just to clarify if it wasn’t clear: no constraint is used .
I’ve discuss about it with support and, yes , they said this behavior is not optimal but they can’t do anything for now .
I understand all what you mean , all possibility to workaround : but it’s still a workaround for something which it should be possible in my simple example :
I want to query ONE item with the operator : “#item” … but I won’t be able to query after item 50k.
Even if i use the operator “random item” it won’t be random on all my database( or all my the record query with a constraint) but just the first 49999 items .
I know, and I’m telling you that if you use a constraint when exporting data, your issue will be resolved.
And if all my 50k + record are in the same time ?
Will your example worked ?
It is, for me , normal that we could not load more than 50000k record in one query .
But not normal that we can’t query item from et item xx when it’s more than 50k..
And what about my item 50001 ? Impossible to know when it is created …
Bubble should request #item, /from /until before the constraints themself .
And for me the problem is deeper:
I used to put those operator to add pagination for my repeating group to consume less Wu because I assume that it was before the constraint …. But it’s not … and nothing about it in the documentation…
I understand what you mean , and your suggestions.. . But I won’t change my mind on something which should be different
Could you provide a clear use case for this. There seems to be a lot of missing details on ultimately what you are trying to achieve.
Are you trying to reduce WUs to display all 140,000 items? Are you trying to export only one item and be able to format the text for the export only?
I think what is missing in these explanations is that cursor is for the Data API, which likely would be the easiest way to solve for being able to fetch via the Data API the single item or at least items beyond 50K as the cursor is how pagination in the API works.
The cursor limitation still exists in Data API (like items from #) but is much easier to deal with 
I see. You’re assuming that he’ll always be using Bubble 
Not at all… basically any platform will discourage returning 50,000 items in one go. I promise.
That’s right as long as you’re retrieving data through API call or using “Do a Search for” in Bubble. However, in traditional deveopment with a traditional relational database, you could retrieve 50,000+ records in 1 select statement and display the item # 50,001 with no limitations although to show them in the frontend, it makes perfect sense to use pagination
Hello guys, I encountered the same issue, but for another purpose.
I wanted to loop on a datatype data, to change a field (national phone to international).
So use to have a backendworkflow with one paramater as a number “startAt”.
I use to create some logs entry to know where I’m in the process :
So it stopped at the 50.000 entry
When I read messages here, seems you think he is trying to retrieve 50K rows, but if it FROM 50.000 until 500, this is 500 rows output, not 50K right ?
ANd what is strange is on my other app, I did the exact same wf (and not for the first time)
and as you can see, it worked great, as usually
As you can see the item #188.001 is not empty.
So am I crazy ?
Why this time returning item FROM 50.000 is not working ? Do I miss something ?
It’s specifically about using items from or cursor to get a Thing at position nin a list. If n > 50,000 Bubble just doesn’t let you do it. It’s not about the number of results returned.
For your backend workflow, do the following:
Do a search for Sportifs sorted by Created date:ascending#items until 500and make changes to list of things- Pass the
:last item's Created Dateof the above expression into the next scheduled workflow Do a search for Sportifs where Created date > (the provided date):sorted by Created date:ascending#items until 500and make changes to list of things
Ok thank you @georgecollier got it
But why the hell it worked on my other app ? that’s crazy.
Another wf where it worked
The other app where it worked is on a dedicated plan, is it the difference ?
In the bubble doc they said only sorted lists are limited to 50K but it was not the case here
ANd I can return more than 50K rows
