Bulk Updates Causing Search Performance Issues Due to Reindexing

Need help Database
1

Hi everyone,

I have a question regarding bulk updates and database reindexing in Bubble.

I have a Data Type with around 300,000 records. Recently, I added a new text field and needed to populate it for the existing records. My first approach was to perform a bulk update, but after updating only about 20,000 records, searches on that Data Type became noticeably slower for nearly three days.

After contacting Bubble support, I was told that the slowdown was caused by the reindexing process triggered by the updates. As a workaround, I started updating the records gradually using Backend Workflows.

At the moment, I’m updating about 2,000 records per day to avoid affecting search performance. However, at this rate, it will take more than 100 days to complete the update across the entire dataset.

My question is: is there any recommended limit or best practice regarding how many records can be updated without causing significant performance issues due to reindexing? Has anyone dealt with a similar situation involving a dataset of this size?

I’d appreciate any insights or experiences you can share.

2

You should follow up with Bubble there. It’s a bug if it’s happening for days.

‘Indexing’ of new records happens instantly I believe (that’s how it works in Postgres and it seems strange for Bubble to specifically try to do something different)

These is a different process where each day, during a low period, Bubble looks at the slowest queries for an app and determines if an index would improve them. If it would, then it creates the index. That means that new queries might result in a new database index being built. But that wouldn’t take three days.

3

It’s not a bug; it’s the three-day timeframe Bubble allows for re-indexing. I found this to be a long period, as it causes slowness and forces a slow update of the necessary files.

4

Yeah I’m saying that seems like rubbish because in postgres when you insert any record, the indexes are updated in the same transaction.

I’d suggest going back to support and getting more details

‘Re-indexing’ over three days implies creating new indexes/cleanup, not maintaining new records in existing indexes.