Full DB copy may take 10 days to complete?

Need help Database
1

Not sure if anyone in the Forum has experience with this or any recommendations on this??

I have a large DB and am currently in the process of trying to copy live to development. In the recent past, it has taken a little over 24 hours, which I thought that was very slow. I started the current DB copy last Thursday the 19th and it is still NOT done? I now guessing it might be 10 days until the DB is copied.

We can’t do any development that requires database.

I hope there is just something wrong here and the normal expectation for my application is not a 10 day process to copy the DB. But they say there is no way to stop this copy (to be able to start over and see what the issue might be).

@josh and @emmanuel … I know you and the team are probably more focused on BubbleCon at this week, but maybe this can be one of the questions you address from a real user with live/paying users and I can’t do a lot of development because of this??

Bubble’s latest response is …

"Jason here, from the Tier 2 App Design team, stepping in for Ranne.

Once the copying of a database from one version to another has started, we are unable to stop this operation on our end and you would need to wait for it to fully complete. That said, when checking this on my end, I was able to see it has made progress since your initial report from yesterday:
<Screenshot 2023-10-24 at 10.03.20 AM.png>

The amount time this operation requires is based on the amount of data that is being copied, so it can take some time with larger databases or if each entry contains a large amount of data. We see this especially with the Z - Statements-Reprocessing datatype where even though there are only 1000 entries, there is a lot of data saved to “MA Ext First Transactions” field.

I sincerely apologize for the limited action I’m able to take here - but I hope this helps clarify things."

1 Like
2

Wow… no matter how heavy your DB structure is 10 days is ludicrous…

Out of interest:

  1. How many things are in your database (sum of the number of things in each data type)
  2. How much data is saved to those list fields? How many list items, and how long is each list item?
3

Not sure how to get that number for you, but I do have over 100 Datatypes and some of them do have heavy data in them. I originally contacted Bubble support a few months ago when it was taking 24 hours to copy my DB and I got the same response. So I have just dealt with the fact that it would take 24 hours to do a full copy. So the problem I have now is that I don’t believe any of the changes I have made to the DB in the last few months is a reason to cause the process to copy full DB from 24 hours to potentially 10 days. The actual time for full copy is still to be determined as it is still not done!!

4

If you’ve got 100 data types your app has obviously had a lot of work on it and you can estimate at least the order of magnitude of number of stuff in your DB.

This is the one that might quite relevant, though.

5

Don’t know… but 10 days seems a little off to me… Unfortunately, if they can’t help you, I am afraid that neither can we… That sucks.

I have a very complex App that I consider it large (40 DataTypes). I am curious to know what kind of App do you have. Do you might to share with us?

6

Yeah, 10 days seems like a very long time!!..

How big is your database?

Out of curiosity, I just pushed one of my test apps live, with a database of approx. 110k things (many of which have very large list fields, of up to 10K items each), and copied the development database to the live database.

It tool precisely 13 minutes 23 seconds.

If my maths is correct, for it to have taken 10 days, my database would have to contain around 110 million things.

Is that the kind of numbers you’re talking about here? (or is it just very heavy data?)

7

What irks me is that we’re essentially handcuffed when it comes to any development requiring user data or data groups during this time.

I think Bubble should give us the option to select which data types to copy during this process, just like we can select individual data types. This would at least allow us to bypass heavy data types and get on with our work.

On a related note, I’ve observed that the copying process seems to go more smoothly when the progress window is open on my screen. When I close my windows and go to lunch, barely any progress is made in an hour. However, leaving the progress bar window open seems to make the process faster, completing in about 15 minutes. Not sure if this is a placebo effect or an actual bug, but it’s worth mentioning.

1 Like
8

The response from Bubble is more about the nested tables and things that have a list of things in them … I just deleted all the data from the heavy table they referenced as it did have heavy things inside of that data type, but not sure that is helping make it go any faster.

I just don’t know what is different in my DB today versus the last time that I copied it where it took about 24 hours versus potentially 10 days this week?