Best ways to handle database changes with iterative development

Hi.

I am building an app that I want to grow iteratively. This means, of course, that I will need to make changes to the live database at the end of each iteration of development, based on changes to the dev database model.

So as my database model grows and changes, so will the relationships change. And the changes needed on the live database could potentially be quite complex, including not only the relationships between database tables/fields but also of the data that needs to be populated into the live database.

Are there any tools, methods, procedures, approaches that are commonly used with bubble to make this easier?

Thanks!

Hi, kind of a broad question, but I’ve used recursive workflows for these sort of database-wide changes. I think that’s the most common method.

Yep, great idea to use recursive workflows, thanks @ed727. :slight_smile:

And how do you handle things like:

  • tables that have references to fields in other tables
  • data migration from dev to live – do you download to .csv from dev and upload from .csv in live?

And how would you trigger the workflows that do the database updating? Would you create a page with forms controls on it, or is there some other way to instigate these?

And are there no bubble tools that can compare databases and update one to mirror the other?

Well, when I was building my app and then ready to go live, I used Bubble’s “copy dev to live” function to copy over my development database into my live app database (which was empty). But once my app was live, I wouldn’t do that since I was maintaining my live database going forward. This function (copy live to dev, or copy dev to live) completely over-writes one database with the other, with no way of going back, and so one needs to be EXTREMELY careful in using it.

Re: the csv upload and modify functionality, I used it initially to upload data into my dev app. I haven’t used it subsequently. My initial CSV upload was tricky – both in getting fields properly formatted, and also ensuring the whole thing uploaded without error – so I personally have shied away from using it to modify live data. Instead I’ve modified data within Bubble using workflows, where I feel I’m less likely to screw something up.

If, for example, I decided to create a new field in a datatype to connect each entry to one or more entries in another datatype, I would probably:

  1. Create and test the new field in dev.
  2. Create the workflows to modify my dev data, and test them on my dev dataset.
  3. Deploy my dev app (just the app, not the database) to live.
  4. Run the workflows to modify the data on my live database.
  5. Go back to dev and build and test whatever new functionality I am gaining by adding those connections.
  6. Deploy the dev app (not the database) to live.

I created a page with a button to start the backend recursive workflow, and also sometimes I’d create a counter or some other method to track the workflow’s progress.

Final note… if my dataset isn’t too big, sometimes I’ll do bulk data changes using a regular, front-end “make changes to a list” workflow. I’ll set up a RG, build some basic search functionality to isolate the data, and then have a “make changes to a list” execute on the data in the RG.

It all comes down to the nature of the change, the size of your database, and the easiest way to do it given the tools Bubble gives you.

1 Like