Bubble doesn't delete data in fields that are deleted

So I came across a forum post about bubble not deleting the data in fields that have been hard deleted in the app editor, the suggested solution there was to reach out to bubble support to get the data deleted so I reached out to them.

The response was quite frustrating.

First the support asked which fields I was talking about - to which I said “all the deleted fields on all the tables and gave several examples”

Then they replied with “we need a list of the fields for each table that you want to delete”

Now, I’ve been bubbling for many years and I’m constantly moving fields, adding and removing fields to data tables as the database evolves and changes. I have some bubble apps with hundreds of data tables and likely thousand+ fields (and likely hundreds of deleted fields). The bubble support basically indicated that if I wanted to actually remove the data from these fields that I would have to give them a list of the fields to clear.

The issue here is that the only way to do that is to load each data on a page and compare the returned data to the fields in the table in the app editor and then figure out which fields are no longer existing in the editor. An extremely tedious and time consuming task.

Apparently there is no way for bubble support to do this and I have to do it manually on my end.

So, my frustration is really:

  1. we are being charged per character returned and all fields (including deleted fields) are being returned whenever the data is used on the page - so we are all being overcharged for data we ‘think’ no longer exists
  2. fields and their data are not actually deleted when we delete them in the app editor and their data is still exposed and returned to the browser (major issue if you had secure data in the wrong place)
  3. you cannot change privacy rules on fields that were hard deleted - so there’s no way to solve for #2 except to set the rule first before deleting
  4. there is no way to actually resolve the issue in any reasonable way with bubble support.

Does anyone have a solution for removing the data in deleted fields - or even a time-efficient solution to identify the deleted fields per table so I can send it to bubble support.

2 Likes

Hi @mitchbaylis not sure about this, but I have a very strong feeling that the “Clean app change history” button would clear any used data that might be existing in your app. Here is where to find it in the image below

I’ve seen some discussions about this. I think that maybe even when we think we’re performing a hard delete on our side, Bubble is actually just performing a soft delete, so that actions like restoring versions of the app or savepoints can occur as expected. This is basically what we do for our clients, we give them the option to delete things, we make it look like they’re performing a hard delete, but for us it’s actually a soft delete to ensure that if we need to restore that data (or even them in some cases) it’s still possible.

Anyway, I think this is the reason why the fields aren’t actually deleted immediately. As well as certain elements like REs, deleted pages, etc… that many users report no longer exists in their apps and yet continue to consume WUs in some way. I think the same principle applies here.

However, it seems that this ends up incurring other problems already mentioned in other topics, including those listed above, such as privacy rules, probable additional cost for data returns, etc.

A possible solution could be Bubble allow us to perform a permanent hard delete at the moment we are deleting a field. Giving us this option and making us aware that by doing so, there would be no way to restore it, not even by restoring previous versions of the app. So this responsibility for choosing would be the developer’s, including all its consequences.

Although the solution suggested by @iwakinomotoye could help in some way, I’m not sure if it really removes the fields definitively, precisely because the fact of restoring a save point if necessary.

Another problem with this option, which I’ve already mentioned in some topics, is that if you forget to perform this action frequently, over time you can accumulate hundreds of styles, fields, datatypes, option sets, properties, etc… a huge list of items that you can’t even open this option without the page crashing (even if your PC is a super power). In some cases you may even be able to open it, but when performing a removal action (even 1 item) you may get stuck on the page.

I’ve already suggested not displaying the entire list at the beginning, but allowing us to select by category of removed items. However, no response so far.

I haven’t tested it yet, but maybe an easy way to do this would be to not provide the fields you want to remove, but the fields you want to keep.

And then, to avoid problems passing the field name (as we see in the datatype) and having the bad luck of naming another field (already deleted) with the same name and Bubble removing the wrong field, maybe it would be better to pass the unique IDs of the fields you want to keep and ask it to remove all the others. For this you could maybe use a plugin like this: Fielder: Get Field Names for Things Plugin | Bubble, or maybe obtain it by accessing it directly in your app or using plugins like: Swagger UI Plugin | Bubble.

2 Likes

hey Mitch :wave: this is similar to something I queried with Bubble support 18+ months ago, so I’ll share what I’ve learned but with the caution that it might be out of date.

You’re correct in that deleting a field neither actually deletes the field, nor does it delete any of the data in that field.

This seems to mean that Bubble support don’t know which fields you’ve deleted and which you haven’t, which is why they’ll have given you that response - they “need a list of the fields for each table that you want to delete” because they don’t have a record of what appears as deleted on your end.

This also means that, if you have deleted a field from a data type, you can restore that field and the data within it by creating a field of the same name and type. For example, if I delete a field called ‘Quantity’ of type number, I can get it back by creating a new field called ‘Quantity’ of type number.

Where this helps your case is that you can:

  1. recreate the fields you’ve deleted to bring the data you thought was deleted back into your app.
  2. then create a backend workflow to clear the data fields you want to delete from each data type.
  3. run the backend workflow as a bulk action on the data type until that field is empty for all entries.
  4. then delete the field as you did originally.

Even if you can’t remember the names of all the fields you had to recreate them, this is something Bubble support might be able to help you out with - they should be able to send you the names of all the fields for each data type. Once you have that, you could export a sample csv containing all the fields you have from the current version of your app and build a simple spreadsheet to work out which things aren’t in both lists.

I totally get why this is frustrating (and not just frustrating - you’re right that there are also security risks if fields are deleted without the correct privacy rules applying to them). It’s something I’m glad I figured out early on in my journey using Bubble, and should be emphasised more heavily in their documentation around how data types are managed as well as functions like ‘optimise application’!

Using backend workflows & bulk data actions to clear fields before I delete them is now something I do every time I want to delete a field, even if the field supposedly contains things that are themselves deleted. For example, if I’ve deleted all the options in an option set, and I have that option set as a type of field in a data table, I will still make sure to clear all the data entries where that field wasn’t empty, just in case those options or their associated properties are still lingering around somewhere in Bubble. I would rather be safe than sorry!

From a Bubble developer perspective, it’s motivation to spend time seriously considering and futureproofing the design of your databases before you start developing a new project (and definitely before it goes live), so that deleting a data field is something you only have to do very rarely. As you’ve unfortunately found out the hard way, truly deleting data fields in Bubble can be a very laborious task that I try to avoid wherever possible.

1 Like

A little tip for this specific problem - if you add the parameter issues_off=true to the URL of your Bubble editor, it vastly improves the performance of the ‘optimize application’ and ‘clean app changes history’ processes. Use it with caution though, because the reason it becomes faster is because you’ve turned off Bubble’s issue checker, so it’s worth removing that parameter so you don’t accidentally push live an app with 100s of issues.

appreciate all your responses

in the end I decided to just put up with the data being not deleted. to go back and cross check a hundred tables or so with about a thousand fields is too time consuming for the ROI.

I hope bubble adds a function to allow us to actually hard delete the fields and data at some point so I can clean it up, but until then I’ll just put up with it and attempt to be cleaner in the deletion process in the future.