Forum Academy Marketplace Showcase Pricing Features

[SOLVED] Preventing "orphaned" files - How would you handle it?

You need to create a LIST “ALL Files” field containing all file types. During the cleaning process have erased those that have not been assigned.

Thanks for the reply, Reece. I’m actually using that approach now. However, using Bubble’s native file uploader, the file itself is added to Bubble storage as soon as it’s “selected”. It’s only the URL (or a Bubble reference of some kind) that gets added to the DB when the Save occurs to associate it with the intended object (User in this case).

As such, using custom states doesn’t really address the issue.

Am I overlooking something?

1 Like

Ah I see, are you autobinding the image to a data type?

Otherwise, I don’t think you are overlooking anything

No auto-binding is used. I wonder, though, if I need to somehow leverage that and/or the “attached to” functionality in order to easily identify orphaned files.

That’s the part I’m struggling with. It’s not clear to me how do identify “unassigned” files. Perhaps I need to leverage the “attached to” functionality to make identifying unassigned files easier.

Hey @JohnMark, thanks’s for your input, and sorry for being so dense, but I think I’m missing something fundamental. I don’t even understand how to list ANY files that aren’t associated with someThing in the DB. And there’s only one file-related action that I can see in the workflow tab, and that’s to delete an uploaded file. How exactly would one create a list of files that aren’t “in the DB”? :confused:

Hi @shot,

You mentionned

During this process, you save the image name in the File LIST. Later, run the cleaning process. You have to create a new data type with no privacy (like ADMIN).

1 Like

Ohhhh, ok, gotcha. Thanks!

Of course, the original issue remains - i.e. if there’s a network disruption or if the user abandons the page, an orphaned file will result.

1 Like

I think I’ve got a workable solution!

Instead of doing so when the “save” button is clicked, use the “input has changed” action on an uploader element (file or image) to immediately flag the file as orphaned upon upload (add it to an “orphaned files” list). Then, during the “save” operation AFTER the “Thing” has been created, remove the file from that list.

This should eliminate the need for a “master list” of ALL files, and it should improve the performance of the garbage collection operation, since the list will be MUCH smaller - i.e. only the [relatively few] files that have been orphaned.

I’ll be testing this out, but does anyone see any reason why this wouldn’t work?

3 Likes

Yes that way sounds good.

1 Like

Just a quick follow-up… I got this wired up and working, and I think it’s going to serve my purpose. Even with large uploads, the “input changed” event doesn’t fire until the upload is complete, which is a good thing. :+1: Being able to flag the file immediately on upload makes the difference. (I guess I overlooked the obvious initially.)

Also, I think I might create the “Orphaned Files” list on the user instead of as a global. That way, I could at least display a list of “abandoned files” somewhere in the user’s account settings - just in case the garbage collection misses a few. I can’t seen any downside to doing so right now, but lots of testing to do.

Thanks for the helpful input, @JohnMark and @help.

Now off to create “replace image” functionality! :slightly_smiling_face:

3 Likes

Hi @shot ,

Can you help me out with this. I’m trying to do the same.

I’ve managed to save the url of the file every time a user changes the image when input has changed. But currently it is saving the url of all files not just orphaned files. How do I determine that eg. 4 out 5 files are orphaned because the the user has decided to stick with the last file? How do I also delete the registry of orphaned files once the files have been deleted from the File manager?

Thanks

I think this would be the solution,

Hi @mangooly,

That seems like the right approach, since you can’t delete a file without knowing its URL.

Technically, a file is considered “orphaned” only if you don’t have a reference to it - i.e. you didn’t store its URL somewhere. Since you’re storing the URL of each uploaded file, you shouldn’t have any orphaned files. Rather, it seems the issue you’re facing is how to delete any previously uploaded files - i.e. all those before the final “keeper” is saved. It seems there are a number of approaches that could be taken. One might be…

When a file is uploaded, “flag” it somehow immediately upon upload. For instance, you could add it to a special table in the DB. Then, when the “thing” with which the uploaded file is associated is saved, do 2 things:

  1. Remove the flag from the file to keep.
  2. Run the Delete a list of things workflow action on the uploaded files you wish not to keep.

Alternatively, you could not worry about clean-up at the time the thing is saved, and instead do just number 1 above and also have a clean-up workflow which runs whenever a user logs in. That workflow would delete any “flagged” files.

You could also create an API workflow as you referenced in your last post. Or you could use some combination of all 3 approaches.

Sounds like you’re on the right track though.

Hi @shot,

Thanks for the help, really detailed explanation. Ok, no I understand what it means when a file is considered “orphaned”. Great!

I have one problem though. Currently I’m saving the file url’s onto a list in another data type with other info. I understand that if I want to make this work I have to save the file names in some other data type (Eg. Files) to be able to flag the files. Is this correct? I understand you can’t flag an item on a list, because you can’t add other attributes. Is this correct?

Thanks

Not necessarily, but that might be the easiest way to go about it. Conceptually speaking, by “flag”, I simply mean there needs to be some way to “filter” uploaded files that you wish not to keep. It’s entirely up to you as to how that’s implemented. What’s key is that any file that you do wish to keep must not show up when the “flagged” search constraints are applied.

As outlined in a prior post, implementing a global “Garbage Collection” list is one approach. Just make sure files you do wish to keep do not appear in that list.

For example, let’s say you have a user profile page which allows a user to save information about themself, including a single profile image. A user might change their mind and have uploaded several photos before saving their profile. Each uploaded image could be added to the garbage collection list immediately upon upload. Then, when the profile is actually saved, the currently selected photo would be removed from that list.

I hope that makes sense. (Unfortunately, I don’t currently have an example to which I can provide edit access.)

EDIT: And if you’re dealing with images and wish to conserve your Bubble storage, you might take a look at Upload Buddy. It allows you to scale images before upload, thereby, saving bandwidth, user time, and Bubble storage.

Thanks @shot,

Yes that makes sense. I just need to figure out how to make this scale as I need to add files to different data types and need to figure out the architecture.

I’m also fighting against using the simple file uploader or the multi file uploader. They both have their pros and cons.

I prefer the multi file uploader for the end user experience but I’m struggling to save the log on “input value change”.

Thanks again, I think with your help, now I just need to tie everything together.

Hi again @shot,

I’ve managed to pull it together and it’s working, thanks to you. It really is great.

I just have one more question as it looks like you’ve already been through this. What is the best practice or what do you recommend I do with the “Garbage Collection” log? I use it to delete the files that the user doesn’t end up using when the user saves the form, but what do you recommend is the best practice to clean (delete) the log or of file urls that have now been deleted?

Thanks

Yeah, you need to delete the file from Bubble storage as well as the database entries which point to them.

1 Like

I just realised I have to also prepare this solution to handle when a user comes back and uploads a different file again. I have to flag the file for garbage, that originally was ok.

All I know is that File handling in Bubble is a topic that can be greatly improved. It’s not there yet, we have to do a lot of work for privacy issues, making sure files are not orphaned.

Thanks @shot