Dear George
Thank you very much for the hands-on guide of how to setup a file system.
I’m also building one for my app and run into a problem with privacy, and wanted to ask you how you approached the issue (or didn’t as you might not be overly concerned with the privacy of the actual document uploaded)
Quick brief of my application: We’re a multitenant SaaS application for construction managers to procure contractor services (planned to launch in October, fingers crossed ;-)).
Our users should have a file system available in which they can upload files and share them within their team automatically, and with tenders based on an invitation.
About the problem:
When uploading a document, I have either the option to upload it without attaching it to a database entry (not private, anyone with the url of the file could view / download the file) or I’m making it private by attaching it to a database entry of ‘file’ with applied privacy rules.
With the first option I see some risks of files of customers and or contractors unintentionally leak into public as with the link, anybody could download it. Or would you say the risk is negligible? Im not quite sure how the S3 instance of bubble is setup in terms of protection from datascrapers that could find the files without access to my tool… perhaps you can shed some light on this?
With the second option, I will not be able to process the file any further within the backend of the app or sending it to APIs of other programs to process. To solve this I came up with the following (non working / not very good) solutions:
Just saving the file again to the file manager without privacy rules for the processing time of the backend workflow and then deleting it again. This - at least based on my trial runs - is not possible as files cannot be processed in the backend because of the limitations mentioned above.
Store the base64 value of the document within the ‘file’ Thing and then restoring it for further processing. This is feasible but it puts a hard limit of 6mb to uploaded files as no bigger base64 string can be processed in the backend due to the hard limits of bubble (I tried to chunk up the base64 and save to database and then retrieve and concatenate during processing but in the log an error is thrown Workflow error - Plugin action inputs too large, should be less than 6 MB (I’m using the plugin Bubble Back End Utilities but it seems that this hard limit applies to all backend processes))
Imo choosing between one of these options – doc = private with severe limitations in backend processing a/o file size vs. doc = not private – is like choosing between pest and cholera.
Do you have any further insights on the topic that could potentially ease up my hesitation of going with the non private document route or some funky solutions for private files processing in the backend?
And to your point 4. to delete all Downstream relatives of a folder. Couldn’t you just construct the workflow so that the first action is to schedule deletion on a list of all child folders (this is now possible with schedule workflow on a list of things) and then delete the current folder and it’s files?
I think it would be much simpler. Alternatively you could construct a temporary Thing like ‘Deletion Queue’ in which you add all files and folders to be deleted and when reaching the lowest hierarchy level, and all workflows of adding files & folders on that hierarchy level were run, schedule a delete a list of things for all folders & all files in there. It would require a system where you have an upstream counter when adding folders, and a downstream counter to identify how many folders there are on the bottom that you add to while running the deletion workflows and then, when the folder is at the bottom of the hierarchy, added its files & folder to the Queue and no other folders need to run a workflow on the bottom, you can schedule the delete of the Queue
How curious - the application I used as an example here is a multitenant SaaS for construction companies to write project tenders and respond to the government with bids.
The TLDR is we don’t use Bubble storage, we use Wasabi, which also comes with the advantage of us being able to store the data in the EU.
I have not found a way to retrieve a list of files programatically from a Bubble app if it’s not stored elsewhere in the DB. That is to say, if you just uploaded a file with a random name in the file manager from the editor, I’m not sure how anyone could find it other than by brute force and trying all possible names - that said, I wouldn’t take any chances.
So, that’s the solution - don’t use Bubble - it’ll be better and cheaper in the long run.
Not quite. I’ve changed the logic a bit since then but A has a list of children B, and each B has a list of children C.
To delete A, we need to delete a list of Children, but SAWOL will only delete Children B and not B’s children… The actual expression would be:
File’s List of Children:each item’s List of Children:each item’s List of Children to infinity as files can be infinitely nested.
There’s probably a decent way to do this that involves triggers and is less complex. Have a ‘deleted’ field on the Folder. A trigger that when Folder is deleted, schedules API workflow delete Folder.
That API workflow deletes the folder, and its contents, and sets the Children’s deleted to yes. That will cause the next trigger for the children to delete the children, and so on, until all children of children etc have been deleted. Note that you have to use an API workflow to do this rather than doing it in the trigger directly as triggers can’t trigger other triggers.
Very cool, hope you’re doing fine with the app! Construction is such a highly underdigitzed industry it’s crazy.
So for the workflow it should be possible to do as follows:
Schedule workflow where ever you want in the app on one folder. Input is ‘folder’
Action1 : Schedule this workflow on all child folders of this folder (only when folders-child:count > 0)
Action2: DeleteAllFiles in Folder
Action3: Delete the folder
As soon as you reach a folder that has no children, it doesn’t reschedule the workflow. Should be possible like this or am I missing something?
And thank you for the tip with wasabi, will check it out.
Any inputs on how to apply the same privacy rules as within the bubble app to the documents themself?
Best
Florian
