Example: If files are related to a Project. Attach the file to the appropriate Project record.

Create a privacy rule on the Project data type with an expression that looks something like “Only when current user’s role is Admin” > enable ability to view attached files.

So, if you attach it to something all users can access in general, then you can create a specific privacy rule to restrict file access to admins only.