Private files processing

DRRR · 2025-02-04T20:45:22.884Z

Hi All,

I’m always struggling with Bubble’s private file setup.
Once files ar in Bubble’s (AWS’s) S3 with ‘private’ = yes, it is really difficult to do something with these files in terms of processing.

The challenge (and my issue )
I have a series of small files that i need to put into a zip container server side.

I managed building the zip file functionality (plugin) server side and setting up the appropriate folder structure to generate the zip file (all seems to work) but somehow I’m not able to access the files that i created (.json) which I created server side and saved as private.
The fact that they are attached to Current user and the current user should be allowed to view files attached to them (based on the privacy rules implemented) doesn’t seem to give access in the workflow.

When I on the other hand perform the create ZIP workflow on a bunch of files that I have stored without privacy rules there is no issue and the zip file is created including the content.

Things I have tried (but that don’t seem to work):

Bypass privacy rules (only work on data but still…)
add an append with “?api_token=xxx” paramater (which worked with sending out files to an api but not in this script (puzzles me but anyways).

It may be that I made an error in the code (I was also able to create attach to objects combining the pointer Type and the ID of the object in the attach_to so there are probably some other things i’m capable of breaking as well…) but more likely i’m just doing something stupid

The only way to bypass the issue that would likely work is generate a presigned url for every file which is then added into the .zip file… this however is a repeating nightmare…

Curious if anyone of you has a good suggestion to solve this!

Thanks already!

DjackLowCode · 2025-02-04T21:18:43.631Z

Have you tried this plugin?

Bubble

Templates for Bubble

Buy and sell Bubble templates to speed up app development. Bubble is a visual programing language that lets you build apps without code.

DRRR · 2025-02-04T21:25:06.406Z

Thanks! It definitely is one of the best plugins out there!

The only thing is that it is probably quite a bit of work to configure this workflow when i’m passing a variety of up to 120 files into the zip, while every file is a variable (to be included or not) depending on certain user characteristics.

I’m now actually trying to understand if i can integrate parts of that functionality into this one script via promises. But i’m pretty far out of my comfortzone creating and reading javascript


function getSignedUrl(originalUrl) {
    return new Promise(async (resolve, reject) => {
      try {
        let signedUrl = originalUrl;

        // Ensure the URL starts with http/https.
        if (!signedUrl.startsWith("http")) {
          signedUrl = "https:" + signedUrl;
        }

        // Attempt HEAD request with the Bearer token.
        let response;
        try {
          response = await axios.head(signedUrl, {
            maxRedirects: 0,
            validateStatus: (status) =>
              status === 302 || (status >= 200 && status < 300),
            headers: { Authorization: `Bearer ${bubbleApiKey}` },
          });
        } catch (authError) {
          // If the authorized HEAD fails, try again without any Authorization header.
          try {
            response = await axios.head(signedUrl, {
              maxRedirects: 0,
              validateStatus: (status) =>
                status === 302 || (status >= 200 && status < 300),
            });
          } catch (unauthError) {
            return reject(unauthError);
          }
        }

        // If we get a redirect location, that’s our signed/redirected URL.
        if (response && response.headers && response.headers.location) {
          resolve(response.headers.location);
        } else {
          // Otherwise, the URL itself is valid for direct GET.
          resolve(signedUrl);
        }
      } catch (err) {
        reject(err);
      }
    });
  }

but i’ts a bit like i’m blindfolded throwing some darts around hoping to hit something

DRRR · 2025-02-04T21:26:42.913Z

This is the script I created and now try to refine:

function(properties, context) {
  const JSZip = require("jszip");
  const axios = require("axios");
  const FormData = require("form-data");

  // 1) Create a new zip instance.
  const zip = new JSZip();

  // 2) Read and validate input properties.
  const folderStructure = properties.folder_structure || "{}";
  const outputFileName = properties.output_file_name || "output.zip";
  let fileUploadBaseURL = properties.bubble_website_url ? String(properties.bubble_website_url) : "";

  // Validate that the upload base URL starts with "https://"
  if (!fileUploadBaseURL.startsWith("https://")) {
    throw new Error(
      "Invalid file_upload_url (must start with 'https://'). Provided: " +
        fileUploadBaseURL
    );
  }
  // Remove any trailing slash.
  fileUploadBaseURL = fileUploadBaseURL.replace(/\/$/, "");

  // 3) Parse the folder structure JSON.
  let folderData;
  try {
    folderData = JSON.parse(folderStructure);
  } catch (err) {
    throw new Error("Invalid JSON for 'folder_structure': " + err.message);
  }

// 4) Recursive function to add files (or folders) to the zip.
function addToZip(currentFolder, data) {
  let promises = [];

  for (let key in data) {
    const value = data[key];

    // If the value is a string…
    if (typeof value === "string") {
      // …and if it starts with "https://", download it.
      if (value.startsWith("https://")) {
        const p = axios
          .get(value, { responseType: "arraybuffer" })
          .then(resp => {
            // Log download info.
            console.log(
              Downloaded "${key}" from ${value} with ${resp.data.byteLength} bytes
            );
            // Add the downloaded file to the current folder in the zip.
            currentFolder.file(key, resp.data, { binary: true });
          })
          .catch(err => {
            console.warn(⚠️ Failed to download "${value}": ${err.message});
          });
        promises.push(p);
      } else {
        // Otherwise, treat the string as plain text content.
        currentFolder.file(key, value);
      }
    } else if (value && typeof value === "object") {
      // If the value is an object, treat it as a nested folder.
      const subFolder = currentFolder.folder(key);
      // Recursively add the nested folder's contents.
      promises = promises.concat(addToZip(subFolder, value));
    }
  }

  return promises;
}

  // 5) Determine the attachment parameters.
  let thingToConnect = properties.thing_to_connect;
  let attachType = null;
  let attachID = null;

  if (thingToConnect && typeof thingToConnect === "object") {
    if (
      thingToConnect._pointer &&
      thingToConnect._pointer._id &&
      thingToConnect._pointer._type
    ) {
      attachType = thingToConnect._pointer._type;
      attachID = thingToConnect._pointer._id;
    } else if (thingToConnect.datatype && thingToConnect.unique_id) {
      attachType = thingToConnect.datatype;
      attachID = thingToConnect.unique_id;
    }
  } else if (typeof thingToConnect === "string" && thingToConnect.length > 5) {
    console.warn(
      "⚠️ Received a plain string for thing_to_connect, but no type info. Not attaching."
    );
  }

  // 6) Build the file upload URL.
  let fileUploadURL = ${fileUploadBaseURL}/fileupload;

  // 7) Download all files, add them to the zip, and then generate the ZIP buffer.
  const promises = addToZip(zip, folderData);

  return Promise.all(promises)
    .then(() => zip.generateAsync({ type: "nodebuffer" }))
    .then(buffer => {
      // Create a FormData instance and append the zip file.
      const form = new FormData();
      form.append("file", buffer, {
        filename: outputFileName,
        contentType: "application/zip"
      });

      // Attach the file to a Thing if a valid attachment was provided.
      if (attachID) {
        form.append("attach_to", attachID);
        form.append("private", "true");
        console.log(🔗 Attaching to: ${attachType} (ID: ${attachID}));
      } else {
        console.warn("⚠️ No valid Thing detected. File will not be attached.");
      }

      // Upload the ZIP file.
      return axios.post(fileUploadURL, form, {
        headers: form.getHeaders()
      });
    })
    .then(resp => {
      // 8) Process Bubble's response.
      const data = resp.data;
      let fileUrl;

      if (typeof data === "string") {
        fileUrl = data.trim();
      } else if (data && data.url) {
        fileUrl = data.url;
      } else {
        throw new Error("File upload failed. Unexpected response: " + JSON.stringify(data));
      }

      // Fix protocol-relative URLs.
      if (fileUrl.startsWith("//")) {
        fileUrl = "https:" + fileUrl;
      }

      // Return the file URL in a way that Bubble expects.
      return {
        zip_file_url: fileUrl,
        zip_file: fileUrl
      };
    })
    .catch(err => {
      throw new Error("Error generating ZIP file: " + err.message);
    });
}

DjackLowCode · 2025-02-05T00:50:05.924Z

@DRRR good luck! Just be aware not to add keys client side actions.

system · 2025-04-15T20:45:26.184Z

This topic was automatically closed after 70 days. New replies are no longer allowed.