Forum Academy Marketplace Showcase Pricing Features

Native File Operations in Server Actions

Motivation

When processing data in a Server Action you will inevitably want to either read or write files within the Bubble service. In this short article I will explain how to use thecontext.request function to read and write files in a Server Action.

Background

As noted in the documentation, within an application, files are saved to an application specific flat store on an Amazon S3 instance. Bubble’s choice in file storage architecture is affordable, and scales well. However, file operations now have to be done through API GET and POST requests; instead of through the Node.js native fs module. While there are many Node.js modules for handling API requests, they are all asynchronous and would require wrapping in the context.async function. Instead following the note in the documentation we will use the context.request function to handle the asynchronous API request, without wrapping it in a context.async call.

Fortunately the context.request function exposes the, now deprecated, Node.js request module . The salient argument to pass to this function is the options object. Both GET'ing a file from a URL, and POST'ing JSON are common place and many examples of how to structure the options object can be found by Googling.

File Reading

To read a file stored in an application we have to retrieve it from the Amazon S3 URL that Bubble provisioned. The critical piece of knowledge is that to receive binary data we have to set the encoding to null, and the Content-type to application/octet-stream. Assuming we have passed the file into the Server Action through the field file, within the Server Action function we would create an options object and pass it to the context.request function:

// Chant the incantation to retrieve the file
var options = {
    uri: properties.file,
    method: "GET",
    encoding: null,
    headers: { "Content-type": "application/octet-stream" }
};

// In memory UInt8Array of the file
var filebytes = context.request(options).body;

As noted in the Server Action plugin page the Bubble file type is automatically coerced to a string URL. So properties.file contains the URL to the Amazon S3 resource.

File Writing

Writing a file from a Server Action is a bit more devious. Following the documentation on sending data to the Bubble API we have three major steps:

  1. If the file is to be stored privately resolve the unique identifier of the thing to attach the file to.
  2. MIME/Type Base 64 encode the bytes of the file.
  3. Wrap the base 64 encoded string in JSON, along with the API URL, file name, private flag, and attach to thing’s unique identifier.

Unfortunately the documentation incorrectly indicates that the sending file name should be written to the field filename when it should be written to the field name. It took digging through the code of the EzCode File Uploader to figure that out. As well, if the file is not private then the private and attach_to fields cannot be present in the POST'ed JSON at all.

Assuming we have passed the application home URL in the field homeurl, the file name in the field name, the privacy flag in the field private, and the thing to attach to in the field attachto we have:

// In memory UInt8Array of the file
var filebytes = ...;

// File is private and attach user specified
if (properties.private && properties.attachto !== undefined) {
    var payload = {
        name: properties.name,
        contents: Buffer.from(filebytes).toString("base64"),
        private: true,
        attach_to: properties.attachto.get("_id")
    };
}

// File is private default to current user
else if (properties.private) {
    var payload = {
        name: properties.name,
        contents: Buffer.from(filebytes).toString("base64"),
        private: true,
        attach_to: properties.currentUser.get("_id")
    };
}

// File is public
else {
    var payload = {
        name: properties.name,
        contents: Buffer.from(filebytes).toString("base64")
    };
}

// Chant the incantation to send the file
var options = {
    uri: properties.homeurl + "fileupload",
    method: "POST",
    json: payload
};

// Upload and store url
var fileurl = context.request(options).body;

Note that to access the file upload API we had to append the "fileupload" string to the application home URL.

Epilogue

The asymmetry in file operations is an interesting side-effect of the asymmetry in the design of client-server architectures. We can download files in raw binary because the client is receiving data from only one place and thus “knows” what it is expecting. On the other hand the server handles many requests from many clients and does not “know” what to expect. Thus the server has to be passed data in a “safe” manner. Hence the base 64 encoding of uploads.

10 Likes

I suspect the base64 encoding requirement was simply a design decision made by the Bubble engineers - perhaps related to the fact that JSON was chosen as the data interchange format, and it’s a text format by definition (hence the need to convert from binary to text). I’m not aware of any inherent technical limitation in “client-server architecture” that prevents binary uploads.

-Steve

Thank you Steve for highlighting the short comings of my epilogue. I was attempting to provide a colloquial explanation of RFC 7231, specifically section 3.1.1.5 on content-type, and the assymmetry between client provided content-type in section 4.3.1 on GET requests (1) versus servers expecting content-type in section 4.3.3 on POST requests (2). Particularly within the context that content-type: application/json has evolved to become the defacto standard for querying server APIs; which in turn requires serializing byte data.

Hopefully you won’t exhaust yourself on the never ending supply of people who are wrong on the internet.

  1. Roughly the content-type in a GET says “when you send me data I will interpret it as…So be advised when preparing your response.” (clients discretion).
  2. Conversely the content-type in a POST says “I’m sending you data that I interpret as…Feel free to reject it if that is not what you want or can safely handle.” (servers discretion).