Optimisation process for Backend workflows

Hi team, in my app there are 15 data types, and each data type contains multiple tables. The app also has several very different workflows that create or modify things. I want to implement logging.

Should I create just one backend workflow to handle all logs—is that enough? Is it even possible? Because there are many workflows and many data types, I’m completely lost on how to approach this. Or, if there are for example 20 workflows, do I need to add 20 actions to create logs?

Please help, my head is about to explode.

Just create a single dataType for a log. Then each of the fields should only be text. Pass along data in the backend workflow based on your needs. Does that make sense? One place makes it, but you pass along the data you need at the end of each workflow. That’s just one idea. I’m sure there are more ideas out there. :blush:

Hi, thanks again for your reply :folded_hands:

I think my main concern comes from concrete cases in my app.

For example, when I do a “create a new task”, multiple fields are sent to the database at the same time:

  • first name

  • last name

  • date

  • description

  • and other fields

So a single action creates multiple values at once.

That’s why I’m wondering:

  • Should I log all these fields every time?

  • If yes, how should I structure that properly if everything is stored as text?

  • And most importantly, how do I keep it clean and readable for audit purposes?

In my case, it’s not just a small change—
it’s often a full set of data being created or modified in one action.

That’s what confuses me:
I’m not sure if your approach (using only text fields) is enough for this level of detail.

If you have a concrete example for this kind of scenario (creating something with multiple fields), I’d really appreciate it :+1:

So if this were for auditing purposes this is how I would set that up:

devLog

Text 1 - ‘Create a new task’
Text 2 - ‘First name, last name, date, description, other fields.’

Does that make sense? That’s how I would do it if it were just logging purposes. Any fields I need to pull in directly would just be a new text field added to the dataType.

You could use database change backend workflows that produce thing now and thing before change, to have all fields from before and after. Set it up for each data type. When it runs you can save the before as JSOn and put onto a text field of log data type, and the now as JSON in another text field on a log data type.

The approach @J805 described works well here. You don’t need a separate backend workflow for each of your 20 workflows — just one reusable backend workflow that accepts parameters and creates a log entry.

For your “create a task” example, you’d call that single logging workflow at the end of your task-creation workflow and pass in whatever you want to capture — you can serialize all the fields into a single text field (e.g. “First name: John, Last name: Doe, Date: 01/01/2025”) or create individual text fields on your Log data type for each piece of info you care about.

What I’d suggest for your Log data type:

  • Workflow name (text) — which workflow triggered it
  • User (User type) — who did it
  • Timestamp (date) — auto-set to current date/time
  • Details (text) — a summary of what was created/changed

Then every workflow just has one extra action at the end: “Schedule API workflow > Log” with the relevant data passed in.

So yes — 20 workflows, but only 1 logging backend workflow. Each workflow just calls it with different parameters.

@atsouisaac03 couple of things to keep in mind between two different proposed approaches.

  1. Workload Units - If you have one backend workflow that you run the action to call and schedule to make the log in the database, you have the cost of 0.7 WUs per action to run the schedule and to schedule (even if for current date time). If you use database trigger changes, you have a charge of 0.05 WUs per database trigger change.
  2. Convenience - If you use database trigger changes, in backend of your app you create a folder, label it database trigger changes, and add each one for each data type there. This will run whether the data type is CREATED, UPDATED or DELETED. If you use a schedule backend workflow approach, you need to remember to run that action to schedule on every create, update and delete to the data type anywhere in the app (do you have separate areas that manipulate the data, like an incoming webhook, user vs admin dashboard?)
  3. Update Simplicity - If you add or remove a data field to a data type, you go to one database trigger change for that data type and change the JSON structure. If you use a schedule backend workflows you go to every area of the app that is scheduled for that data type, which if it is not a simple app, could be multiple places.
  4. Feature Lock - Not necessarily lock, but annoyance. If you want to use autobinding on inputs for a particular data type, you’ll need to set up additional workflows for the schedule of a backend workflow to do your log for each input that is using autobinding. If you use the database trigger change approach, you have no extra work to do.
  5. Simple Overview of App - If you use database trigger change, one area to look at where all logs are running from, and how it is storing the before and after versions of the data in event have any issues, or bring on additional devs or just want a reminder. If you use backend workflow schedule from all over the app, it is a bit more complicated.
  6. One Workflow - When using the database trigger change approach, you are also just using one backend workflow, except it is not an API workflow you have to schedule from everywhere throughout the app, it is instead a custom backend workflow you trigger from the database trigger changes. So, still one simple action in backend that takes the details.

There are always of course going to be various approaches to how people like to do things. I always look for the cheapest (because less WU costs equals more efficient) and easiest to manage and grow the app.

Also, use of JSON for storing Key Value Pairs as a text field, in my experience is far superior than just a text string with simple data values (I could never always remember what each value was for, nor how to make it easy to know a value in list was empty so all entries were consistent and uniform). JSON is easy to understand after an hour, and multiple free plugins on market that make it easy to work with after it is stored in DB.

Our logging system is very simple but works well for our needs…

One table for the logs, containing one description text field & date.

After any activity worth logging we simply call the backend action that writes the log. A description would be along the lines of " user xxx has downloaded file filename" or “user xxx has requested a password reset” etc. get as detailed as rewuired.

Each night this log is emailed to Gmail (using formatted as text). and then the database wiped clean. Sounds strange I know … But Gmail becomes the database… Cheap, platform-agnostic and super fast to search by any of our team, anywhere, on any device. It also adds the legal-validation timestamp.

Not great for detailed stats & queries but as a legal record, crude & effective. :sweat_smile:

Why not just send email immediately instead of logging into DB, then scheduling time for each night and then deleting from database?

That’s certainly another simple approach for sure :+1:t2:

Ours includes a number of summaries for that day too…number of sales, downloads, new users, etc etc etc … a daily log , no frills but we like it. :slightly_smiling_face:

This approach is also very useful, particularly for saving

If you know of any JSON tutorials that you think might be helpful, let me know. I’ll do some research on my own as well.

The JSON suggestion is a good idea. Makes the values more accessible to you.

Use AI to create JSON for the fields you want and then use a JSON validator to check that it’s correct for you.

Then drop in the dynamic data in the places that you need them.

One hint. Use ‘JSON safe’ for the dynamic data and then remove those quotes so it doesn’t give you errors.

Hope that helps. :blush:

I don’t know of any JSON tutorials, I have not yet put one together for Bubble use. General concepts are to understand structure as simple ideas. You have Key Value Pairs like

“name” : “Roger Dodger” which is basically the Key of name and Value of Roger Dodger, just like a bubble data field is the key and the value is the value of the field in the DB.

The basic structure is every key needs to have double quotes, but not every value, as that is dependent on the type of data it is.

If the key value is supposed to be text, it needs double quotes and often is referred to as string. If the key value is supposed to be a number, there are no double quotes and is referred to as integer.

“width”: 150

To start the JSON, you have a { and to end a } with a comma between each key value pair

{

“name”: “Roger Dodger”,

“width”: 150

}

What that creates is referred to as an Object. JSON can be an array of objects, basically a list of things. To start an array of objects you use [ and end with ] and each object in the array is separated by a comma.

[

{

“name”: “Roger Dodger”,

“width”: 150

},

{

“name”: “Suzy Que”,

“width”: 215

}

]

If you want to make the most out of some of the JSON tools in Bubble you can add each array an ID key value pair, and to reference the Bubble thing keep the unique ID as a separate key value pair…but that is just if you will want to use advanced tools to do more with the JSON later.

Additionally, other data types like dates, should be stored as ISO or timestamp. ISO is better as Bubble will convert it automatically to an actual date if you will use your JSON in the API connector or something more advanced. For dates use double quotes around the value. So a dynamic expression if you are storing a date would be this things date field formatted as ISO formatted as JSON safe.

The Bubble operator of formatted as JSON safe means you do not need to use the double quotes, so something like name of Roger Dodger can be this things name formatted as JSON safe.

One thing to keep in mind, is that true to form, Bubble went half way with the formatted as JSON safe operator and didn’t make it work properly for numbers. You see, when a value is a text field, JSON safe, if it is empty will just have empty double quotes which JSON understands as an empty value. But Bubble went half way, and if the field value is a number and you use formatted as JSON safe, an empty value doesn’t show up as null as it should and so you can run into issues with JSON tools later.

Correct approach for number fields is to use an expression like this things number field is empty (yes value = null, no value = this things number field value), so that you get proper JSON for an empty number field.

For any related fields, use the related things unique ID as the value of its Key. For any option set values use the options display value formatted as JSON safe so that if you choose to do any advance use of JSON bubble will automatically pick up the correct option value.

If the field and corresponding key should be a list your JSON would be something like

“list_field“: [each items value formatted as JSON safe] when using each items value formatted as JSON safe, bubble will automatically add the double quotes and comma between each item. If you do not include the opening and closing square brackets, it will be treated as a single string value, so basically a comma separated list as one value difference being something like red, white, blue versus [red, white, blue] where the first would be one text value, and the square brackets are three text values.