This is very hard to test since it involves a race condition.
In backend API workflows, I want to know if the workflow is guaranteed to complete before it fires another workflow.
Example: a workflow (perhaps recursive) makes some heavy calculations and writes a lot of changes to the database. Then it schedules another API which tells the frontend that the values have been updated and can now be used.
Will this work? Or will it start all the calculations and then fire the API even if some of the calculations may have been delayed?
It might happen for the workflows to run too fast and mess up the calculation. You can delay the next scheduled api date by a few seconds.
If you update your bubble app to the latest version you’ll have access to a more advanced way of doing calculations.
Lastly you could have a custom plugin built for this using math.js to handle the calculation.
I know you run a charity so I’m happy to offer you this for free. Message me if you want this option, depending on complexity we might just keep it simple without a plugin.
Thanks. I’m trying to track down this information to teach our students a deep understanding of how Bubble works. Building a plugin isn’t really our goal here!
Basically we’re trying to figure out how to do a promise or a callback in Bubble for situations where we need to ensure a task completes before completing a next step. Trying to guess how long the task will take and throwing a delay in there isn’t really a great solution.
Also, this is part of our strategy to clean up “spaghetti code” (or spaghetti no-code?). For example when there are lots of factors in a calculation, an expression can get truly huge, and a single tiny change can break the whole thing or take forever to rebuild. By breaking the huge expressions down into smaller steps, we make it more maintainable (e.g. subtotals for each section + discount for each section + tax). But the problem with that is that calculations need to be done synchronously.
In some previous threads I have caught some flak for trying to figure out how to do things synchronously, but the fact is that there will always be valid use-cases for synchronous operations.
It’s also really important for all Bubble developers to have a good understanding of how the workflows in all areas work. For example, whenever I see anyone talking about self-recursive backend workflows, the assumption is that each step of each iteration of the workflow will happen after all the steps in the previous iteration. But it seems that that may not be guaranteed, which could lead to some incredibly confusing problems if you’re not aware of that possibility.
The issue is that the subtotals and discounts are themselves the result of many calculations which also need to be done synchronously. Passing them in this way runs the risk of doing calculations on old or only partially updated numbers.
Within a workflow, ensure your steps are executed right after the other by passing a result of step A as parameter for step B - whatever the value - it solved some concurrency issues for me.
Once you call a backend API workflow, assume you have no control what happens afterwards. If you can put everything in a single workflow apply the first point. If impossible, control the order of execution of each backend workflows via database triggers.
I actually am trying to get clarity on this point. The docs says that’s safe if the earlier step is “create a thing.” I’m still trying to figure out if using a result from an earlier step that’s merely making changes to a thing guarantees synchronicity.
I mean, yeah, I could. But seriously I feel like there must be ways within the system to control the order of operations.
Yes, if later backend workflow steps reference an earlier Make Changes to a Thing step, those later actions will happen after Make Changes (I’m mobile ATM) because (if I’m not mistaken) Make Changes returns its changed result to the workflow, yes?
This is easier to test than ask to ask about, right?
You might also be able to do things like use the Flow State SSAs from my List Popper and Friends to further enforce synchronicity in backend workflows that might otherwise run ahead, but I’ve not tested that recently. (Flow State is a minimal SSA that simply takes an expression and echoes the result to its output.)
Basically the synchronicity of a backend workflow depends on what you’re doing.
Race conditions are notoriously hard to test. If I test 50 times and get the same result, does that mean that it’s guaranteed to work that way? Or does that mean it has a 1/51 chance of working the other way? What if I can’t get the server to replicate a condition that causes an operation to hang for 400ms? I’m genuinely open to answers here.
I still don’t get if your question is theoretical or practical. If it’s theoretical, the answer is, “it depends”. If it’s practical the answer is also, “it depends”, but at least you can test variations in your actual workflow(s).
In general, if some workflow step returns values to the workflow and then all subsequent steps refer to prior steps sequentially, such a workflow will be synchronous.
This is to say:
Step 3 depends on result of Step 2 depends on result of Step 1 will be synchronous.
If there’s a Step 4 that depends on nothing, and nothing else depends on Step 4 there’s no reason for Step 4 to wait for anything.
I should say: this should only concern us if we are, at the end of the workflow, kicking off some other workflow. But since we have full control over that (they are custom and we must define them after all), we don’t need to be concerned, right?
Because if we do want to kickoff some “next” workflow, well, we can design that workflow to take the results of all steps as inputs, even if that workflow does not reference (make use of) the value passed from Step 4.
If Step 4 is a native Bubble operation that returns values to the workflow, or an SSA that does the same, the “next” workflow will not be called until all of the Step operations have completed (because the call to it depends on ALL of the subsequent Step action results.)
So I’m now thinking about your question as “how many angels fit on the head of a pin?”
To put a finer point on this: Basic flow control in Bubble backend workflows:
Let us say there is some event (Event) that triggers 2 different backend workflows (Workflow 1 and Workflow 2). And now we want to know, which completed first? Well, we can’t, right?
So if you need to know when some workflow completes (like we want to kick off Workflow 3 after Workflow 1 and Workflow 2 are both complete) well, we can’t have Workflow 1 and Workflow 2 both firing on Event. We need to pick one and do it first and then do what we want second, and then what should happen third.
(But to point this out is completely hypothetical as how else would we get Workflow 3 to fire anyway?)
I know I’m sorry I am probably being a pain. It’s because this particular issue has been a real pain to me many times over. I have probably wasted 100 hours dealing with issues of synchronicity.
It’s theoretical in the sense that I want a general understanding of the theory of how to do this. But it’s practical in the sense that I’ve run into real issues over and over with this.
Some of those issues I solved by taking a fairly simple strategy that depends on synchronous events and changing it to a spaghetti mess that’s insanely difficult to maintain. Some I’ve solved by changing my data structure. And some I have not yet solved and I haven’t yet had the energy to try to explain the full complexity of what I’m dealing with on this forum.
The current issue I’m having is a checkout cart where you update a bunch of line items, apply discounts to some of them, then tally them up into subtotals, tally the subtotals to calculate tax, then tally all subtotals plus tax. And some of these line items depend on an external API. I’ve re-built this particular cart from the ground up three times. Gotten better each time but still extremely cumbersome to maintain.
@keith what you have said has actually been a huge help, and I really hope it’s true. If a step that depends on the result of a former “make changes…” event is guaranteed to run after that former event completes, then I can do a lot with that.
I think you’re also saying that in a backend workflow, things are not guaranteed to fire in sequence, e.g.
API Workflow A
Step 1 - Change current_iteration to current_iteration + 1
Step 2 - Do a calculation based on item#current_iteration of a list…
…
Step 20 -
Schedule API Workflow A in 3 seconds, pass current_iteration.
If steps 1-20 ever take more than 3 seconds, then there’s a chance the next iteration will start running before they finish. …unless I can somehow get the steps to all use a result of a previous step.
Let’s drill in on one concrete example. This is a small part of what I’m working on now. This is frontend workflows. I had initially refactored everything to use backend workflows, but that was a failure, so now I’m moving it all back to the frontend where I can build these big expressions* that eliminate the need for synchronicity but they also just keep getting bigger the more features I add.
(*Example of a big expression. This is only adding two subtotals right now. I expect to have a dozen subtotals soon. Not all visible to the user at once, but a dozen that I need to potentially be calculating. Yes, I could break the page out into multiple, but then I’m maintaining several nearly identical pages…)
Problem: step 3 is dependent on a result from step 1 but not step 2. Step 2 may or may not fire, so step 3 can’t depend on step 2. Potential solution 1: make a copy of step 3, (step 4) and give it the same condition that step 2 has, and make step 4 depend on step 2. Problem with that solution: messy and complicated to maintain.
Actual solution: Add a step 4 which takes the subtotal from line 3 and conditionally adds the one-off discount to the subtotal.
Next problem: how can I know when it’s safe to use line_delivery_subtotal to calculate tax? I can’t wait for a change to line_delivery_subtotal because it might be changed twice. I can’t depend on a result of step 3 only, because step 4 might fire. I can’t depend on result of step 4, because step 4 might not fire.
This situation is not obvious. Just like you, I spent a lot of time adjusting this synchronization of workflows. I even discovered bugs in Customs Workflows (it’s another nightmare). I didn’t have time to report to Bubble too busy finding alternative solutions. With the code you showed, I suggest you stay in the Backend Workflows. I use a lot of Terminate and Steps to finally succeed in doing complex things and in all synchronization. If you are using external API and failed, all workflows will stopped. Take that in consideration.
(Note: don’t go looking for the example pages that I may or may not have linked to. Those pages are surely modified in the intervening years as they’re part of my private playground. Just rebuild them yourself. What we’re discussing here is not complicated in the slightest and you’ll get my point(s).)
Now, there are several other Action plugins in List Popper and Friends beyond List Popper itself, and these are quite useful (they are also open source so you can fork and modify them as you see fit) like the aforementioned Flow State SSA actions (these are stupid-simple as they just evaluate an expression and forward it to their outputs, making them available in the workflow, but as I said previously this might be just the thing you need to enforce synchronicity in certain contexts).
Note that there was a bit of hubbub somewhere around 36 hours ago where these Actions stopped working properly due to a “smarty-pants” change on Bubble’s side. But that’s Bubble for you… that sort of thing happens all of the time. Whaddyagonnado? (Gory details here.)
Everything you’ve said seems spot on from where I’m sitting and I’m in exactly the same boat. I’ve also spent hundreds of hours debugging problems that turned out to be random race conditions or due to some APIs being intermittently slower that one time, breaking order of execution between backend workflows.
This makes it impossible to refactor and extract utility pieces of functionality into separate workflows. For example, I want to do some auditing and house-keeping around calling an API (ie audit log, track usage, add analytics etc). I would love to extract these few steps into a standalone workflow called “Call X API” so that I can reuse it multiple times from other backend workflows. But I can’t because the other steps won’t wait for this step’s execution to finish, even though I’ve configured the dependence in the workflow. Even worse, it seems that neither the IDE or the logs warn you of this…
I’m from an engineering background and frankly find it bonkers that anyone can get anything running consistently in an environment where we have backend workflows that “might” operate in order, sometimes, based on some arbitrary distinction between step types.
I also have an engineering background, maybe that’s why we are both annoyed by this when others don’t seem to be!
I did discover a trick that is a massive help at least for frontend issues: do calculations in the “Data source” field of groups. This took someone 2 seconds to show me and it has been incredible. If you’re working with numbers, you just have to set the Type of Content to “Number.”
The power of this is that you can put full expressions in the Data source field of groups, and Bubble seems to be pretty good at working through all the calculations and get it right.
So you can effectively make each group you use do calculations using any expression you want. Then, you can refer to the value of that group as a variable in other expressions. I believe Bubble automatically updates the values of any expression that uses the value of a group that gets updated.
If you wanted to do this:
A = API_Request();
B = A * Another_API_Request();
C = A + B;
D = C * tax_rate;
E = C * B * A / D;
You’d make a group for A, B, C, D, and E, and set their data source as the equation, and it works out great.
Group A Data Source: API_Request();
Group B Data Source: Group A’s value * Another_API_Request();
Group C Data Source: Group A’s value + Group B’s value;
Group D Data Source: Group C’s value * tax_rate;
Group E Data Source: Group C’s value * Group B’s value * Group A’s value / Group D’s value;
