[WARNING] SAWOL Multi-Write data loss

Tips
1

Dear Bubblers

I’m writing this post to save you the headache and frustration I’ve been experiencing during the final testing phase of my application. After countless hours of debugging and a back-and-forth with Bubble support, I’ve identified a critical issue that might be affecting many of us.

The Problem

When using Schedule API Workflow on a List (SAWOL) to trigger multiple workflows that write data to the same Thing (or same field), some data simply vanishes into thin air. The most frustrating part? The logs confirm that the data was written successfully, but when you check the Thing, the data is missing.

This isn’t a one-time glitch. It happens consistently and reliably at the same action points in my workflows.

What’s Actually Happening

According to Bubble support, this behavior is related to recent “improvements” on the speed and power of scheduling an API workflow on a list. In their words:

“If you have an action that makes changes to the same exact thing in each instance of the workflow, then it can cause complications trying to add different things to the same field at roughly the same time (This race condition only occurs when a ‘Schedule API Workflow on a list’ action schedules an API workflow that contain an action to make changes the same thing).”

In simpler terms: When multiple workflows are simultaneously attempting to write to the same Thing, they step on each other’s toes, and some writes simply get dropped - despite the logs showing successful completion.

Why This Is a Major Flaw

This is, in my opinion, a MAJOR FLAW in the platform, especially when there’s not a big warning in the SAWOL expression tab – moreover when the recommended behavior (no interval) in the tab is causing the issue (according to support). We’re building applications on the premise that when we instruct the system to perform certain actions, those actions will be executed as intended, especially when we cannot control the sequence of actions. This race condition undermines that basic trust in the platform.

The most disturbing part is that there’s no error message. The logs report success, making this extremely difficult to debug. I spent days checking and rechecking my expressions and logic, convinced that I had made a logical mistake somewhere.

The Workaround

Bubble support suggests adding an interval when scheduling API workflows on a list:

“A way to avoid this would be to add an interval to when you schedule the API workflow on the list to give more breathing room between each run of the workflow. We recommend starting with an interval of 2 seconds, and then adjusting based on the results from testing.”

My Implementation

For those facing similar issues, here’s what worked for me:

  1. Added database triggers that check for data inconsitencies and write data if missing (adding major WU overhead to my app)
  2. For critical data operations, implemented a simple queue system to process writes sequentially rather than in parallel
  3. scheduled intended workflow that should happen when data in list A matched data in list B (to check if all workflows are completed) just enough into the future where I’m 99% sure everything that ought to be executed, was executed.

Questions for the Community

Is anyone else experiencing similar issues? Have you found better workarounds than adding delay intervals or without the need to restructure your apps?

It’s really frustrating to have a program not execute exactly what you tell it to do - especially when the logs tell you everything worked perfectly.

Let me know your thoughts and experiences!

4 Likes
2

have you tried a recursive workflow?

3

A recursive workflow in my case doesn’t make sense as the intended user action (1 click) triggers multiple workflows & sub workflows with complex data manipulation and adaption and I cannot control everything as the WFs have different scopes. I have one Thing that tracks all relevant changes and in the end writes everything accordingly. Thx for the suggestion but it doesn’t solve anything in my case.

I just wanted to point out the flaw in the system as this is imo a (nother) severe limitation that you run into with bubble when building more complex apps than something like a To-Do-List app.

This and so many more things really make Bubble nearly unsuitable for bigger, more complex applications which is just really frustrating when you’re locked in.

4

You’re encountering race conditions, which are well documented in the forum.

Why not one thing for each change…?

2 Likes
5

While race conditions are indeed mentioned, I wouldn’t say that there are enough resources from Bubble’s side that guide you on how to avoid everything properly. Acknowledging a problem isn’t the same as properly solving it.

The suggestion to create separate Things misses the point (and doesn’t work in my case which has around 15 downstream workflows that are highly interconnected). This isn’t about finding creative workarounds - it’s about expecting basic reliability from a development platform. Bubble should handle concurrency at the system level gracefully or provide ways where we can control it.

Having to architect your application without extensive documentation from bubble side, and not just random forum posts that you find on the get go when encountering strange issues, imo should be expected at the minimum with the size of bubble team & platform.

Issues arising are;
. hidden bugs that can severely hinder performance & reliability of apps built here.
. workarounds cause another plethora of potential issues to consider and you’re in a sisyphos situation

The reality is that proper concurrency control (transactions, locks, conflict resolution) is a standard feature in mature development platforms. While Bubble offers many strengths, this limitation represents a genuine weakness that impacts reliability and scalability (among many other known limitations).

I appreciate your suggestions for workarounds, but I think it’s just fair to express that this represents a significant limitation that Bubble should address at the platform level rather than expecting developers to build around it (-> or provide better documentation of execution logic etc.)

Sorry, I’m just really frustrated from building on bubble for the past 1.5 years

6

I understand your frustration, and it’s a reasonable criticism. The term “race condition” is not a term that’s familiar to most folks (especially those with a less technical background coming to a no-code platform).

Anyway, in addition to the forum threads, that phenomenon is [now] also described in Bubble’s documentation…

I’m glad you worked it out, and I hope your solution is robust. I personally avoid logic that even has the potential for a race condition. I’d never rely on delay intervals for data integrity, so I disagree with support on that point. :face_with_raised_eyebrow:

6 Likes
7

Race conditions are barely mentioned and there are zeror resources on how to handle it. Out of thousands and thousands of Bubble apps, I’ve never seen one with a Queue or Worker etc. that I didn’t build. But then again few apps even have backend workflows…

That being said, Custom Events with their ability to return data make it much easier to implement Queues and other solutions to Race Conditions.

3 Likes
8

Not exactly guidance on what to do..

9

ChatGPT’d.

Imagine the frustration you’d feel when actually coding. Take whatever you’re feeling now and 100x it.

2 Likes
10

Until its not…

11

known and expected behavior

although most modern databases have concurrency logic built into them to handle concurrent writes - bubble does not.

it’s been a bigger issue since they improved API processing speed. it’s one of the reasons I don’t like to use database triggers because they are so unreliable.

imagine you create 100 things
then you have a database trigger to add those things to a list field on the parent thing
well that will fail for about half the writes because bubble creates several things concurrently so the database trigger triggers concurrently and tries to write concurrently - which fails.

that’s why now I almost never use list of things on a data, instead I put the parent data on the things being created and just search for them when I need them. lots of devs will tell you this is bad because you have to search every time you need it but the other way has issues too because the data is unreliable unless you purposely space out the API workflows so that concurrent writes do not happen. and then that creates the other issue of data taking a longer time to be created before the user can see it…

and also having a list of things on a data comes with other issues like the deleted thing still existing in the list even though it was deleted etc

it’s all about making the right tradeoffs for your app

12

Yep, some of it I took GPT to help writing.

At least in code you have full control, and can drill down on all issues as far as you want because it’s fully deterministic. With bubble, that is not possible and as mentioned;

you should be able to expect from DBs to handle race conditions gracefully by design. It’s a major headache that this isn’t the case with bubble and imo isn’t mentioned prominently in the SAWOL tab expression. I wouldn’t have built my application logic relying so much on SAWOL if this was pointed out more clearly.

13

That is exactly the case that I have which is causing me trouble.

I have Thing ‘folder’ & Thing ‘file’ (and many more similar cases)
File has folder, and folder has list of files. Because we’re providing (among others) a data sharing platform for our users, we expect to have many records which doesn’t really make it feasible to search for everything where parent = thing I want to search.

My solution now is to have a database trigger where if file now's folder isn't empty & file now's folder's files' do not contain file now I perform the write operation on the folder. When creating the file, this will add some overhead but I’m expecting that in the long run it should be more efficient than performing the search as you mentioned.

14

You have full control with plugins. Good luck with that “control!”

In code, the “you don’t know what you don’t know” increases by a factor of 100x.

15

Hi Randomanon

No, you don’t have full control with plugins. I already built 3 & am working on my 4th. The thing with plugins is, is that

  • the lambda where the code is executed is severely limited in memory (128MB) with a base load of around 90MB effectively leaving you with 38MB to play with.
  • you cannot query your bubble database properly when you apply privacy rules to the things you want to query
  • you cannot define values from you database to be returned again as a result from the function

These limitations don’t allow either for full control of bubble.

All in all, I’m happpy that bubble exists and that it showed me the way into Developing. However, if I had known all the pitfalls that come with the platform from the get go, I would have tried something else first. And now, I’m locked in which magnifies the cost of moving off of the platform by a huge factor.

i.E. if the picture that is drawn by the bubble ecosystem about its capabilities, it would have made my decision easier to choose the right dev platform for my case.

16

Wrong.

I didn’t say anything about “full control over Bubble’s made up JSON database.” I said full control, period. Meaning it’s infinitely extendable with 3rd party databases or FaaS. This also addresses your concerns with memory limits. As a final point, I wouldn’t be so confident about the other limitations you’ve listed. The Plugin API is very poorly documented and there are methods that only a handful of people know and keep to themselves.

The problem is that Bubble makes it very obvious when something is outside of its capabilities, unless it’s a bug. With actual code there is no “Support” that will explain things to you. Sometimes you have no idea how to fix a problem; that’s called “debugging.” So if Bubble has known “pitfalls” then code has an infinite amount of unknown pitfalls, which you sometimes won’t perceive until it’s too late. There is no mythical “right dev platform,” everything has tradeoffs.

17

What you’re suggesting is that you can use bubble as a front to other functions / lambdas / systems that you build yourself and get to with only API request. But wouldn’t that just mean you build your system at least partially off platform (not with Plugins / native bubble features) and use bubble only as a frontend. Ok, then you have full control but what is the added value of bubble? You then can just skip it entirely and build everything yourself with much more control.

No need to be passive aggressively explaining concepts as if I’m a toddler. It’s just that the debugging tools (where the logs that should correctly reflect what happens in the system is a KEY PART to that) are very much also half baked (e.g. no breaking points in backend).

Basically what I’m saying is that you need to invest pretty much the same if not more effort (especially with the newly added support of LLMs that have extensive domain knowledge of common programming languages) into getting to know the Bubble system where you have to be profitient to build the thing you want, and you STILL trade it off for much less control than what you would have when starting with building it in code from the get go.

Good marketing on part of bubble but really bad for the users in the long run when complexity exceeds ‘To-Do-List-App’ or include anything privacy related (privacy handling in Bubble imo is the biggest mess of them all).

And to that:

There is no way to authenticate you as user within the lambda, the methods provided by Bubble API (at least the ones partially documented in the docs and in the forum, and also after asking Support explicitly) do not support that. What you can do is pass in all Things into the lambda when firing up the workflow, but this method also has its downfall when you consider memory constraints.

18

That’s a question you need to be asking yourself, not me. Maybe the value was getting to this point much faster than you would have otherwise? Like training wheels.

You are in for a very rude awakening.

1 Like
19

Definitely this is due to “Race Conditions” and to be honest, this is one of the limitations I don’t like in Bubble either. I could solve it more than once but it took some time to ensure the robustness of my solution and tame this beast.
Bubble support team response by adding intervals is one of the solutions and I would say that it’s the easiest one but usually not the solution I prefer.
Recursive workflows have done a good job with me in different situations but sometimes it even required adding complex conditions to guarantee the correctness of the results.
@florianherzog I agree with you and understand your frustration and also I understand your point of view when it comes to coding since it does give you more control, that’s true.
Despite that, still Bubble has its advantages and I did build good things using it that have been reliable and in production for years now but to be honest, there is a learning curve when you deep dive into the details not just build an appealing frontend that interacts with the backend.

20

I once explained a bug (similar to yours) to a friend – an accomplished low-level programmer. He diagnosed as a “race condition” and said: “congratulations, you’re a real programmer now.”

My takeaway: every system has limitations and weird sharp corners. If you’re lucky there’s documentation, good debugger info, or knowledge in the Internet somewhere. If you’re unlucky, well :person_shrugging:, maybe you could investigate it and give back to the community (which you just did).

2 Likes