Im trying to implement a simple unique sequential counter and it is pretty much impossible because there is no way to guarantee two threads dont read the number, getting the same value, before incrementing it resulting in a duplicate use.
Most programming languages implement mutexes or synchronization blocks that bottleneck and only allow a single thread access to a resource at a time.
what could work would be to allow us to specify a workflow as a synchronization workflow or to set a max thread count =1.
all other threads would be queued up and released in the order that they made the call. Threads can timeout like normal API calls if the resource takes to long to be released. But for something like incrementing an integer it would be really fast.
I assume you’re talking about backend sequencing of numbers?
yes, db holds a count at the current ID. code pulls a new one. Impossible to guarantee unique IDs with the current infrastructure. I had to build an external api that would single thread the requests for an ID.
But ultimately the goal would be synchronized access to a resource.
Yeah that’s always been the issue with concurrency in Bubble DB. There’s been various solutions shared in the forums but none have worked reliably enough.
I gave up a long time ago. Too much complexity and WU for something so simple. Plugged a Cloudflare Queue to call my Bubble endpoint. Barely costs me a cup of coffee a month.
1 Like
@ihsanzainal84 do you have a basic example ? 
It’s actually very simple. All CF Queues require a CF Worker acting as a Producer and another Worker acting as a Consumer. The Producer sends “messages” to a Queue which is then consumed in sequence by the Consumer. Alternatively you can POST to a Queue directly but using a Worker as a Producer gives you control over security and makes one endpoint more flexible.
Example
- My booking system will call a Producer endpoint (secured with a Bearer Token).
- It sends a message to the booking Queue. The payload is just a Booking datatype unique ID and quantity values.
- For each message the consumer will call my Bubble app’s endpoint to update the User’s booking. In addition it also updates an event capacity state stored as a KV in CF’s D1 which is atomic. So I know i can trust the values to be accurate to the millisecond.
I’ve handled concurrent bookings of around a thousand in a span of 10 minutes with no issues.
Queues are great!
They buffer and process messages asynchronously. So you:
- Avoid timeouts: If you’re doing expensive API calls or need retries, queues let you do it out of band.
- Rate limiting: Queues can overcome Bubble’s API rate limits. Consumer process messages at a steady pace. Pretty sure you can set the intervals.
- Durability & retries: If your worker crashes or fails, CF retries the message (with exponential backoff). So the Queue pauses and you can set the number of retries if i remember correctly.
Since you can have multiple Producers sending to one Queue, you can code your Consumer to handle different functions based off payload. Only 1 Consumer allowed per Queue.
1 Like
@ihsanzainal84 very cool thanks for the overview. I’ve been looking into Cloudflare more as you’ve been promoting them for extensibility. But a great overview, thanks.
1 Like
the problem is any API call is too slow. I can easily have hundreds of calls per second as database records get created.
im testing it this weekend to see if it can work. any external api call (whether cloudflare queues or not) will work because every other system can serialize access to API resources.
How fast do you need a record to be updated in the first place? If the API workflow is setup correctly then the updates to your Bubble records will be async and serialized already.
One of the reasons I recommend Cloudflare is because CF workers run on the edge with extremely low warm up times. It’s practically instant from my experience.
1 Like
just wanted to followup that I have implemented this with redis which provides a threadsafe data store.
Interestingly the api call now slows down the creation of the object enough that the screen refresh that happens right after happens before the object is created. As a result the object doesnt display on the screen. This makes me laugh and cry.
1 Like
