What will happen if I run 100M backend events in the app at one time?
What will face users who will be in the app at that moment?
I have something similar in our app and was thinking that backend events are slowing down the app on the user side and we should develop a custom code app for those events outside of Bubble.
Is it a wrong hypothesis?
Also, those 100M events are processed daily & sequentially. We can’t parallelize them in Bubble. Therefore it takes 12-16 hours per day for events to be processed.
Quick question: will buying additional capacity units lower down the time we need to process events? We need less than 1 hour per day instead of 12-16 for the same amount of events.
What is your use-case @vovahumnytskiy for such high volume.
Can’t it be simplified?
No, it can’t be simplified.
Solving the case I’ve described there is no option to simplify implementation.
Would you migrate to custom code or pay more $$ for Bubble servers?
I would perform an architecture assessment against the use-case requirements first to provide relevant options.
Is this a theoretical question or do you actually have 100M backend workflows at once? I think at that scale Bubble would be more than happy to help optimize and put you on a dedicated cluster and put your app name on the front page as one of the “successful” apps
Agreed. I’m constantly shown bubble ads specifically about raising 365m in funding. I’m assuming a bubble app was able to raise that money.
If you give them a good incentive, I’m sure bubble will be more than happy to play favorites with your app and place you on a prioritized server
This is what we actually have, but not because of many users. We have to connect and sync with an external API which provides tons of data in a very dumb manner.
If you do see slowdowns just because your backend is slammed with API calls, you could move to something like Xano for your database and let that run all the API calls keeping the database updated, then your Bubble app is just mostly the front end and making some calls to Xano to retrieve data.
But can’t say for sure… definitely make sure you can’t optimize further because it sounds like a lot of API calls… With extra capacity I don’t think it will help you with the non-parallelized workflows. It’s mostly for completely a lot of tasks at the same time, not shortening the actual workflow times (I think).
Also not sure if it applies to your use-case but utilizing the Data API uses a lot less capacity if your other source lets you format the calls to what Bubble wants.
I would highly suggest you switch to an external backend. Because Bubble sadly isn’t made for large backend operations.
There are a few options for you, you can check my post about Xano:
But other no-code solutions exists such as Backendless or Supabase, but I tend to prefer Xano.
It’s hard to recommend something without more details but in general for this type of scenario I’d move to an external backend to process the API data. No chance of damaging your user experience because of server capacity maxing out and will most certainly process the events faster than it will in Bubble.
Xano is my go to backend partner to Bubble but there are others as well.
Thanks for your tips guys, but I think we need to build an external backend outside of Bubble. And most likely Xano won’t help, it is too weak for our use case.
@vovahumnytskiy Xano is my go to as well, but have you checked out Bit.io?
I’ve run this on my own Digital Ocean droplet, this way I can control the scaling.
In our case we must building custom application which will handle backend logic. Also within this external backend we will have a database (can be Xano, why not. Maybe even Bubble database). But the problem is the use case is very complex and we need the most powerful toolset available. Programming is the only option which can help here.
As others have said/implied all you need to do is write a little function in some external function-as-a-service that fetches the data from your Bubble-unfriendly API, transforms it into something more Bubble-friendly and returns that when called. (Basically you create your own API endpoint that is a proxy for the crummy API endpoint.) Or, if you’re in need of Things rather than API objects, your external function might hit your app’s bulk creation endpoints or whatnot.
I’m partial to Google Cloud Functions for this but there are many others (CloudFlare workers, Netlify functions, etc.)
I do exactly this to retrieve and parse iCal files in my GRUPZ app. (This is a very simple case of what I describe above. We don’t have to worry about authentication or whatnot as the iCal endpoints can just be gotten with a simple fetch, but the concept is super simple. iCal files can, in fact, have thousands of events in them going back years, but a simple function that parses and transforms them into a more bubble friendly JSON response takes milliseconds and costs practically nothing.)
This is the type of basic coding project that isn’t very complicated and wouldn’t cost much of anything to have a js/node developer handle it, if you don’t want to DIY.
Surely your bajillions of workflows are just because you’re constantly iterating over lists, right? (This isn’t “complex” or compute intensive, it’s just that Bubble makes it seem that way.)
I believe No-code is not built for this type of application. Low-code can do that
Considering the size of the project, I think you should switch to traditional programming if you have the money.
No-code may not help you.
Maybe low code tools like “creation” or “outsystems” will help you but you will pay more
For that, I recommend the traditional program Ince in your case
(Front-end Angular) (Back-end Node.js) (Database Postgrad)
@tkamal, why do you think we should switch the front-end from Bubble to custom code too?
@keith thanks for your input.
We’ve already tried one platform which is similar to function-as-a-service. It’s not that “traditional” as Google Cloud Functions, but rather a standard modern no-code platform that was launched recently.
Their platform had all the toolset we needed and also it was working very fast (I guess paralleling events in the background).
However, we started with 2000$/month plan, spent 6 weeks finishing integration, and realized that the quota for this price wouldn’t cover even 10% of what we needed. Therefore, we proved that PoC is possible and scalable, but will cost way too much for this kind of thing ($10k+/month, it’s like paying 1000$ for a Starbucks cup).
Guys, do you know how can I access if with this kind of workload I woudn’t pay more then 500$/month for custom coded backend and that I wouldn’t pay that much as I had to with previous Faas? I am frightened if we switch to Cloud Functions  | Google Cloud whether it wouldn’t turned out that workload costs $10k+ again (even 500$/month is too much for a backend application from a business standpoint).
I would use xano. We should chat. It could handle this for a reasonable price!
Wait, an official xano partner would use… xano? Hmm, who woulda thunk it? 
