Monthly Community Update -- March 2022

Hi all,

This is our March community update! Read last month’s update here.

This was a pretty mixed month. We ended up spending a lot of our time firefighting various production issues (more on that below), but at the same time, some of our efforts in terms of scaling up our support starting making a visible difference in our response time numbers, and we added a bunch of great new teammates.

We welcomed:

  • Sam and Patrick on our Success team
  • Alex on our Marketing team
  • Kaela on our People team
  • Shyheme on our Engineering team
  • Yiwen on our Finance team
  • Fabian to lead our Sales team

As always, our open roles can be found here. As mentioned in previous updates, these two roles are ones we think are particularly interesting for current Bubblers:

  • Technical Product Support Specialist: help us do deep investigations of user-reported bugs and issues. Does not require programming experience, but previous Bubble experience is a big plus!

  • Video Producer: help us create amazing educational content to accelerate the learning curve of new Bubblers!

Changes we made this month

We continued to improve our new responsive engine. In addition to numerous bug fixes and small improvements, we added drag-n-drop in the elements tree, row and column gap controls, and padding on containers. We also released a video on using layout and sizing properties in conditionals.

Our work on version control reliability continued with various bug fixes and improvements.

In terms of community-building, it was a busy month:

  • We worked with some of you to help spread Bubble and no-code on social media via the #BuildInPublic and #BuildWithBubble hashtags. We want to thank everyone who applied, and while we couldn’t onboard everyone for this first trial run, we hope to do more together soon!
  • We’re getting Bubble bootcamps in more schools (for instance, a private bootcamp with Notre Dame’s IDEA center). In general, we’ve been investing in bootcamps, because we think it’s a great way to help more people become Bubblers, and because we want to launch the careers of more freelancers. We know there’s a shortage of Bubble freelancers right now and want to help as many people as possible get started!
  • We’ve begun preparing for the next Immerse cohort: see updates to our Immerse homepage for more details. We also posted a follow-up post checking in on some of our previous Immerse participants.
  • We sponsored TreeHacks, and are looking into ways to scale sponsoring more hackathons. If you’re an interested hackathon organizer, please reach out to us!

In addition to the above, a few posts on our blog you may be interested:

This month in numbers

  • New conversations via bug reports or [email protected]: 8,189 (down 7.6%).

  • Average first response time to messages: 3h 47m during business hours (down 43%)

  • Average response time to messages: 3h 44m during business hours (down 42%)

  • Open tickets being investigated by the engineering team: 55 (down from 53)

  • Of those, tickets that have been open longer than 7 days: 22 (down from 35)

Things on our minds

Like most of the world, we’re watching the events in Ukraine. We have a number of Ukrainians as part of the Bubble community, and our thoughts and prayers are with you as we hope for everyone’s safety.

As a reminder, experts expect the current Russian invasion of Ukraine to lead to an increase in cyberattacks. See, for example, CISA’s warning here. The most common and avoidable form of cyberattack is “phishing,” where someone attempts to trick the user into 1) providing secure information (e.g., passwords, bank account information, etc.), or 2) following a dangerous link in an email or text that looks like it comes from someone they trust.

  • Please be extremely skeptical of any communication that seems at all suspicious.
  • We highly encourage you to set up 2-factor authentication on your main Bubble account if you haven’t already done so.
  • If you spot any phishing attempts on Bubble apps, please report these to [email protected] and [email protected].

We spent a lot of time this month dealing with various outages and production emergencies. Some of these were self-inflicted, some of them were the results of scaling pressures that we fell behind on keeping up with, and some of it was simple bad luck and poor timing.

The longest and most severe outage fell in the third category: Bubble’s domain was blocked by CloudFlare as a phishing website, which led to a number of our systems breaking. After discussing the incident in depth with CloudFlare, our understanding is that it was simply bad luck that this occurred, that the timing alongside all the other production incidents was coincidental, and that they’re taking appropriate steps on their end to prevent a similar incident from recurring. That said, the extent of the downtime this caused was due in part to some design flaws in our caching infrastructure that led to the CloudFlare warning page to be cached even after CloudFlare reversed the mistake on their end. We are in the process of improving this aspect of our caching, and expect it to lead to an overall simpler, more resilient infrastructure.

The other major issue we’ve been dealing with has been a series of recurring database outages. After investigating, it looks like this is a result of an upgrade we made to the version of Postgres we run (more specifically, to a database extension). The upgraded version introduces a bug that breaks the tool we use to automatically defend our databases against dangerously-long-running queries. Unfortunately, it would be a tremendous amount of work to downgrade the our databases to a version that does not have this bug. We’re taking the following actions:

  • We’ve disabled all the specific queries that led to the outages that occurred. Since we took this step, we haven’t seen another recurrence of the same category of problem.
  • We’re changing the way we run many of our most long-running queries to avoid this bug in the first place, so that our automatic defense mechanism works going forward.
  • We’re planning to build a secondary backup defense mechanism that works along different lines (preventing dangerous queries from starting in the first place, instead of terminating them after the fact)
  • We’re working with the maintainers of the Postgres extension to get a fix for this bug in place.

A third category of issue we’ve had has been DDoS attacks: we saw two severe attacks against our infrastructure this month. Both attacks had the same technical signature. We are not sure what motivated the attacks or where they originated. For both attacks, CloudFlare was able to mitigate them automatically within 30 seconds, but during the 30 seconds, they sufficiently overloaded our servers that it caused user-facing downtime. We’ve made some changes to block attacks more efficiently on our end, and believe that this would be sufficient to stop a DDoS attack that follows the same pattern. We’re also working on some longer term changes to load pages more efficiently that should improve page load performance and make us more resilient to other kinds of DDoS attacks in the future.

A fourth(!) issue we’ve had is with our real-time updates system. Users have been reporting to us that intermittently, searches would stop updating with fresh data. After investigating, we tracked the problem down to performance issues with the database that powers these updates, and found a key indicator metric that reliably predicted user reports of the problem. We found a band-aid solution to relieve pressure on the database, which has stopped the issue from occurring. We’ve started work on a longer-term fix, which involves simplifying our real-time update system in such a way that this class of problems (as well as some other problems we’ve dealt with in the past) won’t be able to occur.

Finally, we had some self-inflicted mistakes where we rolled out code that we later had to revert because it was breaking user apps. The worst incident involved an application change in behavior that, on reflection, we should have released as a new Bubble version rather than as an immediate bug fix. We had a long conversation as a team about that incident, and came out agreeing on some operating principles for avoiding that kind of error in the future, including taking advantage of tooling to better predict whether apps are relying on a “bug”, and a process for declaring a new Bubble version that encourages the team to rely on it more frequently.

The good news on the self-inflicted bug front is that our investment in automated tests have been catching more issues before they affect users, and we’ve seen the rate of releases that we have to roll back go down. We also to continue to see the Scheduled tier working as a tool for sheltering apps in production use from these kinds of bugs (although the Scheduled tier does not provide protection from the infrastructure-level issues described above). All that said, we still break user apps with new code far too frequently, and changing this is something we’re working towards.

Anyway, that’s a pretty long list of issues, and it’s been an exhausting month for the team as we’ve dealt with all of them. As all of those different categories are unrelated, it’s not obvious to us why this month in particular has been so bad. Zooming out, though, we think it’s an overall consequence of our userbase growing, Bubble becoming more prominent, and all our systems and processes having more pressure on them. We’re doing some internal reprioritizing to make sure we’re spending enough time on preventative maintenance to keep up with these various pressures as we grow, because we know how much Bubble downtime can hurt businesses and organizations that rely on us.

What we’re currently working on

Due to the outages described above, we didn’t make as much progress this month as we were hoping, so most of this section is copy-pasted from last month’s update. The main thread we were able to push forward was the new responsive engine. Now that we’ve shipped the improvements described above, we’re focusing our attention on the plugin editor, so that plugin developers will be able to start upgrading their plugins to work with the new responsive paradigm.

Also, as mentioned above, we’re allocating more time for preventative maintenance; we’re exploring a number of different investments we could make to improve reliability. Most of the contemplated changes are behind the scenes and won’t be directly visible to users.

Repeating the relevant updates from last month:

Our major performance push, focused on data loading and rendering, continues. We’re continuing to work on optimizing invisible elements, building a data dependency graph to optimize querying, and generating HTML and CSS upfront instead of on the fly. All of these are major projects and we expect them to take months to quarters of work, though for some of them we expect to see performance improvements along the way. We’re also pursuing smaller performance bug fixes as we find them.

Another note on performance: the new responsive engine performs significantly better than the old one, because the old engine relies on a lot of Javascript-based manual manipulation of element positioning, whereas the new engine relies on CSS, which is much faster. So we see continuing to improve the new engine and making it easier for both new and existing projects to move onto it as a major performance priority as well. Some of the upcoming performance work we’re doing will only benefit the new engine, because it builds on top of the technical changes the new engine introduces behind the scenes.

We’re also doing work around the new user experience and general in-editor education, to make Bubble easier to ramp up on for new users. This involves fixing some ongoing painpoints such as how long it takes to load the plugin installation UI, as well as tweaks to the onboarding and new app experience.

Another workstream we’re kicking off is an overhaul of our network architecture with the goal of hardening our security posture. We aim to be SOC2 compliant by the end of the year, and this is one of the key prerequisites.

As mentioned above, we’re continuing to improve version control, both from a bug-fixing and feature development perspective.

On the QA front, our outsourced partners have now built 544 tests. For migrating code to Typescript, we’ve now actually started the Typescript-ification part of the process – we have one file in our main codebase that’s in Typescript – and we’re making slow but steady progress off of coffee-script, which is down to 44.6% of the codebase.

Thank you for your patience with us this month, and earnestly hoping March brings better news both for the Bubble community and for the world. Best,

Josh and Emmanuel


Great update @josh , really love the honest and detailed sharing!

I’ve noticed that elements tend to load more slowly, and tend to disappear/not show in the new responsive editor when first loaded/coming from another page. However, performance on the actual page works the same. Wondering if it’s because of all the rules and calculations needed.

Just some feedback!


1 Like

Hi @josh

Any chance you could look into the long requested feature to allow searching a repeating group case insensitive? This should be such a simple thing to do but we are forced to hack something together or clog up the app with unnecessary hidden elements/workflows/plugins.

Thank you for all your hard work Josh!


Thx for the update

We stand with you Bubble - your hard work is seen!

We grow and fall together


It’s great you promote awareness about security issues. To make the next step I like to suggest making 2-factor authentication on Bubble apps available for all plans so we can offer this to all our users. I think it’s a basic need for application security nowadays.

Thanks (again) for the extensive update.


I understand this was a tough month with the outages. But I sleep easier because of you guys so hats off for all the work that you guys are doing.

And definitely hats of for the new flexbox engine + features (gap spacing/padding!!!)


1 Like

Thanks for all that you’re doing for us!

We’re really eager to see the plugin editor be updated to the new responsive engine, and for updates to the version control system.

Fingers crossed for a better month in reliability :crossed_fingers:t2:

Be safe, ya’ll :heart:

1 Like

Great update - glad you guys found the issues and have a planned accordingly. This is why we are with Bubble


Thanks for the update! The database upgrade may explain some of the issues we’re seeing with significantly increased load times (we’re talking increased by a factor of 50 from seconds to minutes) on data-heavy pages. Are there any projections as to when you expect to have these issues addressed?


Appreciate all you do, Bubble team!

1 Like

Thanks for the update. Outages aside, all the new features – responsive engine updates in particular – are game changers.

On the QA front, our outsourced partners have now built 544 tests.

@josh Are you willing to share what company or service you use for this?

1 Like

To be honest we haven’t had much bandwidth for feature development lately, but the best bet is to upvote it on the – when deciding what new features to build, we generally base it on some combination of a) how many people care about it, b) how much work it is to build, c) how strategic it is for our technology or business goals.

Yes, we’ve been working with the folks at (but please don’t blame them for any issues – they’re just slowly catching up on all the millions of corner-cases we built out before starting to work with them!)

1 Like

Thanks, I’m looking for similar service for my company. Let me know if you have a referral code :slight_smile:

Any chance we can get the ability to share a direct link to an issue?

It’s hard to drum up support for a feature when one has to post a message explaining what to search for and then busy people are expected to visit the idea board, search, browse, and find the issue before being able to upvote it.

Being able to share a link directly to an issue would eliminate a lot of friction for the original requester as well as the upvoters (and would likely result in higher quality data which would enable you to more accurately gauge interest).

I’d be happy to point you to the relevant portion of the Bubble documentation that explains how to do this.



Cheeky! :joy:

But I do agree - I have gone to vote for an idea a few times before and given up because it’s just so damn hard to scroll (why so sloooooowww :flushed:) and therefore hard to find the actual idea.

So the votes on the ideas board would (I believe), in no way whatsoever currently accurately represent the sentiment of the bubble user community.


Proud to be a part of this team!

1 Like

These sound like consequences of success! Only positive imo.

Love these updates and the transparency. They make us feel heard and part of Bubble’s evolution / journey / building process. Thanks Josh & team :smiling_face_with_three_hearts:


Great work everyone.

1 Like

Yes, the great irony is that Bubble is good at building Bubble, but Bubblers are much better at building apps on Bubble. There’s a bunch of stuff in the Idea Board and Plugin Store that needs to be fixed/improved, and should be easy to do since those are built on Bubble!


This update shows why bubble warrants a team of about 200 developers, to make meaningful progress. You hired 1 :frowning: