Monthly Community Update -- May 2020

Hi all,

We’ve decided to start doing a monthly update for the Bubble community. The goal is to give the community some more insight into what’s going on at Bubble: what we’ve been working on and what’s on our minds.

Emmanuel and I haven’t been as present on the forum as we were in the earlier days; as we’ve grown the team and community (we just crossed the 400,000 signups mark!), our time has gotten spread pretty thin. It’s been great to see the rest of the Bubble team pick up the slack, but I think it’s important for us to periodically touch base with you all. Hopefully this post gives some insight into what’s going on with the company, and I plan to periodically check the replies for the next few days and try to answer questions where I can.

Changes we made this month

We also made a some important behind-the-scenes changes:

  • We partitioned the database tables we use to store the change history for application data; this allows us to grow them and clean up old data in a scalable way. We also improved our monitoring capabilities for storage usage in our main cluster. This is important to keep our main user databases healthy as we get more and more customers.

  • We recently changed the way we handle incoming support requests. We now have the success team take turns playing separate roles: some people are quickly triaging requests and replying to the ones that can be answered in a minute or two, while others are doing deep dives on the ones that require significant investigation or debugging. We think this will help us respond faster to the easy requests, and help the people working on difficult bugs work more efficiently and get to the bottom of them quicker.

  • We switched the provider we use to monitor our infrastructure. We collect thousands of metrics streams on all aspects of the health of our servers and databases, which we use to keep our infrastructure healthy and respond quickly to problems. We switched providers because of features that allow us to better organize our metrics as we add more systems to keep track of, and to monitor more things in cost effective manner so that we can maintain high visibility.

This month in numbers

  • Total customers who reached out to us through bug reports or support@bubble.io: 1,408 (up 52% from last 30 days)

  • Total received messages: 4,744 (up 64% from last month)

  • Average response time to messages (4h 30m counting only our work hours; 12h 27m in absolute terms)

  • Total bug reports: 508 (up 116% from last month)

  • Bug fixes that we deployed: 35

Things on our minds

  • We’ve received a number of reports from users that they’ve had elements or workflows disappear while using the editor. We always get some number of these reports, and many of them are due to people accidentally deleting something and not noticing til later. But in the last couple weeks the volume has been higher than usual. These issues tend to be hard to investigate, because we have to look at our logs and guess what might have happened, so we don’t have definitive answers yet. Our best guess is that our recent changes to our version control and deployment feature might have led to the increase in these reports. We’re actively working on a project to make our version merging more reliable, so hope that will solve the problem, but we’re also investigating several of the most recent bug reports to make sure there isn’t something else going on.

  • This month was a fairly bad month in general for production breakages and downtime. Several of these were self-inflicted: we pushed bad code, which either caused an outage or broke an important feature on user apps, and had to roll back. Others were caused by scaling pains as we change our infrastructure to handle the growing volume of usage. And we also had some caused by issues with upstream infrastructure providers. We’re worried about the negative trend, and thinking about ways of mitigating it. One thing we’re doing is investing more time in building automated tests: that’s an important part of making sure we don’t make the same mistakes twice, although it’s not a cure-all because it’s easy to test for things that you anticipate but hard to test for issues you weren’t thinking about. Another idea we’re discussing internally is rolling out code to users on our free plan prior to rolling it out to customers on our paid plan, so that people with apps that are in production are less likely to be impacted by bugs. We don’t currently have the ability to do this, but an infrastructure project we’re working on that we hope to complete in the next couple of weeks would open up that capability. Finally, we’ve noticed that a lot of bugs keep coming back to several parts of our code that aren’t very high quality. We keep on doing small patches and fixes to them in response to user reports, but often those patches actually cause new bugs. So we’re planning on prioritizing spending the time to just rebuild those parts of the code in a higher-quality way so that we fix the problem for the long-haul.

  • Relatedly, we’ve been thinking about ways to build trust in our bug reporting system. We’ve been doing some reflection on this and realized we haven’t been doing a good job as a company of telling users who report bugs to us through [https://bubble.io/bug_report] what’s going on with their report; we’ve heard a bunch of users tell us that they feel like the bug report form is a blackhole and things disappear into it. It’s important the community trusts that medium of communication, because the quickest way of us getting to the bottom of a situation is if we get a detailed, clear report, which often involves confidential details about a user’s app that can only be shared through a private communication channel. So it helps us respond more quickly if people’s first reaction when they see a problem with Bubble is to send a report in. But that will only become a habit for the community if it gets rewarded with a fast response and clear communication. The bug progress tracking page I mentioned above is our first step in doing this; we’re also making changes to the way we triage and handle bugs to try to up our game here.

  • Another big thing we’ve been dealing with is the impact of COVID on our company and team. We’ve been relatively lucky in the grand scheme of things: all our work can be done remotely, so far no one on the team has had serious health issues, and while we’ve lost some of our customers because COVID forced them to cut costs or shut down their business, it’s been a small percentage of our total users, and we’ve gained more customers than we’ve lost over the past month. That said, transitioning from an all-in-person company to an all-remote company has been a big change, and we’re still getting used to communicating with each other entirely over chat and video. Several of us have been stuck in small NYC apartments by themselves for over a month now, so we’ve been doing various things to socialize remotely as a team to try to keep everyone from getting too isolated. We’re also monitoring the larger economic situation, and trying to trim back on unnecessary costs and improve our internal financial reporting, so that if we start losing even more of our customers we’ll be able to keep the business healthy.

What we’re currently working on

In addition to the initiatives mentioned above, we’re actively working on the following:

  • We’re alpha-testing a feature to allow importing much larger CSVs than our current CSV import feature can handle.

  • We’re alpha-testing a videochat plugin to make it easier to build apps that have video chat as a big component.

  • We’re alpha-testing an integration with Google Optimize to make it easy for users to do A/B tests in Bubble.

  • A complete rebuild of our system for doing calculations on top of application data. The current system can sometimes result in the calculations getting out of sync with the app, which can cause things like elements disappearing or pages not rendering correctly. Some of the recent bugs related to our version control feature have been from interactions between that feature and this system. This is a big project, but unfortunately the only way to completely eliminate this category of bugs is to replace the system with one that works using a different overall approach.

  • We’re in the process of rolling out a connection pooling layer in front of our main cluster databases. This is big infrastructure change (and was the source of a couple of our more recent outages), but it’s critical to let us keep scaling up our systems. Right now, one of our big performance bottlenecks is that we’re unable to add as many servers to the main cluster as we’d like because of pooling limits, which leads to slower response times throughout the system. This infrastructure change, once complete, will unlock that, which is a major victory in making Bubble faster and more scalable.

  • We’re also planning out a project to be able to seamlessly transfer apps between databases in our main cluster without causing disruption to the app. This would allow us to rebalance databases when one database has too much activity on it, which would enable us to have more even, predictable performance.

  • We’re doing an overhaul of the code that powers the Input element; this is one of the areas I mentioned above where we have a constant stream of bugs, so we’re investing time to improve the overall reliability.

  • We’re about to start working on an upgrade to our SSL LetsEncrypt integration. This won’t add any direct user value, but if we don’t do it this month, the current system will stop working.

  • We’re working on a complete redesign of our editor. This involves changing the visual appearance, moving components around, and smoothing out some of the jankier interactions. This is a huge project that’s been going on since last year and that we don’t expect to finish before the summer (we currently have two and a quarter engineers working on it almost full-time).

  • We’re doing a ground up overhaul of all our educational and reference content, including the reference, manual, and video tutorials. This is a big, multi-month project to get our learning resources to the next level.

  • We’re continuing to scale up our customer success team and professionalize its operations

  • We’re hiring for our growth team to help us connect with the entrepreneurship community and support our customers

  • We’re sketching out changes to the design of our homepage to make our branding more modern and appealing

As always, thanks for all the contributions you’ve given to the Bubble community – we appreciate all the community does to help each other out and to support Bubble. We look forward to seeing what you build in May.

– Josh and Emmanuel

101 Likes

Big yes to this one :point_up_2:

Overall I love the updates idea, thanks Josh and team keep up the great work!

10 Likes

Hats off to team bubble for getting through all the above during the Covid situation!

Here is some :cake:

Cheers!
ZubairLK

4 Likes

This is a very detailed and informative update. Really appreciate going into this much detail, definitely provides clarity and peace of mind.

Sending well wishes to the whole team, COVID-19 has definitely transformed the way we all do work on way or another.

1 Like

Thanks you guys. Now that I’m relying completely on Bubble for income it’s even more imperative that Bubble improves its downtime. Thank you so much for making that a priority. If Bubble fails then www.nocodeminute.com fails too. Thanks again for your informative post. This helps a lot. Keep in touch and stay healthy amid the worldwide epidemic :mask:.

:partying_face: Congrats on the 400,000+ sign ups! :raised_hands:

Jason - NoCodeMinute Founder

2 Likes

Glad to see this @josh - As a longtime Bubble user, the transparency is appreciated - Especially now as I’ve started to invest more and more into the platform. Past 30 days have been rough and have shaken confidence but glad to see it’s being attended to. Thank you!

8 Likes

Thanks @josh great to see a transparent communications approach to the community it’s appreciated.

It all sounds very positive.

With ref to rolling features to the free plan, many of the free users may not be experienced or have complicated enough apps to spot/reports bugs. I wonder if some form of opt in beta testing flag on a plan might work? - maybe linked to a discount. That way people who don’t yet have customers but are paying could test & feedback.

We have a number of apps we would be happy to stick into Beta testing.

Thanks
Simon

4 Likes

@josh we greatly appreciate the detailed plan and transparency. Looking forward to the infrastructure upgrades as well as the new editor.

3 Likes

@josh I love this new update idea! Just a quick note to say I think the bootcamp link is broken in your message :slight_smile: thank you!

1 Like

@josh thank you for providing this detailed update. Yes, it helps a lot to hear from you and/or @emmanuel periodically. For me, the last few days have been a bit nerve wracking with the issues we’ve seen but after your candid update, I feel a lot better. You obviously know where the issues are, but it is comforting to see it clearly communicated. I am definitely having a great weekend and wish you and everyone at bubble the same. Cheers. :clap:

5 Likes

Definitely a fan of testing the code on the free plans before pushing to main cluster with production apps! Willing to wait a little while longer for new features for the sake of stability.

1 Like

Great point – if we go down this route, it’d be easy to let paid users decide for each of their apps.

thanks, fixed!

2 Likes

Thank you for the update :slight_smile:

1 Like

Thanks for the updates @josh

On a side note, I have a project that I was going to do a video integration for. I was wondering if I can be part of this alpha and of course I will be happy to report bugs and suggestions as a result of my usage.

Thanks.

3 Likes

+1

Would love to help test the videochat plugin!

1 Like

Bubble has turned my entrepreneurial world upside down. With about a dozen bubble projects, with 4 already launched, it’s an amazing feeling to be able to go from idea to launching in a few days. Bubble is legos for adults!

Thanks for the continued hard work!

5 Likes

Love the legos for adults comparison!

3 Likes

Thanks @josh! As a user who was very active in 2018 then recently came back to Bubble, I was initially a little dismayed to see that Bubble still seems to be in an “early stage” of product maturity. I would be hesitant to sink my time and energy into this platform if it had stopped growing. It’s encouraging to see that the community is very active and the company still growing the product. I love Bubble so much and I want to see it succeed!

Here are some things that would support improved confidence in Bubble:

#1 Improved documentation:

I am still spending half my time searching through the forum to cover gaps in Bubble’s official documentation. It is still somewhat of a science project to figure out Bubble’s idiosyncrasies. This, more than anything makes Bubble seem “early stage.” Glad to hear you are working on a “big, multi-month project to get our learning resources to the next level.” It would be helpful to hear more about that… how big? Can we help?

Positive stability trend:

The experience of using Bubble this month seemed less stable than it was in 2018 which made me worry about supportability. It’s common for startups to create a giant mass of spaghetti code and not prioritize the less glamorous work of cleaning that up as they grow. It is a relief to hear that you understand this and kudos to you for doing that!

Evidence of growing the team successfully:

It is a major hurdle for a startup to grow beyond the “founder(s) and a small team of helpers” to a larger sized company. But at some point the work grows beyond the founders’ ability to have a direct hand in everything. It sounds like Bubble is at that point right now. Hearing new voices at Bubble that have the same passion, dedication, and most importantly knowledge as the founders would be encouraging and build confidence.

2 Likes

Thanks for the update @josh - this is the one!

1 Like

Hey @josh

Thanks for the update and initiating the monthly updates, really great to peak behind the scenes and appreciate whats being done and currently worked on.

One quick suggestion, it might be worth appending the post title to include the month and year, just for search-ability in the forum and later context I guess.

This is something I’m very excited about, especially from the teased images. Will this include some changes to the responsive design/editor view? I remember Emmanuel mentioning late last year that there was something on the cards to make improvements for this early 2020.
Also I know you stated this is still a bit of a way out, but interested if any plans have been discussed on how this will be rolled out e.g. beta program, excluding legacy - as its sound like quite a substantial UI/UX change and something needing to re-adjust to.

Cheers!

1 Like