Transparency - Bubble Bugs

[sridharan.s]

Hey, we ran into this same issue with our application and started troubleshooting it with our video conferencing platform provider and our 3rd party video development team since the error message sounds much more likely to have come from a WebRTC implementation than a website creation tool. As such, it probably cost us and our other partners 10 hours of time in total going down the wrong path.

Glad it’s fixed in Bubble, but I’m on a dedicated server so we have to push a change to our Bubble code to get it to work and there was no mention of it. So, it could have gone on for many months without me coming across a Bubble forum thread from Google. This isn’t okay.

@Josh, @Emmanuel, we need better transparency!! Can you please systematize this type of thing so that those of us who want to know how our apps are broken (because Bubble is broken) are in the loop?

[josh]

I’m sorry you got burned on this. Let me make sure I understand the timeline, so we can think about systematizing this better like you suggest:

-We deployed a release with a critical bug
-The bug got packaged into the latest dedicated release
-Issue was reported on the forum, we fixed the bug, and updated the forum post
-You deployed the latest dedicated release, which introduced the bug onto your server
-Since this was fixed on the main cluster, the thread got buried on the forum, so your team spent time hunting around without knowing what was wrong

Is that more or less right? If so, I’ll investigate how the bug got packaged into the dedicated release – we take some precautions to prevent that from happening, so something broke somewhere along the line.

[sridharan.s]

Thanks for the info Josh. I posted this as more of a general problem with significance well beyond this specific instance. At it’s core, we don’t have transparency into the bugs that are causing problems in our app when they’re actually Bubble bugs.

When you fix something that’s affecting someone else’s app (and likely many other people’s apps whether they know it or not) we rarely hear of it. In fact, very few are posted publicly on the forum and we’re generally discouraged from doing so.

As a result, there are times where we spend time running down bugs to fix them when the root problem is already known to someone else and it’d be obvious to us that it’s not a problem with our app if we only knew that other users had the exact same problem on theirs or that you had identified it as a problem with Bubble. For context, eliminating all possibility of it being a bug in our app probably takes 5x longer than finding a typical bug in our app, because we have to go down all paths instead of the most likely 1 or 2.

In this specific instance, one of our users brought this bug to our attention on June 28th. Since then I’ve involved our WebRTC development team and our WebRTC Platform’s (tokbox) team. As of this morning, both of them were still looking into the bug and we were having back and forth communication.

I don’t read every post in the forums (and nobody seems to) so I only came across the original post on this issue when one of my Developers in Russia sent me an image of a Google Search result showing other sites with a similar error message (Bubble’s forum post was ranked #1) so that’s how we figured out the problem wasn’t with our WebRTC stack.

If we hadn’t realized it that way, then we would have burned many more hours and we also likely wouldn’t have updated Bubble on our server (which we’re reluctant to do because we don’t want to introduce new bugs since we’re pushing hard to get to a stable version). Furthermore, this would have remained an outstanding bug on our list and we would have categorized it as known bug, no known cause / no know way to reproduce, and we’d have it in our bug board for years (so that if something like it comes up, we’ll have 2 examples to draw from).

Our success is dependent on Bubble (and we’re huge fans of Bubble), so we need to know what’s going on with Bubble (bugs, product roadmap, etc.) in order for us to make the best use of our time in growing our own businesses.

It seems to me the solution is to put in place a systematic way to communicate these things to people who want to know about them. For example, a public known bugs list would be great. I’m sure there are many other great solutions as well.

[josh]

Thanks, understood. Generally for a severe bug that breaks the apps of a number of users, the time from us knowing about it to fixing it is measured in hours or minutes, and our priority is generally getting a fix out ASAP rather than communicating. That said, communication is really important, and we’re planning on a couple things to help with that: a) we’re hiring customer support help, so that we’re not split between engineering and communication in a high-pressure situation, and b) we’re working on formalizing what an “incident” is, and how we report, manage, and post-mortem them.

In this particular case, I think what would have helped you is an easily googleable list of common bubble error messages. The message “Your browser was unable to load the application data. We’ve been notified of the issue. Please try again in a few moments and make sure not to use ad-blockers” is a generic message we display when we detect that we were unable to load the javascript file with the page data. There are a lot of different things that can cause that message, including things out of our control (such as ad blockers).

The forum post you linked to above was in response to a 25 minute incident yesterday where a lot of apps were getting that message because of bad code we released. (We’re currently doing a post-mortem to see why our automated tests allowed the bug to get through to the live site). Given that your issues started on 6/28, it’s likely a different cause. My suggested next steps for you (if you haven’t already solved the problem) is to see if you’re getting a 500 http status response from Bubble’s servers for the page javascript; if you are, then it’s likely an issue with Bubble), if you’re not, then it’s likely an issue with either your custom development or with your users’ browsers / networks.

Anyway, I think it would be valuable for the information in the above two paragraphs to be easily googleable, so as we continue to update our documentation (which right now is nowhere near as a good as we want it to be) we’ll look into including it.

[mishav]

Thanks for raising this, Scott.

@josh, another reason to publicly show bugfixes released is that a fix can break an app if it is using a workaround for the original bug.

For example, there was a fix to the server-side current date/time timezone offset, and an app developer that came across the original problem might have thought “oh that’s just how it works” and put in their own adjustment.

The fix would mean the adjustment is now incorrect, and the app developer may not find out that there is some incorrect data for months.