Why do the apps KEEP going down?

Thanks for the update! Always appreciate the hard work you do to keep everyones apps online, especially with the small team Bubble has.

1 Like

Really appreciate the detailed reply.

Looking forward to hearing more about how it goes

1 Like

Thanks for looking into this. It’s definitely slowed me down a bit over the last couple of days. (And hey, I wouldn’t be surprised if I’m the source of the weird query. I’m very good at breaking software.)

1 Like

@josh what is the most constructive way for us to report issues that we feel are system wide that aren’t picked up by status.bubble.is?

Along similar lines, provided we have threads like this, I feel it would be helpful for admins to nudge structure to responses so you have more to work with. (I have a feeling “same here” responses may not be the most useful…)

Here’s a simple proposed structure for replies that admins could encourage:

  • Observed Issue: across all apps, encountering both test and live apps timing out.
  • First Occurrence/Duration: started ~8:30am EST, still encountering issues
  • Location: California (US)
  • Open to Bubble Team Contact: yes (via forum)
  • Supporting info: (ex. screenshot or video of issue; error message, etc.)
4 Likes

@josh thank you for all your hard work getting this issue worked out! :slight_smile:

Good question. “same here” responses are better than nothing (they help us get a sense of whether this is a widespread problem, or something specific to someone’s app), but yes, the more details the better. The most helpful details are specific timing, screenshots of the error (and if possible the javascript console), and geography, but everything you listed is useful

1 Like

Latest update:

  1. I found at least one category of request that were responsible for problems, and deployed a fix. That might be enough to completely resolve it, but I’m continuing to monitor because there might be similar problems elsewhere.

  2. I made some tweaks to our infrastructure alerts that would have notified us about this issue sooner. Specifically, we weren’t alerting on the CPU over short time intervals for one critical system: we were tracking long-term CPU changes but not short-term spikes. I’ve fixed this, as well as adjusting a couple of backup alerts.

3 Likes

That was more than 7 hours of issues… Wonder how many emails you got from StatusPage reminding you you have an incident open.

29 August, Major Outage:

image

The bubble crisis management team (if there is any) should be more transparent about major issues before people start reporting them.

Imagine you’re building Facebook and people start subscribing then suddenly the service is down… goodbye entrepreneur, there is no credibility.

Still seems to be an issue. Sporadically I’m unable to even get to my site. At the moment I’m not even able to ping it.

image

Same to me 20 mins ago, long loading time or just the app doesn’t opens and had to refresh 4-5 times.

Just had 2 different sites that could not be reached for about 3-4 minutes (11:22-11:25 Eastern timezone (Toronto)

Pages would not load at all, and browser just gave a generic ‘Site could not be reached’ message

London, 17.45 local time, couldn’t reach server for about 5 minutes.

Same here. I am getting the “Site could not be reached error” as well. Overall, the editor and apps seem really slow.

1 Like

@josh, if I’m not mistaken, it was you who stated that “same here” posts work because they tell you other people are having the same issue. Can you please address this issue? Having this as well myself repeatedly:

App went down again! US WEST / 3:40PM PST Unreachable / No Error

Okay, is this enough for someone at Bubble to tell us what’s going on? @josh @emmanuel

Hey @ryan8 and @Profamii, sorry you’re still having issues here.

I’m pretty sure that the problems you are having are not related to the Bubble-wide problems earlier this week. The issues we were having earlier were caused by one of our key databases hitting 100% CPU periodically. It took a while to resolve that because it turned out there was more than one thing driving the CPU usage, but via a combination of hunting down and fixing the things that were driving CPU usage and upgrading that database to heavy duty hardware, we were able to fix it. The last occurrence of 100% CPU was on Aug 30, 11:25 am ET / 15:25 am GMT.

I’m not sure why the two of you are still seeing issues, but it may be related to either your individual apps or your network connections to Bubble. If you keep seeing problems, we’ll continue to try to diagnose it – can you please email support@bubble.is so that we can diagnose things with you offline?

2 Likes

@josh After your fix everything works nicely for me. This I can confirm. I’m based in Poland, Warsaw.

This topic was automatically closed after 14 days. New replies are no longer allowed.