My apps have gone down at least 10X in the last 2 days? What is going on and is there a fix planned? Anyone know why this is going on?
3 Likes
Can you share more details about what you mean by “gone down?” Does your app not respond at all or do you get some sort of error message? Also, you can check https://status.bubble.is to see if there was an outage at that time.
No basically it just goes down… cloudflare comes up and says cant reach host… then a few minutes later it comes up again. This happens to all my apps at the same time.
2 Likes
There’s definitely been some random unreachability in the last couple days that doesn’t show up in status. Bubble updates or something most likely.
1 Like
great it is down again
2 Likes
Yup, this is painful with a Live app
2 Likes
Yes, for me the same.
2 Likes
Same here
1 Like
Same here. Could we get a response from the bubble team?
Yes would be nice to know what is up and what the plan is to stop it from continuing
Ok it’s working for me now.
Mine came up too… but the issue is this keeps happening. I have seen it go down like 15 - 20 times in the last few days. And that is just the time that i am working on my app and am aware of it. It is probably happening even more. One of clients keeps sending me screenshot of the CF error, and i keep making up stories of AWS outages to prevent them from blaming bubble and questioning their stability. I would like to not have to do that please.
3 Likes
There seems to be an internal rule “if it only goes down for 15 minutes we won’t write it on the status page”. It’s very disingenuous and not appreciated.
1 Like
Hi, @emmanuel @josh could we get some feedback from Bubble team regarding those issues ? Anything bad is happening behind the scenes ? 
1 Like
I would like to request that everyone also respond with what country you are from. I’ve noticed that Cloudflare causes issues - even with Bubble’s own tutorials - when trying to be accessed outside of the US.
Can confirm I’ve had issues accessing from New Zealand.
Well i am operating out of California, West US - Is there a way to check what AWS region the bubble servers are actually at?
I’m looking into this now. Our overall system metrics were healthy throughout the day but I do see a sub-system that was having issues around the times that people are reporting outages on this thread. Not sure what’s been going on just yet, and why it didn’t set off our overall infrastructure alarms, but am looking into it.
3 Likes
Thank you
Quick update:
a) There’s definitely a problem. It’s been causing periodic brief downtime across our system every few hours. Not sure why status.bubble.is hasn’t been picking it up – that’s on my list to look into after I fix it. I’ve put a manual alert up there and will keep it up until the situation is fully resolved.
b) It seems to be caused by a specific query that is occasionally running, which has a bug that causes a major performance impact.
c) I deployed a mitigating fix that should at least contain / shorten the outage windows.
d) I’m still investigating the root cause. I’m expecting given the nature of the problem that it might take a few more of these brief outages to get to the bottom of it, so may not have a complete fix til late this night / early tomorrow.
23 Likes
@josh I for one certainly appreciate your commitment!
3 Likes