10/26 outage - question on what to tell end users

Just saw notice there is a partial outage and the repairs will cause downtime. Before I proactively alert users (and field complaints), @Fede are there any more details about exactly what’s going on and who might be impacted by it? I don’t want to cause alarm with customers if its not something they’ll really notice significantly

hi there, you tagged the wrong Fede :smiley:

let me get more details on how long we expect this impact to be and anything else I can share

Some app data on my site is unreachable all the sudden.

Clients, I am unable to upload files to Bubble.

Status page updated, it has some info:

@fede.bubble, will this affect the editor too? All of a sudden I’m completely dead in the water. Not only is my live app down, but the editor won’t even load.

We are encountering a 500 error in a backend workflow where we use a Server Action.

A lot of parts of bubble might see impact because the platform team is changing a lot of things around. So maybe or maybe not. It’s hard to say but assume that if you run into issues around this time, it’s related to this incident

Is there an estimated return date?

So this is self inflicted?? its being caused by things the team is changing??

self-inflicted in the sense that we ran into issues, and “the way out is through” so to speak.
To fix the current issues impacting Bubble the team might have to change/update things that break other things.

They are working as fast as they can

@contato62, I don’t speak for Bubble, but this is clearly an unplanned outage, and I’m sure the engineers are feverishly working to restore service; so the ETA is almost certainly “as soon as they can”.

My editor and app appear to be working. However certain data type’s data is unreachable. Seems like this is affecting things wildly.

All pages are showing the default "Your browser was unable to load the application" message.

I’m seeing this in the browser console for all apps (edited app name and url): refused to execute script from ‘https://appname.com/package/dynamic_js/xyz/appname/test/app/xnull/xfalse/xfalse/en_us/xfalse/xfalse/dynamic.js’ because its MIME type (‘application/json’) is not executable, and strict MIME type checking is enabled.

We’re nearing 30 minutes downtime on our dedicated app. The live frontend is still working but our backend is failing. Can’t even put in a message to our users letting them know. Some of us can log in to the bubble editors, and others cannot. :grimacing:

why not revert to the previous working state and then deploy things correctly without affecting other peoples businesses¿?

it’s too early to talk about why things are the way they are or why the fixes are this or that way. The team will conduct a postmortem as usual afterwards.

uploads are not working for me too, hope it gets done fast

Rough timing for anyone managing client apps right now. What I usually tell end users is something simple like “we’re experiencing a temporary service disruption and our team is working on it” without getting into the technical details. Most users just want to know you’re aware and on it. If you have a status page or even a simple banner you can toggle, that helps a lot too so people aren’t refreshing and wondering.