Thanks for sharing. The most important part of the log is the hour of deployment more than the detail. It will help to identify if we personally inserted a bug or it was maybe inserted by an update. I wish I can take a look at that log, so it will save me some time debugging a false bug and can react faster with a bug report.
4 Likes
Access to commit logs with a user friendly description(as friendly as it can be without disturbing your internal workflow) wouldn’t be a bad idea at all.
1 Like
I wanted to say, first and foremost, I sincerely appreciate your attention to this issue–on a weekend no less. It is above and beyond, so a sincere thanks for that.
I wanted to respond to your reply to my post.
After re-reading my post, I left out a crucial point, which unfortunately looks to have muddied the waters of the point I was trying to make.
As a shared cluster user, I would not expect any sort of formal SLA or uptime guarantee. I completely understand the limitations (and the cost) of that.
Speaking for myself (and perhaps few others)
We ARE NOT asking for perfection, a service level guarantee, or a ‘five 9s’ from bubble
I really cannot stress this point enough.
We ARE asking for is better communication tools and strategies for maintenance & updates
Perhaps this is easier said than done. It seems that offering better communication on updates and maintenance is a scalable solution that can be offered to the main cluster userbase. Right now for sysadmin communication we have 1) a status page and 2) a free-for-all in the forums, which is basically nothing.
I don’t see too many comments complaining about uptime or end-user websites being down, but rather the absent communication from bubble on maintenance and how that translates into a poor development experience.
Because the development environment and the site environment are basically the same thing, this means unless developers are enterprise customers, they have to deal with the service level of the main cluster (again,not complaining about that), but the real issue here is that they have basically zero communication from bubble on routine maintenance of main cluster.
Again, the issue is about communication. Not SLAs or uptime. Asking for better communication on updates and maintenance.
Thank you for your attention on this matter.
5 Likes
Yeah, I like the idea of sharing our deployment logs as a way of providing better communication and transparency into what we’re doing. Will discuss with the team (probably not for a week or two – I’m traveling this week)
6 Likes
Hi there @josh…
Many thanks for your prompt reply, and as @jon2 said, especially for replying at the weekend. I really appreciated hearing how you guys feel the same pain as us with regard to your dependence on other platforms. It is great we can understand each other’s circumstances in this way.
I agree with @jon2 in that I don’t think we are looking for SLA and dedicated type of options. We are also not talking about cluster down time: you seem to me to have a good handle on that.
As your eyes and ears of how well clustered Bubble is working, we want to work in cooperation with you to optimise the process when bugs or changes to Bubble cause things to go wrong with our apps.
As you say, there are different kinds of bugs. To summarise what you say about this, they can be broken into two categories:
- “Normal Bugs” - ones that have been around for a while or are a misunderstanding on the user’s part as they work with some feature new to them in their app development.
- “New App Breaking Bugs” - ones where the app developer has done nothing new but their app suddenly breaks.
So the problem that happened last week, where the console.log messages were swamping the console and hanging some apps when large repeating groups were displayed, was a good example of the latter. It took over 4 days for the process to go from bug introduction through to the bug being recognised as serious by app developers and Bubble. As you say, once that happened, it took half an hour to fix.
I hear your call that you want to be able to better recognise when an App Breaking Bug has occurred. I propose two things that I think will enable us to help you much more efficiently:
-
Clear Communication of Change Log
For each item that is in your change log, someone at Bubble writes a couple of sentences to help us really understand what has changed, a reference to any relevant bug report, and the implications for app developers.
So an example for bug #5404 which I submitted a few months ago may be:
"27 Feb 2019. 1530 EST. Fix of bug #5404. APIs now escapes the output for a new line in the payload, so \n becomes \\n.
Possible implications for existing APIs with multi line payloads such as email or SMS messages".
(By the way, for this bug fix, a month later I found a post on the forum from a user who couldn’t understand why their API had appeared to break regarding new lines on 28 February!).
-
New Forum Category “New App Breaking Bugs”.
This will give us a place to post when we believe we have found a new app breaking bug, and will bring lots of benefits:
-
As app developers we know where to go each day to see if any potentially app breaking changes have occurred. We will be able to immediately go and check if we have the same issue. If so, we can add it to the same thread.
(I think we have the same “noise” issue with the forum as you, so this will really help us to focus on the potential big issues!) -
Along with the new communication of the change log, we will be much clearer if what we are seeing is related to something that has just changed in Bubble.
-
From your side, that clarity we have of where to look for app breaking bugs and communicate if we also have them will improve the quality of your information about how serious the issue is.
-
Us app developers will feel less frustrated at the process, as we know where to post, and we know that Bubble will give high priority to a posting on that group.
Josh, I believe those two simple changes could really help to improve your signal to noise ratio, our frustration, and dramatically cut the time from the introduction of an app breaking bug to it being recognised and fixed.
Do you think these are two processes you could introduce so we can all see if it improves things?
Looking forward to hear from you,
Best wishes,
Antony.
PS - I wrote this offline just before your last post… so thanks for the offer to communicate the change logs. I hope the detail I have given here will help in what format of that would be most useful.
4 Likes
This is a really great discussion. The collective mind seems to have come to a pretty good solution here.
I too think that a visible “change-log” is a great idea. So I’d like to just add why I myself think it is a good idea as well.
If we take a look at the ‘internal deployment tracker’ that @josh has embedded here - we can see even now, an example of what may be valuable insight, for tracking bugs and issues.
And as @JohnMark has pointed out
So from the log entries alone. Without having to send a bug report to Bubble, without having to consult the forum - a bubble user such as myself can see that a change was made to “datepickers” at a specific time.
In best case scenario - each log entry would have a more detailed “explanation” to provide more context of the update, like @antony has suggested
We all recognize that the group of people who are building and maintaining bubble are working very hard, and have a big responsibility. So we don’t necessarily need a detailed “full-report” for each micro-update - but some kind of short message. Just that and the timestamp is enough for some precision Bubble users, to be able to figure out on their own what might be going on.
On a psychological level too, I think that exposing the change-log creates more trust between users and the team behind bubble.
Really, the engineers working on bubble, are like “co-founders” with thousands of us here on Bubble!
Yet I, and I’m sure most others - have never actually met them - and basically know nothing about them.
Providing change logs gives us the feeling of having more control over our apps. A better understanding of our apps.
And as we have seen, with some of the more technical oriented bubble users, such as plugin developers - providing a simple note and a timestamp, of your micro-updates could be all they need to recognize an issue.
And depending on how detailed the notes on the log entries are - it could actually be a very
effective way to automate the process and “seperate the signal from the noise”
What if it was a habit, for all (or most) Bubble users, to check this proposed “change log” whenever they were running into, what they think is a bug.
This could actually cut back on the influx of “bug report” emails - and reduce the energy for the person to person emails that I am sure some people on the bubble team do all day!
I think its a good idea!
As a side idea - if each log entry also had a (%) probability of how many apps the new update might effect. That would be pretty cool
5 Likes
Hey there,
I’m wondering if we are any closer to this? I’ve encountered some weird issues today with the db and client-side JS that was previously fine. It occurred to me that a deployment log would be invaluable to troubleshoot.
Thanks in advance
2 Likes
I just wanted to follow up with this thread again to bump it to the top for visibility, since it seems relevant this morning and I posted it on a Friday before the weekend. 
Hopefully there will be some news on this soon! Thanks for all your hard work @josh & team
From 9m ago…
Discussed w/ the team and we’re in favor of doing this, but it looks like it’ll take some engineering effort to do this, so we’re putting this into our backlog as a ticket item that we’ll work on when we get some free engineering capacity
3 Likes
Hi @josh…
Thanks for the update. I’m glad to hear it is something you would like to do, but somewhat disappointed that it does not have any kind of time frame, as for me this is a vitally important part of being able to release and maintain a stable product that will run my client’s businesses.
Would you be willing, in the meantime, to create the “New App Breaking Bugs” type of forum category I suggested in my post? To have one place where we can rapidly share our experiences will help to alleviate the “noise” issue you described and bring faster awareness and fixes.
Many thanks!
Antony.
1 Like
We just added the list of all releases to Bubble.
2 Likes
Going through this thread, is there any uptime tracking for dedicated plans? I understand Main Bubble cluster is about 99.8% uptime while there is no information for dedicated plans.
Someone mentioned above ‘we’re not looking for SLA’s and 5 9’s of uptime.’
I am. 98% would be fine but it would need to be in an SLA. People lose jobs over this stuff.
This is kinda basic (some would call it ‘table stakes’) and required to take this from hobbyist level to as serious of a platform as it seems like it should be.
I really do not want to relegate bubble to MVP’s only and have to recode the thing just so we can set up high availability system infrastructure.
What you seek might be part of a dedicated plan, but you should contact Bubble directly for the definitive word.