Hi all,
Over the past month or so, I’ve been reporting weekly on our progress against stability. If you missed it, you can read last week’s update here. This is my last planned weekly stability update — as detailed below, we’re transitioning from short-term to medium-term work, so a monthly cadence will be a better fit. Check out our monthly community updates for progress on stability work going forward.
(Also, as a personal side-note, our next monthly community update won’t be written by me — my wife is expecting a baby, and I plan to take parental leave. You’ll be in good hands with other members of our leadership team covering for me, including Emmanuel, VP of Product Allen, and Director of Platform Engineering Payam.)
This week, we circled back on an objective I mentioned in one of our first stability updates: Fixing a performance bottleneck with our page loads that led to a hot redis shard during the DDoS incidents in early May. This week’s progress completed the short-term hardening work we had planned out in response to the DDoS. It also led to a small (roughly 0.2 seconds) improvement to page load speeds. This was a lower-priority project coming out of our initial roadmap of short-term fixes, and by wrapping it up, we are now in a substantially more stable place than we were in the first half of Q2.
The other open project from that roadmap is migrating off of our legacy stored procedure framework to make it easier for us to cancel running queries. As I mentioned in my last update, we are rethinking our approach to that workstream — while we plan to continue it, we are shifting from a short-term, “release ASAP” approach to a medium-term workstream so we can build additional protections and make the transition process safer.
Our primary stability priorities are now all medium-term projects:
-
Migrating off our legacy stored procedure framework
-
Expanding to multiple shared environments, which we are aiming for in the fall. This is a big project, but it will be a game-changer for the overall robustness and stability of our infrastructure
-
Deeper investments in the performance and scalability of our user databases, including further work isolating especially large payloads and building multiple backends to serve different kinds of queries
Focusing on these projects continues to be our top priority as a company: In addition to improving Bubble’s stability, this work will increase the performance of Bubble apps and our overall scalability as a platform. We know this is incredibly important and impactful, and we have teams of dedicated infrastructure engineers pushing them forward.
Finally, I’m happy to report that we didn’t have any severe incidents this week. The stability improvements we’ve put in place continued to hold firm, and there have been no incidents causing widespread user impact.
We hope this pattern continues, and we plan to use this lull in emergencies to make fast progress on everything we are working on!
Best,
— Josh and Emmanuel
