Important update: Changes to how Bubble integrates with SendGrid

payam.azadi · 2024-07-17T00:03:10.557Z

Hi all,

I want to address some of the repeated issues you may have recently experienced with SendGrid (our email provider). By end of day today, July 16, we’re changing what kinds of emails can be sent for users who don’t have their own dedicated SendGrid API key. I understand how much of a pain these interruptions have been for your businesses, and it’s been difficult for us as well. This change should dramatically improve our reliability across the board.

Why we’re making this change

Today, Bubble allows users to try out SendGrid email functionality using Bubble’s shared SendGrid API key. This means one key is shared between all users who don’t have their own. While this has made it easy for users to get started, we’ve recently seen some misuse of the shared SendGrid API key, leading to spam and phishing attempts. These emails contained links directing recipients to external sources not associated with any Bubble apps. This not only compromised the security of our email system but also affected email deliverability for everyone using the shared key.

To stop these malicious actors and protect the integrity of our platform, we’ll be blocking the use of the shared SendGrid key for sending emails to external destinations across all plans effective today.

What this means for you

We know there are instances where it’s necessary to route your users outside of your Bubble app. These changes will not shut down your access to SendGrid or your ability to send emails to end-users. However, if you want to send emails that direct recipients to external links outside of your Bubble app, you’ll need to configure your app with your own SendGrid API key. Please find directions for how to do this below:

How to set up your SendGrid key:

Obtain a SendGrid API key: Sign up for a SendGrid account if you don’t already have one, and generate an API key.
Update your Bubble settings: Update your Bubble app settings to use your new SendGrid API key. Detailed instructions on how to do this can be found here.

I know that this change may cause some inconvenience, but it’s crucial for maintaining the security and reliability of our platform for everyone. If you have any questions or need any help making these changes, please visit the Support Center to get in touch with our Support team. Thanks as always for your trust and patience.

Best,
Payam

Jici · 2024-07-17T00:15:49.742Z

Just to be sure @payam.azadi :

Confirmation and reset email are not affected?
Email that contain a link that point to Bubble page are not affected?
Email that doesn’t contain any link are not affected?
Finally, only email that contain “external” link are filtered and will not be sent until you entered your own Sendgrid API keys?

payam.azadi · 2024-07-17T00:19:31.673Z

Hi Jici, thanks for the great questions, which I’ve answered in-line.

Confirmation and reset email are not affected?
Correct - confirmation and reset email are not affected by this change.
Email that contain a link that point to Bubble page are not affected?
Correct - emails that contain links pointing to Bubble domains you control are not affected.
Email that doesn’t contain any link are not affected?
Correct - emails that do not contain any links are not affected.
Finally, only email that contain “external” link are filtered and will not be sent until you entered your own Sendgrid API keys?
Correct - only those particular emails would be blocked. If they do get blocked, we will both send you an email (exactly 1 per day), and log it so that it appears in your Logs tab.

Thanks for this!

chris.williamson1996 · 2024-07-17T00:29:28.899Z

For anyone looking for some info on how to setup their own sendgrid and how to get off of bubbles instance refer below.

[Solution] Bubble Email Sending Issues Tips

Hey As many of you know and countless threads have popped up bubbles sendgrid shared server is having an issue. Below I’m going to walk you through your options to not see down time in your app as well as some of my favorite email recommendations Firstly I’d like to acknowledge a few things: 1: Bubble itself has little control over this situation and has to rely on sendgrid to get back to them. Unfortunately an “emergency fix” isn’t in their hands. 2: Bubble gives us the fle…

This is more so a caused by email servers such as Google and yahoo more so than bubble. Sendgrid and bubble are just being effected down stream.

trevpierson · 2024-07-17T01:08:50.482Z

Thanks for this @payam.azadi.

I did notice today, and today only, that some of my users tried to reset their passwords with the standard out-of-the-box email template using Bubble’s Sendgrid API, and the emails were never received. Is this something that will be fixed after today, or no? From reading your response to @Jici it seems as though a pw reset email that directs the user back to Bubble shouldn’t have been affected by the change.

Thanks for the clarification - really appreciate it,

Trevor

georgecollier · 2024-07-17T01:23:23.460Z

Sounds like a good change that allows most simple testing! It would be helpful if it returns an error in the logs when it fails, as otherwise users will still be confused about why their email isn’t being delivered.

But yeah, healthy middle ground that allows users to test while protecting the platform. Continuing to do the same thing while expecting different results from SendGrid would be insanity!

stuart8 · 2024-07-17T01:24:21.124Z

Good change, Bubble should do a blog post or series on different email providers and setting them up in Bubble, would probably help a lot of users.

I’ve been using Postmark for the past 2 years and been super happy with the deliverability and support they offer. Also heard good things about Loops

payam.azadi · 2024-07-17T02:54:32.675Z

@trevpierson , there is an active issue with SendGrid right now, where our account was suspended because of this phishing incident, and we’re working with them to reinstate, which I expect should happen tomorrow. So, yes, standard emails will have failed, but not because of our change. The status page will have real time information on the status with SendGrid, at status.bubble.io.

payam.azadi · 2024-07-17T03:00:34.899Z

thanks for the positive feedback George! and yes, the change will email once per day and also create log messages in the log tab.

bonjour_17 · 2024-07-17T09:39:22.292Z

Hi, this is probably good overall, but… at what point does Bubble come with a native feature breaking change and :

give us 0 upfront notice and come with “effective today” message ??? I don’t even know what timezone we are talking about, so my live apps right now might have lost functionnality and I might urgently make changes to my apps
why just a forum message I could have easily missed (and almost missed!) and not an email with a release note?

georgecollier · 2024-07-17T10:18:11.055Z

It’s not a breaking change for live apps. Live apps are supposed to be using their own API keys, not Bubble’s shared one. If they’re not, that’s not on Bubble

bonjour_17 · 2024-07-17T10:27:34.578Z

A thing that worked some way yesterday and does not work the same way today is not a breaking change?

From the documentation “we recommend setting up your own SendGrid account when your app goes live”

I am not native but to me that does not really sounds like “do not rely on this service as deliverability is not guaranteed and this feature can be shut-down over 1 night with no upfront notice”

→ in addition → bubble doc is evolving positively, I am not even sure 2 years ago that warning was there as it is today.

→ in addition → given the amount of feedback regarding this in the thread about last month mail outtage, it seems clear that the limitations about the native mailing feature was not advertises enough.

Personally, I am not relying on this feature for critical stuff. I do use the native mailing feature but for less important admin reporting stuff.
But what I am concerned about is the short notice and the chanel these notice are brought through - I should not be monitoring the forum 24/7 in order not to miss some important announce.

georgecollier · 2024-07-17T10:40:07.117Z

bonjour_17:

From the documentation “we recommend setting up your own SendGrid account when your app goes live”

manual.bubble.io

Email settings | Bubble Docs

This section covers the settings for sending emails from your app through Bubble's built-in Sendgrid integration

Shared email address: if you haven’t set up a custom domain and/or SendGrid account, Bubble offers a shared email address from which you can email your users. For apps in development, this solution is suitable. However, for live apps, we recommend using a dedicated SendGrid account to ensure email deliverability. The shared email address also has a limitation of 50 recipients per email.

Custom domain: when you own a custom domain, you can customize SendGrid and Bubble’s built-in email features use that domain instead. This is a more reliable solution for apps with live users.

sudsy · 2024-07-17T11:38:24.658Z

bonjour_17:

I should not be monitoring the forum 24/7 in order not to miss some important announce.

FWIW, Discourse (the Bubble forum software) makes it easy to receive email notifications only for specific forum categories or threads. For instance, I selectively “monitor” specific forum categories that especially interest me, and the Announcements category is one of them.

Just go to your forum settings and check out the Tracking sub-tab. You can opt to be notified of every post or just the first post - known as ”tracking” and “watching” respectively. (You can also subscribe directly from a thread itself.)

Anyway, just saying one doesn’t have to visit the forum and scan every message.

bonjour_17 · 2024-07-17T12:38:34.992Z

Lets agree we disagree.

Main point : 0 upfront notice.

payam.azadi · 2024-07-17T15:27:16.478Z

Hello bonjour (haha),

Let me try and provide some more clarity on the sequence yesterday.

Sendgrid blocked us for Phishing.
to prevent this from happening again, we got a code change together. During this time, we were still blocked.
This morning we shipped the code change, AND Sendgrid unblocked us.

I certainly understand it was short notice for a potentially breaking change. We decided to rush the code change in order to help maintain trust with Sendgrid, and to not run the risk of us getting blocked again the next day, and going through that whole process. This is a rare situation, and we moved urgently because of the sensitive nature and large blast radius. We generally provide 2 weeks or more of lead time for potentially breaking changes.

Hope this helps, and thanks for being a certified Bubble developer!
-Payam

pb.racing · 2024-07-17T23:56:26.068Z

Hi Payam,

Would this have been what affected my ‘send magic login link’ workflows yesterday? These wont work without setting up a Sendgrid account in future?

austin3 · 2024-08-13T20:27:38.818Z

I know I’m late to the party here, but that’s because I only recently discovered this change after an issue in a live app. Which exemplifies the point I want to make:

@bonjour_17 is absolutely correct. I appreciate that the team had a major abuse issue which led to denial of service for a vast number of users, but that doesn’t justify suddenly and without warning removing a feature that many apps rely on for their operations. Furthermore, doing so with merely a forum post to notify is unacceptable.

The team could have enforced this requirement on new apps while mass-email existing app owners notifying them of the change and giving them a reasonable amount of time to update their systems. Instead, app owners are discovering the change when it is already too late.

georgecollier · 2024-08-13T20:30:46.585Z

austin3:

I only recently discovered this change after an issue in a live app.

The manual clearly states that the default SendGrid shared email should only be used for testing

austin3 · 2024-08-13T20:36:31.711Z

As bonjour_17 previously pointed out, the manual only recommends setting up external email service, and did not even say that when many apps (including the one I had the issue on) were created.