Bubble can simply delete your app or subscription

Questions
1

I have an AI app with over 30,000 active users and had a terrible experience today with Bubble. I woke up to my app returning a NOT FOUND error. Initially, I thought it was a problem with Cloudflare, but everything was fine there. When I checked Bubble, I was shocked to see a free subscription and my app blocked. I couldn’t duplicate it without getting an error, I couldn’t pay for a plan to get it back online because of errors, and I couldn’t do anything at all. Fortunately, by sheer luck and coincidence, I had duplicated the app two days ago to test some things, so I managed to save my database (with two days of lost records) and my site. As an alternative, I set up a new subscription on this duplicated app today so that users wouldn’t be left in the lurch.

But why was the app blocked? I contacted Bubble support, and they said their bots found a word in my database… “ONLYFANS” :laughing:

My app is AI-based and generates images for users. The AI has generated over 570,000 images (all saved in the database). Despite having robust anti-NSFW policies (to the point of returning a black image if explicit content is detected), there’s no way to manage how prompts are saved (unless I ban word by word). So, a user probably typed “onlyfans” in the prompt, it was saved in the database, and Bubble simply ignored my other 570,000 healthy records and blocked the app.

A further tip for Bubble developers: create additional anti-NSFW protection layers in your app so that this crazy bot doesn’t get you too. Apparently, any mention of “Onlyfans” can trigger it, even if your app has nothing to do with it, hahaha.

In my opinion, they could at least give a warning before making such a drastic decision. Terrible experience, waiting for a resolution from support (which will only get back to me on Monday) so that I can restore my old app and not lose thousands of data records (including financial transactions - a huge headache for me and the users).

I would appreciate it if you could take this to the team, @fede.bubble: Measures like this (which simply block apps) should not be left to robots; there should be a manual review because such things cannot have false positives.

23 Likes
3

Can we see this response? I’d like to know if this is actually a thing cause I don’t believe this is a good practice on Bubble’s end. It’s very concerning if they actually scan people’s app’s databases for “unauthorized words” or “content”. This highly intrusive and will make me think of alternatives for databases more than usual.

I understand they don’t want suggestive content or explicit material, but this is by far going over any threshold of explicitness in my opinion. There are better ways to accomplish what they’re trying to do without being intrusive.

If this is the case, clearly Bubble can just read your records at any time they want and kill your application just because some of their “filter words” pop up. So much for privacy.

11 Likes
4

They responded to me with this:

I see that our automated systems took down your application as an anti-abuse measure. Specifically, there were mentions of “onlyfans” in your Live app data.

Please note that this effort is to keep the platform a safe place for our users. Building an app with the inclusion of adult content is strictly prohibited on the Bubble platform.

As per paragraph 3 of our Terms of Service: You may use the Platform to create applications for your customers. You can’t use the Platform for illegal purposes, to post pornography or hateful content, harass others, or do anything else your mother would not approve of."

Note that under FOSTA-SESTA, ISPs or platforms like Bubble are liable for any sex trafficking that takes place on their infrastructure by third parties. Any app that runs the risk of being misused by end-users along these lines would be taken down.

In the case that an app does not run afoul of FOSTA-SESTA – and bear in mind that we’re not lawyers and cannot provide you with legal guidance – it’s your responsibility to ensure that the application does not facilitate or promote illegal, harmful, or offensive content on our platform. If we find that an app on our platform does, we may suspend or terminate the creator’s use of Bubble, suspend or delete their applications, and cancel their account with Bubble.

So, after I explained my case, they said:

Please give me some time to investigate this matter further and share this information with our team. Although our team is out of the office today, I’ll be sure to follow-up with an update as soon as I can!

In the meantime, I recommend removing any mention of “onlyfans” from your Live app data along with adding a Terms & Conditions page to your app that explicitly prohibits adult content generation.

I checked the database and there really was a mention of “onlyfans” made by a user (I’m not a robot, I can’t analyze the thousands of data records for various NSFW words that exist in the vocabulary).

I really don’t know why they didn’t just send me an email alerting me. I would have been happy to make some corrections and delete those records. Blocking the app and leaving my users stranded made me very concerned :grimacing:

4 Likes
5

This is definitely bad practice. Especially since it’s automated.

  1. They don’t know for sure you’re supplying adult/explicit content.
  2. They instantly shut down your app based on a singular keyword.

This is harmful in so many ways.

Now, on their side – I can understand they don’t want to take a chance with what COULD BE explicit/adult content, and they want to take action immediately before possible issues arise on their platform itself, but I’m not sure.

https://directus.io – don’t have worry about anyone touching my data. I’ll make a guide on it later, but it’s basically a Xano replacement (and more).

EDIT: Why doesn’t Bubble supply us these keywords somewhere so we can take proper actions to prevent this in the future? If there are any, I’d like to know where to find them.

Another edit: What if made a blog about why adult sites are bad and reference their sites? Is that illegal? No. Is that supplying adult content? Maybe, but not on Bubble itself, nor directly linking to said sites, but just “referencing” them. Would this be grounds for immediate app removal?

5 Likes
6

That is very concerning.

1 Like
8

Yep. They can access our data without our consent.

Would also assume this is why medical fields cannot (or will not) use Bubble as an app.

Just imagine one Bubble employee being a bad apple. The damage they could cause.

2 Likes
9

This is why i encrypt and dencrypt all my database that contains users custom text. You never know what they can write.

10

That sounds like a lot of extra legroom for something that shouldn’t need to be encrypted (unless of course you’re running a fully encrypted app?). Either way, this shouldn’t be an issue. Our databases should be secure. Can’t imagine the extra WU you have to use just do that.

4 Likes
11

100 percent agree. Bubble should have a person in the loop to discuss this with the account holder (who is the customer) there’s far too many edge cases for this.

So we can shut down any Bubble hosted website by just chucking :face_with_symbols_over_mouth: a word into a client data field? That’s insane. And obviously Bubble can stop this given a keyword has been mentioned on this forum which is hosted in Bubble now if I’m right??

2 Likes
12

@josh @emmanuel You need to review this topic and we are waiting for a reply about that. (Sorry @fede.bubble , but this one need a response from founders…)

9 Likes
14

This makes me wonder… would this trigger potential abuse if I were to register “bad names” on other peoples apps? That would mean I could turn off a bunch of Bubble apps just by adding a bad name into their database or somewhere, and depend on the “automatic system” to turn off their app.

Whoop we just opened a can of worms.

6 Likes
15

As much as I love Bubble, it is one of the strangest and unprofessional tech companies I have come across.

It’s as if the management are yet to grasp the fact that people’s livelihoods rely on every action they take, which would call for careful thinking before taking any action. But no, from forcing half baked updates and breaking live apps, to using our production apps as testing grounds to now deleting entire businesses because of one brand name!?

I’m all for keeping Bubble a clean and safe place, but the lack of thinking from Bubble here, or a single employee being given the freedom to do what’s happened here (which is even scarier) is for the lack of a better word, baffling.

For a company that kicked the small businesses to the curb with the new pricing structure to go after enterprise clients, I’m very interested to see their justification for intruding on a live database. That’s a big no no for enterprises.

7 Likes
16

I used to be afraid of getting shot in the street, but now I’m afraid of a user putting ‘Only’ as their first name and ‘Fans’ as their last name in the account registration :joy: :grimacing:

2 Likes
17

They clearly have a keyword list somewhere. Doubt they’d give it out publically for “security” purposes.

18

This is the same reason so many hosts are leaving the Airbnb platform, or why sellers are walking away from selling on Amazon. Way too much control for the company and they can literally shut down your livelihood at any moment (if you’re only getting bookings through them) and put you out of business. And it can be for a silly reason like this. I love the idea of Bubble, but they’re doing some really terrible practices right now that are making me not want to build with them. I’m also currently locked out of paying or launching on Bubble. Support hasn’t given me a reason, but I’ve also been trying for three weeks to upgrade my account. It’s been super frustrating as I can’t pay and can’t launch.

1 Like
19

FYI bubble recently added a setting for who can see what in the company

image
image2386×284 25.4 KB

1 Like
20

Bubble never ceases to amaze :joy: Now anyone can deliberately write one of the forbidden words and the app will be banned. And we don’t even know exactly what words are banned

1 Like
21

Were you using the Bubble storage or external? It seems a bit bizarre not to allow an AI Adult content generator. Massive over-reach to treat such as a porn site. If there are disclaimers to the user about what the site is and nothing pokes you in the eye prior to logging in / agreeing to the terms then I don’t see why that should be a problem.

I would really like this clarified

You can’t use the Platform for illegal purposes, to post pornography or hateful content, harass others, or do anything else your mother would not approve of.

  • Please define what you class as pornography? Is an AI image generator which is clearly catered to the adult market letting adults make AI images of other fictional adults classed as pornography?
  • Please define hateful content? Who polices this, what are the appeals process etc?
  • Do anything else your mother would not approve of?! This is a bizarre statement in an actual terms of conditions. It should not be fluid and open to interpretation, it should be outlined clearly what is and what isn’t possible. What if my mother is an alcoholic, gambling, degenerate who has her own Onlyfans? You get the idea of why such a statement is absurd.

I was under the assumption that anything we made with Bubble, the responsibility was on us and if it wasn’t illegal then it was fair game. I did not notice this term change but this is a massive issue and requires clarification.

Basically who defines these things? What are the processes for appeal and what are the processes for suspension. I am super concerned here.

@fede.bubble I think this really requires a lot of clarity and a proper statement as the current terms are incredibly vague.

Any product which has an AI image generator would most likely let the end-user enter a prompt and a negative prompt. Any end-user could write one of these unknown words and get banned. How do we know what these words are? Could a word be misconstrued? The whole process needs to be clarified and clarified urgently.

Why are we so afraid of treating adults like adults in 2024? It is my understanding that Bubble uses AWS as a backend. AWS quite rightly won’t allow illegal content (AWS Acceptable Use Policy) but allows pornography if legal and consensual.

My final point would be what is the definition of pornography per Bubble? You see how everything being ultra vague is unsettling for people who would want to develop on Bubble.

Urgent clarification is required here.

6 Likes
22

So, to delete any Bubble app it’s enough to registrate there and post something with keyword “onlyfans” or mention black-orange “youtube” for adults :thinking:

1 Like
23

I’m in chats with a client about an app for lawyers and managing their cases. What if these restricted words were in their documentation? This is pretty wild and I upvote that we need clarification.

10 Likes