Well said. I expect there are many true-positives that have been taken down and protected all of us in terms of ISP auto-blocking and the platforms (and therefore our) reputations. Sorry for OP though.
4 Likes
Then it needs to be said. Not supposed and hoped for.
There is no shame in emphasizing the truth, if OP lies. Then we need to hear it.
Just to clarify, I understand the concerns about thinking this could all be a lie on my part, but Iâm sure that if it were a lie, I wouldnât bring up such a matter on an open forum â a topic that has already garnered over 800 views in just a few hours and causes uncertainty among developers. The last thing I would do is jeopardize the transparency or integrity of my business (which, by the way, can be accessed at any time by clicking on my profile). I only created this topic because my site with thousands of users became inaccessible due to this exact issue (confirmed by support):
And, after several hours of waiting and a lot of headache trying to come up with explanations for my users, I received confirmation that it was all a mistake. Confirmed by the support team and a team member in the thread itself:
I am absolutely certain (and Iâm not hallucinating), I do not have a site that sells explicit photos. I can guarantee that. This was flagged because of the database and not the content or purpose of the site; hence it was a mistake.
In fact, I feel a bit sorry for the pressure Laura is under and I also understand wanting to see both sides of the conversation, but this topic was not created to attack Laura or Bubble (just to clarify for the support team so they donât think a rebellion is starting here, haha). We, as users, just want clearer explanations and to avoid future headaches. A simple âno, youâre misunderstanding and it was a huge mistake; we wonât punish your app if someone types âonlyfansââ or âthank you for the suggestions, and weâll change how our automated bots make decisions for appsâ would solve the main problem of the topic.
16 Likes
Just a note for you: the link to squaadinc.com in your bio still goes to the ânot foundâ screen. Might be the same across other places where you have linked your siteâŚ
I totally agree, I said something similar in an earlier message, and in all honestly I expect it WONT happen again.
But⌠unless this is addressed properly a tarnished reputation means a lot to new people thinking of using bubble for their business.
I get consultations all the time about the one topic, âis Bubble is good my appâ and they all want to know its weaknesses. If these people catch wind of random app deletion from âBig Bubbleâ and I donât have a good explanation then Bubble has done a disservice not clarifying and addressing the problem.
I literally lost a client after 9 months of work because a MASSIVE company wanted to use the service we built and they found out it was built in Bubble and so they blindly rejected it, which in turn put that client into a spin and they left Bubble and me.
This wasnât Bubbles fault but they should be doing their best to be seen as trustworthy and strong, and the rarity of this problem wont matter to people that heard on the grape vine that Bubble randomly deletes apps.
So like you said, this does need to be clarified.
6 Likes
Hi @laura.oppenheimer ,
I am sorry to keep pushing this point but I think it is an important one to keep pushing. If these summaries in italics are providing inaccurate information which they are based on the terms then I am surprised that either the terms arenât updated or the summaries arenât taken down.
Totally get that these summaries were made in good faith but when the summary is creating new restrictions which simply donât exist in the terms or the AUP then it is problematic for users. The silence on this will not simply make it go away.
I, like others are very concerned if Bubble is flagging apps which say things like OnlyFans, wherever you stand on the moral compass, there is absolutely nothing in your terms which would prevent someone from literally making an OnlyFans clone at this point so I am trying to get to the bottom of this and clarity for everyone by keep driving this point.
I am not going to drop this as it is something that should be discussed / remedied / clarified. If Bubble support and yourself are quoting summaries which have no basis in fact or are not legally effective then that is a massive problem and it is incredulous that you feel that this should be swept under the carpet.
2 Likes
Hello, I came here to bring a general update on this topic after a long conversation with Bubble support.
In fact, there were more NSFW prompt records than just OnlyFans in my app. Support helped me locate them after reversing my suspension, and Iâve already deleted most of these records. Iâve put my app into maintenance mode and am working on implementing filters in the workflow to prevent obscene words from being saved in the database, especially since my app involves AI and there are many ill-intentioned individuals.
I tried to inquire whether this automated system had broad access to our appâs DB or if there were specific keywords that made us targets, but apparently, this is an internal security matter for them, and they are unlikely to disclose such information to us, as Laura herself mentioned.
Overall, my only remaining concern is how deep this could go. For example, in a chat app⌠if a couple has intimate conversations, will I be affected? I canât just ban these users and say, âoh, I looked at your personal conversations and you canât talk about your nipple with your spouse.â
I believe itâs in everyoneâs interest, and Iâve emailed Bubble requesting that these automated processes give app owners advance notice before taking such drastic actions, so owners can take action and delete any harmful records, which often go unnoticed amidst thousands of new lines in the DB each day.
8 Likes
Iâm amazed people seem to be taking for granted bubble wonât allow our apps to have explicit content. If you need a specific app to be SFW for your own reasons and build protections in yourself of course I get that. I may have to just bite the bullet and learn to code though if bubble is enforcing that themselves.
Outside of the understandable subset of âbuilt on bubbleâ free plans and apps with no custom domains when at least thereâs arguably some branding skin in the game(still crazy if it applies only here frankly), I canât believe theyâd go above and beyond the AWS policy of anything legal goes. Am I seriously in a room full of people who make websites and (almost, hi Stuart)nobody is surprised or concerned to hear that we canât make a porn site lol?
This statement from Laura is doing a lot of heavy lifting right now because a less crazy possibility is wide open. Somebody, maybe an AI maybe a human I donât care, made NON LEGALLY EFFECTIVE summaries of the terms. Thatâs a fact we know that. Support agents and Laura have been quoting the summaries and thatâs deeply embarrassing for all involved especially with that mother line. We need to get clarification though if that particular summary is a mistake and nothing in the underlying LEGALLY EFFECTIVE terms outlines a ban on legal explicit content.
Most of the internet is porn, this just wouldnât be a sensible policy for bubble when they have no more reason to expect a publicity backlash for our apps legal content than AWS does. I refuse to believe its true until we get the clarification promised above. If it is true,I am most certainly out.
ps: nice to see the thread came back to life on just this issue while I drafted this, good man Stuart, Arthur could you please ask those support agents to justify the ban with reference to the actual terms not the non binding(FOR YOU!) summaries?
2 Likes
Are you now saying that Bubble took your app down because of genuine detection of abuse? And not simply because of the word âonlyfansâ?
1 Like
Bubble was clear when he said, days ago, that he detected a mention of âonlyfansâ in the app (thatâs all they told me. Those were the only details they gave). However, they unlocked my app and in a more detailed audit, I found more NSFW records from malicious users, partly because all prompts were saved in the database without exceptions. The filtering system I used was at the point of displaying the image to the user (returning a black image if sensitive content was detected, a standard anti-NSFW feature of Stable Diffusion), not in preventing the prompt from being recorded in the DB. I still donât know the real root cause of the suspension and I donât know if there are specific keywords; they havenât clarified that yet and I donât think they will to preserve their security, as Laura mentioned
Are we talking a handful of records, or 1000âs? (the ones you found yourself) Just to get an idea.
1 Like
Question: has/will Bubble provide a list of words and phrases its automated search bot is looking for? Or possibly an API that we can integrate into our solutions? In addition, it would be a better approach to notify the bubble account holder with a deadline to resolve before suspension or deactivation, allowing an acceptable time for resolution.
Iâm very concerned around the lack of âhuman-in-the-loopâ approach by Bubble.
3 Likes
Definitely 1000âs, several NSFW prompts were typed by some users (though it doesnât even reach 2 percent of the total saved prompts, which was over 570 thousand). There was one user alone (after the audit and discovery, banned) who send over 250 prompts with the word ânake*,â for example⌠but thatâs the real issue I keep asking support about. Even though the images arenât displayed to the user in the interface (due to filters), but the prompts are saved in the database, am I at risk? Iâm still awaiting a response.
In my mind, I think âonlyfansâ might be some kind of forbidden word.
1 - Why did support specifically specify âonlyfansâ in the message they sent me?
2 - Why did support instruct me to delete all records containing âonlyfansâ exclusively and then unlocked my application right after?
Actually, thatâs what I said: if I have a Discord-style app with over 300 thousand active users exchanging messages, inevitably several will type NSFW things. Bubble canât just immediately ban the app; they need to contact the owner to find a resolution if adult content is so alarming to them.
5 Likes
Lindsay Iâm new to the forum and not sure if that âbubble certifiedâ title means you work here to some extent. If so, can we take this as the clarification Laura was talking about regarding the terms? Having NSFW language on your app is abuse for real?
The support agents and the auto system made a mistake blocking his app. Theyâve been working with the non legally binding summary of the terms. Amazingly its an open question right now(again unless this is you confirming bubbleâs position) whether anyone actually does need to be filtering out explicit content other than for their own reasons. Bubble needs to hastily review the actual terms not the summaries and clarify if they are moderating the content on our apps beyond banning illegal activity
1 Like
@arthuribeiro Ok thanks, 1000âs puts a slightly different colour on things. If it is un-restricted content from the internet around image generation, I am sure that ânakedâ and âonlyfansâ were the least of the problems. Here in the EU, the fashion is to make platform providers responsible for managing illegal content, so this will probably become more of a thing.
@GaryIreland I donât work for Bubble. I work on Bubble as a freelance developer. I am not a lawyer. Maybe take your questions on Bubbleâs T&Cs up directly with them.
1 Like
Interesting, I always knew things were pretty strict in the EU. I saw an article saying that in Germany the police can send you a fine within a few days if you consume torrents, for example (Iâm not sure if itâs really true). Here in Brazil, where I live, things are usually more relaxed, and weâre not used to such strictness, so itâs quite a shock that Bubble has this much control over my application and the freedom to access my database
1 Like
Iâm not sure why text is a problem and why Bubble should care. I can curse on Teams all I want with no issue.
I do understand why images are an issue. Even Microsoft and Google do autoOCR on all stored images in their personal and Enterprise storage services to ensure specifically no child porn images are present. Healthcare use cases where you may have photos of skin conditions are a challenge for these processes.
Iâm now very concerned as I have an app that lets Enterprise teams manage status updates, meeting notes, and tasks. That includes internal security teams where they will be discussing NSFW incidents that are occurring within their companies. They also have the ability to include screenshots in their meeting notes as well. Can I get banned over a clientâs legitimate internal conversation in my app? It appears so.
This certainly becomes an argument for adding your own data encryption within your app.
2 Likes
Yes, if a service has a specific purpose to create a CSAM portal or any illegal content, I 100% agree on not giving space for it on Bubble. The issue is that there are doubts about how this is judged, especially when it involves image generation. In AI-generated images, a user can choose between 2 methods: processing the image with the anti-NSFW tool from Stable Diffusion (which is quite crude, by the way) or analyzing the image and returning a âNSFW detectedâ label.
In this latter example, which is more common, most of the time to set up an NSFW filter, we use NSFW detection libraries, like the one Zeroqode itself offers in the form of a plugin: NSFW Detection Plugin
In this case, the image inevitably has to pass through Bubbleâs servers and be saved in the database, which could potentially give room for their automated services (which nobody really knows how they work exactly, hence Iâm working only with assumptions) to block the application.
I truly hope that Bubble changes this approach and starts notifying the app owner in advance, so they donât get surprised and have to come up with explanations for thousands of customers.
1 Like
Suggestion: you can collect the prompt from the user and run it through OpenAIâs moderation API (free), and if it passes, you can process the prompt (store, use, etc.).
8 Likes
@travis.polland Great suggestion. That should become a recommended best practice for all the Prompt based apps that are being built.
1 Like