Hello,
My app includes a chat between users and I want to be able to pre-emptively block or edit chat messages that have profanity in them.
I found some compiled lists of vocab online. I’m assuming I could go about doing some regex search? Any suggested method?
I’m a little worried about the lag scanning for words each time.
When a message is created you could do a call to Google NLP API (or a similar service) for sentiment analysis via a database change trigger and flag the message if needed. Won’t be perfect, but should be very quick (1 or 2 seconds overall).
You’ll probably want to avoid triggering that API call client side and waiting for the response before publishing the message, as it would impact the message creator’s perceived performance of your app. However, you can make it so the message becomes visible to others only after it’s been checked by the API call.
The regex method won’t be able to account for typo, or ways users can easily come up with to go around a list of forbidden words, whereas AI should be substantially harder to trick.
Great approach!
In order to avoid slowing down the app this could take place after messages are posted and creating flags to be sent to app admins
Right. So there’s a risk it would appear before it is detected. So there’s a trade off. But once detected—in seconds—it could be removed / flagged.
What I’m worried about is what constitutes profanity is very subjective. I’m not sure as to what the Google API does in that regard, haven’t looked into details yet. But when looking into these lists of words, there are plenty of words that only would be slur in a certain context.
Initially I was considering just censoring certain words that are clearly and unambiguously words that have no reason to exist on my app. However I hadn’t considered the flagging / moderation option.
I guess I won’t have such traffic that my hands would be full with content moderation. So actually it may be a good approach to start with.
If it get to the point where there’s so much I can’t handle it, then I’ll open a bottle of Champaign first thing! 
I have seen your many posts on your app and I can see it is a very interesting project!
Best of luck in your journey 
Hey 
@umiumansa
Are you already using this? PurgoMalum Profanity Filter Plugin | Bubble
I haven’t used it before, but maybe you can at least delay the post until after this is checked through on the back end, or something like that.
Anyways, hope that helps! 
For All Your No-Code Education Needs:
I didn’t know! Thanks 
This topic was automatically closed after 70 days. New replies are no longer allowed.
