The Bubble team has stated earlier that yes, adding plugins might slow down your site, even when they’re not in use. I’d assume it depends on the nature of the plugin though, so the best way to know how it affects your app is to test it. Personally, I haven’t seen any major difference.
But in general, uninstall the plugins that you don’t need.