I use AD for our org setup. Only users in our tenant have access. There’s a few plugins for this, but we ended up setting up the API connector to control the scopes of what users have access to.