Using stripe API to create customers for sellers

I only see an option to create a customer in the API.

But, wont that make them my customer, rather than the customer of my seller?

Short answer - you can absolutely create customers of your sellers. Technically, they are all users of your marketplace.

Check out the documentation on Stripe Connect. It’s all there.