API calls for OpenAI are slow for lists

Need help APIs
1

I’ve noticed OpenAI calls for lists (as the output) seem to be a lot slower lately. Aso, it seems the more list items requested, the slower the response.

I have an application where I would like a list of 10 (that is the right size for my application) and users choose among the list generated.

Please advise thoughts on how to speed up this process. I can think of the obvious. Reduce the list size (not ideal) and try to simplify the request (albeit the request is fairly simple now).

What am I missing?

2

Missing: OpenAI demand has skyrocketed since their latest releases, so their servers are slower to return a response… Try using it via MS Azure instead.

3

Sounds like rubbish. Roughly speaking it’ll respond at a fixed TPM depending on server load etc. The content of the message or length of it doesn’t appear to have a factor (except perhaps for extremely long content).