OpenAI offers many ways to approach using their chat models in software the main consideration is whether to use a “bulk response” or a “streaming response” in your project.

The most obvious difference is that with a “bulk response” there is a slight delay before your user will see anything from the model and then the entire response will appear all at once, whereas with “streaming response” the model will respond with each token (3-4 characters including spaces and punctuation, etc.) as they are generated. Many users prefer to see a “streaming response” because it seems both faster and more natural instead of seeing a paragraph or more suddenly appear after a pause.