stream to true to receive incremental output. Clients should handle a complete stream and a terminal event.
Chat Completions stream
choices[0].delta.content and stop when the [DONE] marker arrives. Usage can be returned in the final usage chunk when enabled by the model configuration.
Responses stream
POST /v1/responses uses named events:
response.output_text.deltacontains a text fragment indelta.response.completedmarks successful completion and includes the public model and usage summary.
Cancellation and retries
- Abort the HTTP request when the user stops generation.
- Do not retry a stream after partial output unless your application can safely start a new request.
- Retry a connection failure or
502/503with exponential backoff and jitter. - Do not retry
400,401,403, or404without changing the request or credentials. - Treat
429as a policy response and wait using the reset headers when present.