An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
Five-minute response times in two regions are not normal client-side latency, especially if the same payload previously completed quickly. First separate caching from inference performance: prompt caching only applies when the prompt is at least 1,024 tokens and the initial 1,024-token prefix is identical. Cache entries can disappear after 5–10 minutes of inactivity, so a zero cached-token count alone does not prove an incident.
For the slow requests, capture timestamp, region, deployment name, request ID, status code, total input/output tokens, and whether streaming was enabled. In Azure Monitor compare Time to Response, Time to Last Byte, Model Availability Rate, 429s, and 5xx errors for both deployments. Also check personalized Service Health.
As an immediate mitigation, route to a healthy secondary deployment, use streaming, lower the maximum output-token limit, and honor Retry-After with exponential backoff for 429s. Global Standard can improve availability but may still have latency variation; PTU is the option for predictable capacity. If five-minute latency persists on minimal test prompts, open a support case with the captured request IDs—only Microsoft can investigate shared inference capacity.