Azure Foundry deployed model exploded in latency all of the sudden.

Hello @Julius Dietmar ,

Welcome to Microsoft Q&A .Thank you for reaching out to us.

Thank you for sharing the telemetry and details regarding the embed-v-4-0 latency increase.

Based on the information provided, the behavior indicates a significant latency anomaly compared with the reported normal response time.

The approximately 60-second plateau closely matches the configured one-minute application timeout. Therefore, these values may represent requests reaching the client/application timeout rather than exactly 60 seconds of model-processing time.

Regarding the Central US and deployment type - The deployment SKU/type should first be confirmed before associating the issue with Central US.

Foundry deployment types have different processing and performance characteristics. Global deployments can dynamically route inference across available Azure datacenters, while Data Zone and geography-based deployments use different processing boundaries.

If the affected deployment is confirmed as GlobalStandard, the Central US location of the resource does not by itself establish that inference was processed in Central US. Global Standard can also experience greater latency variability at high, consistent workload volumes. Provisioned deployment types are intended for workloads requiring more predictable throughput and lower latency variance.

Regarding Service Health showing no degradation - The absence of a Service Health incident does not rule out a workload-specific performance regression.

Service Health covers broader service issues, planned maintenance, and health advisories, while Azure Monitor can detect workload-specific regressions such as increased errors or latency

The following are the troubleshooting steps

1. Please confirm the deployment and incident details
   1. Resource ID and deployment name
   2. embed-v-4-0 model/version
   3. Deployment SKU/type
   4. Resource region and endpoint/API
   5. Exact start and end timestamps in UTC
   6. Configured application timeout
   7. Exact metric represented in the supplied latency graph
   8. Whether the latency returned to normal without an application or deployment change
2. Reviewing Azure Monitor for the same UTC interval Kindly check for :
   • ModelRequests, filtered or split by StatusCode
   • ModelAvailabilityRate
   • HTTP 429, other 4xx, and 5xx responses
   • Latency/performance metrics available for the applicable deployment type
   • Region, ModelDeploymentName, ModelName, and ModelVersion dimensions where available
   • Request/response and trace logs, if diagnostic logging was already enabled
   ModelRequests supports status code, region, deployment, model, and model-version dimensions. ModelAvailabilityRate provides availability information based on server errors A rise in 429 responses can help identify throttling or rate-limit conditions and, depending on deployment type, possible capacity constraints. Increased 5xx responses together with reduced ModelAvailabilityRate can indicate a service availability problem

Please check if the following workarounds help-

1. If sustained 429 throttling occurs, use exponential backoff and honor the Retry-After header.
2. If request volume or concurrency increased during the affected period, reduce the workload temporarily and compare latency against the normal baseline.
3. For consistently high workloads where low latency variance is important, consider a provisioned deployment type where supported. Provisioned deployments provide reserved processing capacity and more predictable throughput
4. For resiliency-sensitive production workloads, a multi-region deployment design can provide an alternate path when a deployment or region becomes unhealthy

The following references might be helpful , please check them out

• Understanding deployment types in Microsoft Foundry Models - Microsoft Foundry | Microsoft Learn
• Stay informed about service health regressions - Microsoft Foundry | Microsoft Learn
• Monitor Model Deployments in Microsoft Foundry Models - Microsoft Foundry | Microsoft Learn

Please let us know if the response was helpful

Thank you