Application Gateway for Containers: Endpoint update timeout issue during pod deployment

Question

Application Gateway for Containers: Endpoint update timeout issue during pod deployment

Jason Kolios 0

Problem description

I am experiencing issues with my Application Gateway for Containers integrated with an Azure Kubernetes Service (AKS) cluster. Starting around 08:35 UTC on October 1, 2026, the in-cluster ALB controller stopped accepting configuration and endpoint updates, returning OPERATION_STATUS_TIMEOUT errors. Successful updates occurred until approximately 08:30 UTC, after which all subsequent attempts timed out. Restarting the ALB controller deployment and deploying new pods temporarily restored the update functionality, but the problem recurred. I need assistance understanding the root cause and how to prevent or resolve this issue permanently.

Environment

Application Gateway for Containers integrated with an AKS cluster in the specified region; ALB controller version 1.10.37; incident occurred on October 1, 2026; the exact region and SKU/tier are not specified in the case information.

What I've already tried

I documented the incident timeline, noting that successful updates occurred until 08:30 UTC, and from 08:35 UTC onward, all endpoint and configuration pushes failed with OPERATION_STATUS_TIMEOUT. During the incident, I restarted the ALB controller deployment and triggered another application deployment, after which updates began succeeding again. I also reviewed the case materials for troubleshooting references, including checks on the controller pod health, logs, and Kubernetes resource status, but the available logs from the incident period are not recoverable. No configuration changes or cluster updates were made shortly before the incident.

Current status

Currently, the update failures have ceased after restarting the ALB controller and redeploying applications. I am seeking insights into the root cause of this timeout issue, guidance on how to prevent recurrence, and recommendations for ongoing monitoring or configuration adjustments to ensure consistent operation.

0 comments

1 answer

Your answer

Answer 1

Let me repost the initial support request here, because this LLM-generate summary was less than helpful.

"This Azure Application Gateway for Containers (AGC) stopped accepting updates from the in-cluster ALB controller for at least 20 minutes at the time we specified on the ticket. Every endpoint and config push from 08:35:42Z onward returned OPERATION_STATUS_TIMEOUT, after an unbroken run of successes until 08:30:48Z. This resulted in production downtime for our applications, since we deployed new application pods, so AGC endpoint updates were needed.

20 minutes later, after restarting the ALB controller deployment on the cluster and trying another application deployment, the AGC updated correctly. However, since we can't be sure that the issue is resolved and we can't afford production downtime, we have started a deployment freeze since then."

More information beyond the automated summary:

The region is West Europe. There is no SKU/tier to report, AFAIK there's only one AGC SKU.

We have faced a similar issue before, it was https://github.com/Azure/AKS/issues/5778 , Azure support has assured us that it doesn't affect our current alb-controller version. The docker image tag for our alb-controller is "`mcr.microsoft.com/application-lb/images/alb-controller:1.10.37` It's completely unclear to me which actual alb-controller software version this maps to. The assumption was that the docker image tag follows the software version.