AMA Agent Silence: No Heartbeat from one VM despite identical DCR/DCE config, IAM roles, and network connectivity

Hello Dijoux-Brezot Thibaut,

It sounds like you're issue with the Azure Monitor Agent (AMA) on your Windows VM. Since you've got two identical VMs and only one is successfully sending logs to your Log Analytics Workspace, there are a few things we can check to troubleshoot the issue further.

Given your detailed diagnostics, here are some steps you can take to investigate what's happening with the silent VM:

1. Check Extension Status: Ensure that the Azure Monitor Agent extension is properly installed and provisioned on the VM. You can do this from the Azure portal by navigating to your VM and checking in the Extensions + applications section. If it's not listed or shows provisioning errors, you may need to reinstall it.
2. Validate Network Connectivity: Since you have already confirmed connectivity with Test-NetConnection, make sure that your firewall settings are not blocking any required endpoints. Specifically, ensure that the following endpoints are whitelisted:
   • *.ods.opinsights.azure.com
   • *.oms.opinsights.azure.com
   • *.blob.core.windows.net
   • *.azure-automation.net
3. Configuration of Data Collection Rule (DCR): Verify that the Data Collection Rule associated with both VMs is configured correctly. Sometimes, a slight misconfiguration or a missing setting can cause one VM to not send data.
4. Check Heartbeat Data: Run a query in your Log Analytics Workspace to check if the silent VM is indeed sending heartbeat data. You can use the following Kusto query:

   
      Heartbeat 
   
      | where Computer contains "<VM_NAME>" 
   
      | where TimeGenerated > ago(1d)
   
   

   Replace <VM_NAME> with the name of your silent VM. If you don't see any results, there might be an issue with the agent's ability to connect and report.
5. Examine Agent Logs: Beyond the Event Viewer, you can check the specific agent log files located at:
   • C:\ProgramData\Microsoft\Azure\AzureMonitor\Logs\
   Here, you might find logs that provide more insight into why MonAgentCore.exe is failing to transmit data.
6. Review Managed Identity Configuration: Even though you confirmed that the Managed Identity is enabled and an Access Token can be retrieved, double-check if there are any specific IAM roles that the silent VM might be missing that are necessary for logging.

If you've gone through these troubleshooting steps and the issue persists, it might be helpful to gather the following additional information to provide a more tailored solution:

• Can you confirm that both VMs have the same version of the Azure Monitor Agent installed?
• Are there any specific errors or warnings logged in the agent log files that you've checked?
• Have you tried any additional testing, such as temporarily disabling any security features (like antivirus or additional firewalls) to see if that changes anything?

References:

• Azure Monitor Agent data collection endpoint

Hope this helps! Let me know if you need any more information or if you have other questions. Thanks

Hello,

Apologies for the late reply, as I wanted to perform more in-depth testing to better isolate the issue.

Here are the additional steps I have taken:

1. Change Tracking Test (Auto-generated DCR): I tried enabling Change Tracking linked directly to the Log Analytics Workspace for the silent VM. This automatically created a dedicated DCR for this VM, but it did not work. I performed the exact same steps on the working VM, and it worked perfectly.
2. Identity Validation (IMDS): I verified the token retrieval . The test was successful, and I received a valid access_token. Managed Identity is functional.

Network Validation (DCE): Connectivity to the Data Collection Endpoint (DCE) on port 443 is confirmed (True).

AVD Environment: It is important to note that the silent VM is part of an Azure Virtual Desktop (AVD) environment. I suspect a specific AVD process or image configuration might be interfering with the agent's communication.

In response to your specific questions:

Versions: Both VMs run the exact same AMA version.

Error Logs: I identified a recurring critical error: The system cannot find the file specified when the agent tries to open the local pipe .\pipe\CAgentStream_CloudAgentInfo_AzureMonitorAgent.

Security: Disabling security features/antivirus did not resolve the issue.

Conclusion: Even though the agent processes are running, the local communication "Pipe" is not being created, preventing the Heartbeat from being transmitted. Is there any known AVD-specific issue or policy that could prevent the AMA from establishing its internal communication channels?Hello,

Apologies for the late reply, as I wanted to perform more in-depth testing to better isolate the issue.

Here are the additional steps I have taken:

Change Tracking Test (Auto-generated DCR): I tried enabling Change Tracking linked directly to the Log Analytics Workspace for the silent VM. This automatically created a dedicated DCR for this VM, but it did not work. I performed the exact same steps on the working VM, and it worked perfectly.

Identity Validation (IMDS): I verified the token retrieval . The test was successful, and I received a valid access_token. Managed Identity is functional.

Network Validation (DCE): Connectivity to the Data Collection Endpoint (DCE) on port 443 is confirmed (True).

AVD Environment: It is important to note that the silent VM is part of an Azure Virtual Desktop (AVD) environment. I suspect a specific AVD process or image configuration might be interfering with the agent's communication.

In response to your specific questions:

Versions: Both VMs run the exact same AMA version.

Error Logs: I identified a recurring critical error: The system cannot find the file specified when the agent tries to open the local pipe .\pipe\CAgentStream_CloudAgentInfo_AzureMonitorAgent.

Security: Disabling security features/antivirus did not resolve the issue.

Conclusion: Even though the agent processes are running, the local communication "Pipe" is not being created, preventing the Heartbeat from being transmitted. Is there any known AVD-specific issue or policy that could prevent the AMA from establishing its internal communication channels?