Standard unit testing framework fails on Linux: DllNotFoundException for kernel32.dll (GlobalMemoryStatusEx), then tests hang

Richard Coleman 40 Reputation points
2026-11-09T10:31:21.6533333+00:00

Describe the bug

On Linux, Logic Apps Standard unit tests created with the VS Code extension's testing framework (Microsoft.Azure.Workflows.WebJobs.Tests.Extension) fail or hang. On Windows the same tests pass. Running the unit tests in a CI pipeline on a Linux agent is therefore not possible, and the documentation doesn't mention any operating system requirement.

The workflow engine reads memory utilisation through kernel32!GlobalMemoryStatusEx, which doesn't exist on Linux:


Microsoft.Azure.Workflows.Common.ErrorResponses.ErrorResponseMessageException: Workflow processing failed. Error: 'Workflow '<id>' is unhealthy. Error:'System.DllNotFoundException: Unable to load shared library 'kernel32.dll' or one of its dependencies. ...
<test bin>/runtimes/linux-x64/native/kernel32.dll.so: cannot open shared object file: No such file or directory
...
   at Microsoft.Azure.Workflows.Common.PerformanceCounters.WebAppWorkerMemoryUsageCounter.NativeMethods.GlobalMemoryStatusEx(MEMORYSTATUSEX& lpBuffer)
   at Microsoft.Azure.Workflows.Common.PerformanceCounters.WebAppWorkerMemoryUsageCounter.ReadCounter()
   at Microsoft.Azure.Workflows.Common.PerformanceCounters.FlowPerformance.get_CurrentMemoryUtilization()
   at Microsoft.Azure.Workflows.Data.Configuration.EdgeResourceGovernancePolicy.<>c.<.ctor>b__21_0()
   at Microsoft.Azure.Workflows.Data.Configuration.EdgeResourceGovernancePolicy.ShouldThrottleWorkerPulsation(JobTriggerEvent trigger, String& throttledTraceMessage)
   at Microsoft.WindowsAzure.ResourceStack.Common.BackgroundJobs.Execution.JobDispatchingWorkerRegistry.CanAcquireDispatchingWorker(JobTriggerEvent trigger)
   at Microsoft.WindowsAzure.ResourceStack.Common.BackgroundJobs.Notification.IBackgroundJobsNotificationChannelExtensions.CanAcquireDispatchingWorker(...)
   at Microsoft.WindowsAzure.ResourceStack.Common.BackgroundJobs.Execution.JobExecutionEngine.PrepareJobDefinitionForImmediateExecution(...)
   at Microsoft.WindowsAzure.ResourceStack.Common.BackgroundJobs.Execution.JobExecutionEngine.ScheduleJobExecution(...)
json

Symptoms

The failures follow a fixed cycle, regardless of what each test does. Across 33 tests in 4 test classes the results were fail fast → pass → pass → hang eight times in a row:

  • Fail fast (2–3 s): the exception above, surfaced as "workflow is unhealthy".
  • Pass (~3 s each): the next tests pass normally.
  • Hang (~10 min): the next test never completes and fails with Microsoft.Azure.Workflows.UnitTesting.Utilities.WaitTimeOutException: Exceeded maximum wait time. Message: '<null>'.

That cycle matches how FlowPerformance.CurrentMemoryUtilization works. When the cached sample has expired (Microsoft.WindowsAzure.Performance.MemorySamplingWindow, default 10 s), it moves the expiry forward before calling ReadCounter(), and nothing catches the exception. So on Linux roughly one call every 10 seconds throws, and every call in between gets the cached value:

  • If the throw happens while a run is starting, the test fails fast.
  • If it happens part-way through a run, while a later action is being dispatched in the background, the run never completes and the test hangs. This part is inferred from the stack trace and the timing; I haven't traced it directly.

Root cause (from inspecting the 1.0.3 assemblies)

  • FlowPerformance.GetMemoryUsageCounter() always returns new WebAppWorkerMemoryUsageCounter(), and that class P/Invokes kernel32!GlobalMemoryStatusEx.
  • By contrast, FlowPerformance.GetProcessorUsageCounter() already checks IsContainerAppEnvironment (MANAGED_ENVIRONMENT / CONNECTED_ENVIRONMENT) and uses FlowProcessorUsageCounter.CreateLinuxContainerWorkerCounter() on Linux. The memory counter has no equivalent.
  • Memory-based throttling can't be turned off to avoid the call. In the embedded AppSettings.resources, Microsoft.WindowsAzure.ResourceStack.JobResourceGovernance.MemoryBasedThrottlingEnabled is fixed at value="true", with no appSettingName or hostSettingName. So neither app settings, environment variables nor host.json can override it.

To reproduce

Reproduced on .NET 10 and .NET 8, in Microsoft's own SDK images:

  1. Create unit tests for a Standard workflow with the VS Code extension. Our test project uses Microsoft.Azure.Workflows.WebJobs.Tests.Extension 1.0.3 and MSTest 4.2.3, targeting net10.0 (also tried net8.0).
  2. Build it and run dotnet test on Windows: all 33 tests pass, on both net10.0 and net8.0.
  3. Run the same compiled tests on Linux:

   docker run --rm -v "<logic app workspace>:/src:ro" -w /src mcr.microsoft.com/dotnet/sdk:10.0 \
yaml
 dotnet test Tests/<project>/bin/Release/net10.0/<project>.dll \
   --results-directory /tmp/results --blame-hang-timeout 3m

The first test fails with the DllNotFoundException above, which the engine also logs as Workflow '<id>' validation and creation failed. Error: 'Unable to load shared library 'kernel32.dll'.... Two tests pass, then the fourth hangs until --blame-hang-timeout stops the run. | Image | Runtime | Result | |---|---|---| | mcr.microsoft.com/dotnet/sdk:10.0 (Ubuntu 24.04.5, x86_64) | 10.0.14 | fail (4 s), pass, pass, hang | | mcr.microsoft.com/dotnet/sdk:8.0 (Debian 12, x86_64) | 8.0.33 | fail (10 s), hang | | Self-hosted Linux x64 GitHub Actions runner | 8.0.33 | fail, pass, pass, hang repeated 8 times; 16 of 33 failed |

Which tests pass, fail or hang depends on timing, because of the 10-second sample window. In the .NET 8 container the first test took 10 s, so the window ran out during the second test and that one hung.

Expected behaviour

Either:

  • the unit tests run on Linux, so they can gate CI/CD on standard Linux build agents, or
  • the documentation clearly states that the unit testing framework needs Windows, including when it runs in a pipeline.

Suggested fix

  • Give the memory counter the same Linux path as the processor counter. On Linux, read memory from /proc/meminfo or cgroup limits, or use GC.GetGCMemoryInfo(). At minimum, skip memory throttling when the counter isn't supported.
  • Catch exceptions from ReadCounter() in CurrentMemoryUtilization, so a counter failure can't break or hang a workflow run.
  • Under the unit test host, consider not applying memory-based throttling at all.

Environment

  • Microsoft.Azure.Workflows.WebJobs.Tests.Extension 1.0.3 (the latest on nuget.org at the time of writing)
  • net10.0 (runtime 10.0.14) and net8.0 (runtime 8.0.33) test projects
  • Fails on: mcr.microsoft.com/dotnet/sdk:10.0 (Ubuntu 24.04.5), mcr.microsoft.com/dotnet/sdk:8.0 (Debian 12) and a self-hosted Linux x64 GitHub Actions runner
  • Works on: Windows 11, net10.0 and net8.0

Workaround

Either run the unit tests on a Windows agent, or stop the engine from reading memory on Linux before any workflow runs.

The engine already has the path this needs. FlowPerformance.CurrentMemoryUtilization first checks MemoryUsageCounter, and if it's null it returns the cached sample without reading anything. Setting the counter to null on the shared FlowPerformance instance therefore removes every kernel32 call. We do this in an [AssemblyInitialize] in the test project:

using System;
using System.Reflection;
using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestClass]
public static class LinuxWorkflowEngineWorkaround
{

private const string FlowPerformanceType =
    "Microsoft.Azure.Workflows.Common.PerformanceCounters.FlowPerformance, Microsoft.Azure.Workflows.EventSource.Common";

[AssemblyInitialize]
public static void DisableWindowsOnlyMemoryCounter(TestContext context)
{
    if (OperatingSystem.IsWindows())
    {
        return;
    }

    var type = Type.GetType(FlowPerformanceType, throwOnError: true)!;
    var instance = type.GetProperty("Instance", BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static)?.GetValue(null)
        ?? throw new InvalidOperationException("FlowPerformance.Instance not found; the Linux workaround needs updating.");
    var counter = type.GetField("<MemoryUsageCounter>k__BackingField", BindingFlags.NonPublic | BindingFlags.Instance)
        ?? throw new InvalidOperationException("FlowPerformance.MemoryUsageCounter backing field not found; the Linux workaround needs updating.");

    counter.SetValue(instance, null);
}
  • [AssemblyInitialize] runs once, before any test, so the counter is gone before the engine first checks memory.
  • It does nothing on Windows, where the counter works.
  • On Linux it gets the shared instance through the static FlowPerformance.Instance and nulls the compiler-generated backing field of the get-only MemoryUsageCounter property.
  • It throws if any of those members are missing, so a package update fails loudly instead of quietly bringing the hangs back.

Results in mcr.microsoft.com/dotnet/sdk:10.0 (Ubuntu 24.04, runtime 10.0.14) with the workaround:

  • All 33 tests pass in 1 m 43 s (about the same as on Windows), with no DllNotFoundException and no hangs.
  • Invalid definitions are still caught. With a workflow that adds an invalid action, the affected tests still fail in about 200 ms. So the workaround only removes the memory reading; it doesn't hide validation errors.

This relies on private internals, so it's only a stop-gap. It does suggest a small supported fix: have GetMemoryUsageCounter() return null, or a Linux counter, when not on Windows. CurrentMemoryUtilization already handles null.

Azure Logic Apps
Azure Logic Apps

An Azure service that automates the access and use of data across clouds without writing code.


Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.