Azure Data Factory Copy activities remain Queued for several hours intermittently

The behavior matches a pipeline run waiting on orchestration or Integration Runtime capacity rather than a Copy failure in the source or sink. In Azure Data Factory, long Queued time can happen because of concurrency limits, transient service or network issues, or IR resource contention. The fact that the affected Copy activities vary between runs and can stay queued for more than five hours also fits an intermittent platform or orchestration issue.

Recommended checks:

1. In ADF Monitor, look for older pipeline runs still marked In Progress.
   • Even if the pipeline JSON does not explicitly define concurrency, queued pipeline runs can still be blocked by runs already in progress.
   • If old runs are still active within the last 45 days, cancel the stuck ones and rerun.
2. Check the pipeline’s concurrency property from the authoring canvas.
   • Select the pipeline and verify whether a concurrency value is assigned.
   • If it is assigned, confirm in Monitoring that no previous run is still consuming that slot.
3. Review the Copy activity execution details in monitoring.
   • In the copy activity monitoring view, check whether the longest duration is in Queue.
   • Queue means elapsed time before the copy activity actually starts on the Integration Runtime.
   • This helps confirm the issue is startup/resource allocation, not transfer or sink write.
4. Validate IR capacity and region-related behavior.
   • A queued Copy activity commonly means the activity is waiting for available compute resources from the Integration Runtime.
   • If Azure IR or Managed virtual network IR is involved anywhere in the path, queue time can be longer by design, especially on Managed virtual network IR because of warm-up.
   • If using Self-hosted IR, long queue time can indicate the IR is busy or needs scale-out/up.
5. Consider transient service issues.
   • Azure Data Factory has an internal recovery process that monitors runs and restarts them when transient issues such as network failures, credential failures, or service outages are detected.
   • That recovery process runs about every hour.
   • If a run remains stuck for more than one hour, create a support case.
6. If many activities run in parallel, reduce contention.
   • Too many parallel activities can cause queuing.
   • If applicable, split large workloads into smaller datasets and run them in controlled parallel batches.
7. For stuck activity runs, cancel and retry.
   • For activities running much longer than normal with little or no progress, canceling and retrying is a documented first step.
   • Since the affected activities change between reruns, this further suggests an intermittent queueing/orchestration problem rather than a deterministic issue in a specific Copy definition.

Based on the details provided:

• The blank Integration Runtime field on queued runs is consistent with the activity not yet being assigned to runtime execution.
• The self-hosted IR showing Running, queue length 0, and available concurrency at inspection time suggests the bottleneck may not be the node itself at that moment.
• Because the issue is intermittent and affects different Copy activities on different reruns, transient service or orchestration delays are a strong candidate.

Most likely next actions:

1. Cancel any stale in-progress runs in Monitor.
2. Recheck pipeline concurrency in the authoring canvas.
3. Open the affected Copy run details and confirm the delay is specifically in Queue.
4. If the run remains queued for more than one hour again, create a support case.

References:

• Troubleshoot pipeline orchestration and triggers in Azure Data Factory
• Troubleshoot copy activity performance
• Troubleshoot Azure Data Factory and Synapse pipelines
• ADF pipeline stuck at "Queued" status around 5 hours. - Microsoft Q&A