Azure IoT Operation Back-Pressure

A subscriberQueueLimit.length of 1000 does not by itself reject the 1,001st publish: if the queue strategy is None—the default—the queue continues growing until the session expires, which matches the observed delivery of all 1,300 messages after reconnection.^1^

Queue limit versus back-pressure

These are related but distinct mechanisms:

• Subscriber queue limit controls one subscriber’s outgoing QoS 1 queue.
  • length: threshold at which the configured overflow strategy applies.
  • strategy: None (default): do not drop messages; the queue can grow until session expiry.
  • strategy: DropOldest: when full, discard the oldest queued message and admit the new one.
• Memory back-pressure is broker resource protection. With disk buffering disabled, queues remain in memory. Back-pressure starts as the relevant internal buffer pool approaches capacity; it is not guaranteed to begin at exactly message 1,001 for a queue configured with length 1,000.^2^

Consequently, the successful 1,300-message test indicates that either:

1. the effective strategy was None; or
2. in a multi-partition deployment, messages were distributed across independently limited queues.

Each partition maintains its own subscriber queue, so the effective cluster-wide count for one subscriber can exceed the configured per-partition length.^1^

Configure a bounded queue test

To test overflow behavior rather than broker-wide memory back-pressure, configure both length and strategy:

{
  "advanced": {
    "clients": {
      "subscriberQueueLimit": {
        "length": 1000,
        "strategy": "DropOldest"
      }
    }
  }
}

Deploy using the broker configuration file:

az iot ops create ... --broker-config-file broker.json

Advanced client options are initial-deployment settings. Changing them requires redeploying the Azure IoT Operations instance.^1^

Repeat the test with:

1. A persistent subscriber subscribed at QoS 1.
2. The subscriber disconnected without allowing its session to expire.
3. More than 1,000 uniquely numbered QoS 1 messages published.
4. The same client ID and persistent session resumed.

With DropOldest, the broker should retain newer queued messages by dropping older ones as the queue fills. Offline persistent sessions queue QoS 1 messages; QoS 0 messages are dropped while the subscriber is offline.^3^

Also account for these controls:

• maxSessionExpirySeconds: clears the session and its entire queue when the session expires.
• maxMessageExpirySeconds: bounds how long messages remain eligible for delivery.
• Client Receive Maximum: controls the separate in-flight queue, not the offline/outgoing subscriber queue.^1^

What the publisher receives

Azure IoT Operations does not document an application-level notification saying that a particular subscriber queue is full. The publisher and subscriber are decoupled: successful MQTT publication acknowledgment does not mean every subscriber retained or processed the message.

Under actual broker memory or disk back-pressure, packets can be rejected or flow can be constrained. For MQTT v5 QoS 1 clients, normal MQTT acknowledgment and flow-control behavior applies, but the documentation does not specify a queue-specific publisher reason code that identifies which subscriber was full.^3^

Monitor the broker directly instead:

• aio_broker_buffer_pool_used_percent — pool utilization; back-pressure begins at about 75%.
• aio_broker_backpressure_packets_rejected_memory — packets rejected by memory back-pressure.
• aio_broker_backpressure_packets_rejected_disk — packets rejected by disk back-pressure.
• aio_broker_publishes_received and aio_broker_publishes_sent — compare ingress with deliveries.^4^

Disk-backed buffering changes where queued data is held, not the publisher-feedback model. It spills eligible subscriber queues to ephemeral disk when memory pressure rises; it is not durable broker persistence, and buffered data can be lost when the pod exits.^2^

References

1. Deployment planning - Advanced MQTT options
2. Deployment planning - Disk-backed message buffer
3. MQTT feature support in MQTT broker - Azure IoT Operations
4. Metrics for MQTT broker - Azure IoT Operations | Microsoft Learn