azure event hub using kafka , ingress and egress performance issues

Hi Ruchi Manuja & thx for join me at Q&A platform,

what u're describing could indicate resource contention in Azure Event Hubs, but I'd first distinguish between Event Hubs being overloaded and your Kafka consumers being unable to keep up with incoming traffic.

With 6 Processing Units, you're using Event Hubs Premium. Unlike Standard tier, Premium doesn't have a fixed ingress/egress throughput allowance per PU. Actual performance depends on CPU, memory, partition distribution, message size, and producer/consumer behavior.

I'd start with Azure Portal > Event Hubs Namespace > Monitoring > Metrics.

Check NamespaceCpuUsage and NamespaceMemoryUsage alongside IncomingBytes, OutgoingBytes, IncomingMessages, OutgoingMessages, and ThrottledRequests. Use the same time window and preferably one-minute granularity.

If CPU or memory usage rises significantly whenever ingress increases and egress drops, that's a strong indication of namespace resource pressure. However, with Kafka there's another important detail. Throttling doesn't always appear as an obvious error or an increase in the Azure ThrottledRequests metric. Kafka clients can receive delayed responses, so check throttle_time_ms in the producer and consumer responses as well.

I'd also examine consumer lag, fetch latency, and partition distribution. For Premium namespaces, Event Hubs application metrics logs can provide ConsumerLag information per partition and consumer group. If lag increases during high ingress, you know consumers are falling behind, but you still need to establish whether the bottleneck is in Event Hubs or the consuming application.

One thing I'd specifically check is whether messages are evenly distributed across partitions. A hot partition can limit throughput even when the namespace has available capacity. Similarly, adding more Kafka consumer instances won't increase parallelism beyond the number of partitions assigned to that consumer group.

A useful controlled test would be to temporarily increase the PUs during a high-ingress period while keeping producer and consumer configurations unchanged. If egress improves significantly and consumer lag starts decreasing, that would support the resource-contention hypothesis. If nothing changes, I'd focus more on partition skew, Kafka fetch settings, and consumer processing performance.

I wouldn't increase PUs permanently until you've identified which resource is actually saturated. If you can share the approximate peak ingress and egress in MB/s, the number of partitions, consumer instances, and CPU/memory utilization during the slowdown, we should be able to narrow this down considerably.

rgds,

Alex

p.s. if my answer is help - pls mark it as an answer