How can I reduce API latency in an Azure Functions application?

Question

How can I reduce API latency in an Azure Functions application?

Raza987 20

I’m working with an Azure Functions application that exposes several HTTP APIs and some endpoints are taking longer than expected to respond. The APIs perform database queries and call a few external services, so the latency varies depending on the request.

I have already reviewed the application code and database queries, but I’m still seeing occasional slow responses, especially during periods of higher traffic. I’d like to understand which areas I should investigate first, such as cold starts, connection management, database performance, function hosting plans or external API calls.

What are the most effective ways to identify the main source of latency and reduce API response times in an Azure Functions application?

0 comments

Answer accepted by question author

Marcin Policht 108.3K MVP Volunteer Moderator

First isolate whether the slowdown is caused by Azure Functions infrastructure or by downstream dependencies. Since your APIs perform database queries and call external services, Application Insights should be the starting point before changing the hosting plan or application architecture.

Enable Application Insights for the Function App and examine the slow requests, particularly P95 and P99 latency rather than just average response time. Open an individual slow request and inspect its end-to-end transaction details. The transaction view can show how much time was spent executing the function and how much was spent waiting for SQL/database operations or outbound HTTP calls. Application Insights automatically tracks many SQL and HTTP dependencies, so a dependency with several seconds of duration is a strong indication that the function is waiting on a downstream service rather than spending that time executing its own code.

You can also query the requests and dependencies telemetry to correlate slow requests with specific functions, dependencies, instances, and time periods. This gives you a much better starting point than making changes based only on the fact that the API is slow.

Because you see increased latency during higher traffic, determine whether the slow requests coincide with new Function App instances being started. Cold starts can add significant latency when Azure has to initialize a new instance during scale-out.

Compare the total HTTP request duration with the actual function execution duration and look for latency concentrated on the first requests handled by newly created instances. Also correlate slow requests with instance counts and scale events.

If cold starts are confirmed, consider the hosting options available to your workload. Flex Consumption and Premium provide mechanisms for keeping instances ready, which can substantially reduce cold-start latency. Also examine application startup code. Avoid performing expensive initialization, loading large amounts of data, or establishing unnecessary connections during every invocation. Reuse initialized clients and services where the runtime model permits it.

Next, check connection management and SNAT exhaustion. High traffic combined with database and external HTTP calls makes connection management important. Creating new outbound connections for every invocation can increase latency and can eventually contribute to SNAT port exhaustion.

For .NET applications, avoid repeatedly creating HttpClient instances inside the function method. Use dependency injection and reuse HTTP clients. Database clients should similarly use the appropriate connection pooling mechanisms rather than establishing a completely new physical connection for every request. For services such as Cosmos DB, reuse the client instance rather than creating one per invocation.

In the Azure portal, use Function App > Diagnose and solve problems and investigate outbound connection and SNAT-related diagnostics. SNAT exhaustion can manifest as intermittent delays, connection failures, or timeouts that become much more noticeable under high concurrency.

Another consideration is whether the hosting plan is becoming a bottleneck. Check Function App metrics during the exact periods when response times increase. Look at CPU, memory, instance count, execution count, execution duration, and scaling activity. If CPU or memory utilization becomes constrained during traffic spikes, the function instances may not have sufficient resources to process the workload efficiently.

For Flex Consumption, increasing the configured memory allocation also provides more CPU resources. For Premium or Dedicated/App Service plans, scaling to a larger worker size or adding instances may help if the telemetry shows that compute capacity is the limiting factor. Do not increase capacity simply because requests are slow, though. First establish that CPU, memory, concurrency, or scaling is actually contributing to the latency.

You should also investigate database performance under load. Application Insights dependency telemetry can tell you whether database calls are responsible for slow requests. If they are, investigate the database independently during the same time periods.

For Azure SQL, examine query duration, CPU/resource utilization, blocking, waits, connection counts, and indexing. A query that performs well under light load but becomes slow during traffic spikes can indicate database resource contention or locking rather than a fundamentally inefficient query.

For Cosmos DB, check RU consumption and throttling. HTTP 429 responses or increased request latency during periods of high RU consumption can indicate that the database is becoming the bottleneck.

In addition, try measuring independently external API calls - especially if Application Insights shows that the Function spends most of its time waiting for an external API. In such cases, the function itself may not be the source of the problem. Measure each external call separately and examine response time, failures, retries, and timeouts.

For independent external calls, consider executing them concurrently rather than sequentially where the application logic allows it. For frequently requested data that does not change frequently, caching can also eliminate unnecessary calls. Azure Cache for Redis is one option.

For operations that do not need to complete before the HTTP response is returned, consider an asynchronous architecture rather than keeping the HTTP request open while waiting for a long-running operation. Durable Functions or a queue-based pattern can be appropriate depending on the workload. For external services that are unreliable or slow, use sensible timeouts and controlled retry policies rather than allowing a request to remain blocked indefinitely.

Finally check the networking path. If the Function App uses VNet integration, private endpoints, NAT Gateway, API Management, Application Gateway, Front Door, or other intermediary components, include those components in the investigation. DNS resolution, routing, TLS connection establishment, and outbound connection limits can all contribute to latency.

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin

0 comments

1 additional answer

Your answer

Answer 1

Start by separating latency into four likely sources: cold start, function runtime behavior, outbound connection behavior, and downstream dependencies.

Measure first with Application Insights

Enable Application Insights for the function app and use it to collect request logs, performance metrics, exceptions, and dependency telemetry.
Check request duration, dependency duration, and exceptions to see whether the delay is inside the function code, in database calls, or in external service calls.
Set alerts for performance degradation so spikes during higher traffic are visible quickly.

Check for cold starts and hosting-plan effects

Cold start is a common cause of slow first requests and intermittent latency under bursty traffic.
To reduce cold starts, prefer Flex Consumption or Premium plans.
On Flex Consumption, use always ready instances.
On Premium, use prewarmed or Always-Ready instances and consider a warmup trigger.
If latency is most visible after idle periods or when new instances are added, cold start is a strong candidate.

Review scaling and deployment configuration

Enable autoscaling and verify scaling behavior under load.
On Consumption, review whether Singleton patterns, trigger concurrency settings, or functionAppScaleLimit are artificially limiting scale-out.
Use deployment slots or rolling updates to reduce deployment-related latency and downtime.
Run from package to reduce file copy locking issues and potentially reduce cold-start time, especially for JavaScript apps with large npm dependency trees.

Reduce startup and execution overhead in the function code

Avoid long-running functions. Refactor large functions into smaller functions when possible.
For HTTP-triggered APIs, return quickly and defer longer work to a queue-triggered function when the scenario allows it.
Make sure background tasks complete before the function returns. Untracked background work can be interrupted and create inconsistent behavior.
Minimize heavy global variables because they increase cold-start time.
Reuse database connections and HTTP clients through connection pooling.
Clean up resources such as connections and timers.

Optimize Node.js-specific behavior if the app uses Node.js

Keep the event loop free. Heavy synchronous loops can block other invocations and increase latency.
Break large workloads into batches and yield with setImmediate.
Measure event loop delay to detect blocking.
Bundle dependencies to reduce the number of files loaded during cold start.
Use lazy loading or dynamic imports for heavy modules that are not needed on every request.
Use efficient data structures such as Map for frequent lookups.
Use FUNCTIONS_WORKER_PROCESS_COUNT carefully. It can improve throughput for CPU-bound or some I/O-bound apps, but too many workers can increase load times and context switching.

Investigate outbound connection limits and connection reuse

Slowdowns under concurrency can come from connection pool exhaustion or SNAT port limits when opening many outbound connections.
Reuse connections, use keep-alives, and use connection pooling for repeated calls to the same destination.
Scale out to more instances if outbound connection pressure is high.

Validate downstream dependency performance

Since the APIs call databases and external services, inspect dependency timing in telemetry.
If dependency duration dominates request duration, the main latency source is downstream, not the Functions host.
Apply retry patterns for transient failures when accessing cloud resources, but avoid overly aggressive retry behavior because it can increase latency and outbound connection pressure.

A practical investigation order is:

Enable and review Application Insights request and dependency telemetry.
Compare slow requests after idle periods versus during sustained traffic to confirm cold start impact.
Check whether outbound connections are being reused.
Review scaling limits and hosting plan choice.
Reduce startup cost and long-running work in the function code.
If using Node.js, inspect event loop blocking and dependency loading.

For the biggest latency reductions, the highest-value changes are usually: Application Insights-based dependency tracing, moving to Flex Consumption or Premium for cold-start-sensitive APIs, connection pooling and HTTP client reuse, running from package, and refactoring long-running HTTP work into asynchronous queue-based processing where possible.