User-Level Token Usage in Azure AI Foundry

Hi @Sree Aravind M ,

Thank you for reaching out to the Azure AI community. I can certainly clarify how token tracking works in Azure AI Foundry and guide you on the best practices to achieve user-level monitoring.

Is User-Level Token Tracking Natively Supported?

Out of the box, Azure AI Foundry's built-in monitoring dashboards and Azure Monitor aggregate token usage at the project, deployment, or model level. For privacy, security, and architectural reasons, Azure AI Foundry does not natively track or map token consumption to individual user identities (such as Microsoft Entra UPN or Object IDs).

However, you can easily achieve this tracking by implementing one of two industry-standard architectural patterns.

Option 1: The AI Gateway Pattern via Azure API Management (APIM)

This is the enterprise-standard pattern. By placing Azure API Management in front of your Azure AI Foundry / Azure OpenAI endpoints, you can intercept requests, identify the user, log token metrics, and even enforce rate limits.

Step 1: Identify the User

You can identify the user making the request using APIM in two ways:

1. Azure Active Directory (Entra ID) JWT Validation: Use the <validate-jwt> policy to extract user claims (such as upn or oid) from the bearer token.
2. Subscription Keys: Assign unique APIM subscription keys to individual users or user groups.

Step 2: Log Token Metrics with APIM Policies

Use the specialized <llm-emit-token-metric> policy in APIM to parse the request and response payloads, extract token usage, and send it to Azure Application Insights with the user's identity attached as a custom dimension.

Add the following policy to your API's outbound pipeline:

<policies>
    <inbound>
        <base />
        <!-- Extract user identity from JWT and store in a variable -->
        <set-variable name="userId" value="@(context.Request.Headers.GetValueOrDefault("Authorization", "").Split(' ').Last().AsJwt()?.Claims.GetValueOrDefault("upn", "anonymous"))" />
    </inbound>
    <outbound>
        <base />
        <!-- Log the token usage metrics to Application Insights with User ID -->
        <llm-emit-token-metric namespace="AzureOpenAI">
            <dimension name="User" value="@((string)context.Variables["userId"])" />
        </llm-emit-token-metric>
    </outbound>
</policies>

Step 3: Enforce Token Quotas per User (Optional)

If you want to actively control costs, you can also apply the <azure-openai-token-limit> policy to rate-limit users based on their token usage:

<azure-openai-token-limit counter-key="@((string)context.Variables["userId"])" tokens-per-minute="5000" estimate-prompt-tokens="false" />

Option 2: Application-Level Telemetry (OpenTelemetry & Application Insights)

If you are building an application (like a custom chatbot or web portal) where users authenticate directly into your app rather than calling the API directly, you can log token usage programmatically.

Step 1: Extract Token Usage from SDK Response

When calling Azure AI Foundry models via the SDK, the model response contains a usage payload. You can programmatically read this block:

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Extract token metrics
prompt_tokens = response.usage.prompt_tokens
completion_tokens = response.usage.completion_tokens
user_id = "******@domain.com" # Obtained from your app's authentication context

Step 2: Push Metrics to Azure Application Insights

Log this data using OpenTelemetry or the Application Insights SDK as custom metrics:

from opencensus.ext.azure.log_exporter import AzureLogHandler
import logging

logger = logging.getLogger(__name__)
logger.addHandler(AzureLogHandler(connection_string='InstrumentationKey=your-key-here'))

properties = {
    'custom_dimensions': {
        'UserId': user_id,
        'PromptTokens': prompt_tokens,
        'CompletionTokens': completion_tokens,
        'TotalTokens': prompt_tokens + completion_tokens
    }
}
logger.info("Token usage tracked", extra=properties)

Step 3: Query and Visualize

In the Azure Portal, navigate to your Log Analytics Workspace or Application Insights, and use Kusto Query Language (KQL) to build charts:

traces
| extend UserId = tostring(customDimensions.UserId)
| extend PromptTokens = toint(customDimensions.PromptTokens)
| extend CompletionTokens = toint(customDimensions.CompletionTokens)
| summarize InputTokens = sum(PromptTokens), OutputTokens = sum(CompletionTokens) by UserId, bin(timestamp, 1d)

You can import this query directly into Azure Workbooks or Power BI for automated cost reporting.

Next Steps

1. Review your application architecture to decide if the API Gateway Pattern (APIM) or Application-Level Logging works best for your team.
2. Note that preview features, API structures, and policy specifications can change, so we recommend verifying current limit policies in the latest Azure API Management Policy Reference for AI Gateway.

If you run into any issues during the setup or need help configuring the KQL queries, please let me know, and we can investigate further.

Thanks,
Suhail Ahamed J