Guidance on AI Foundry resource deployment

Question

Guidance on AI Foundry resource deployment

Bijaya Rai 60

Our team has multiple applications using a handful of models deployed in AI Foundry. The foundry and the models are deployed in SwedenCentral. Other teams in the company use our product, and we now have interest from our clients.

Questions:

Any guidance on how foundry resources should be deployed? 1 per client or 1 for all?
1. 1 for all: We do not want 100 clients to be blocked when one hits the TPM limit since the quota is shared across a subscription.
2. 1 per client: Not all users are equal, so we want the option of limiting how much TPM and RPM a team gets. E.g. we might want a higher quota for dev and sales teams but not for others.
We do not have any visibility into which team or client is using AI Foundry and how much it's costing us to host the foundry resource per group. Is there anything in the product to support this use case?

Sairam Reddy Rajamuri 0 Reputation points Microsoft External Staff Moderator

2026-11-05T15:22:16.0633333+00:00

Hello Bijaya,

Thank you for sharing the scenario.

Microsoft Foundry does not require one resource per client. A shared resource is suitable when clients have similar security and operational requirements, but consumers using the same deployment share its rate limits. Separate deployments can be assigned different TPM capacity, while per-client limits on a shared deployment should be enforced through the application or an API management layer.

Please note that separate Foundry resources do not automatically provide separate quota pools, because quota scope depends on the subscription, model, region or data zone, and deployment type.

For your scenario, we recommend a hybrid design: shared infrastructure where practical, separate deployments for capacity control, and separate resources or subscriptions only where stronger security, networking, operational, or cost isolation is required.

Foundry and Azure Cost Management can provide resource and project-level cost visibility. If multiple clients share an endpoint, application telemetry is recommended for accurate client-level usage attribution.

Could you confirm the models and deployment types in use, whether client-level security isolation is required, and whether you need showback, chargeback, or invoice-level reporting?

Kind regards,

Sairam

1 answer

Your answer

Sairam Reddy Rajamuri 0 Reputation points Microsoft External Staff Moderator

2026-11-05T15:22:16.0633333+00:00

Hello Bijaya,

Thank you for sharing the scenario.

Microsoft Foundry does not require one resource per client. A shared resource is suitable when clients have similar security and operational requirements, but consumers using the same deployment share its rate limits. Separate deployments can be assigned different TPM capacity, while per-client limits on a shared deployment should be enforced through the application or an API management layer.

Please note that separate Foundry resources do not automatically provide separate quota pools, because quota scope depends on the subscription, model, region or data zone, and deployment type.

For your scenario, we recommend a hybrid design: shared infrastructure where practical, separate deployments for capacity control, and separate resources or subscriptions only where stronger security, networking, operational, or cost isolation is required.

Foundry and Azure Cost Management can provide resource and project-level cost visibility. If multiple clients share an endpoint, application telemetry is recommended for accurate client-level usage attribution.

Could you confirm the models and deployment types in use, whether client-level security isolation is required, and whether you need showback, chargeback, or invoice-level reporting?

Kind regards,

Sairam

Answer 1

Hi Sairam,

We are using gpt-5.6-terra, gpt-5.6-terra, gpt-5.1, gpt-5.2, and gpt-4.1, with the 'GlobalStandard' deployment type.

"A shared resource is suitable when clients have similar security and operational requirements."

This would have been ideal because most have the same requirements, but since we can't separate usage, we won't have a way to bill our clients. I noticed there is an option to create multiple projects; as I understood the documentation, Foundry projects mostly organise work, not isolate it. I also noticed that all projects in a Foundry resource use the same API key, so I assume Foundry captures cost data per resource and not by project.

If we were to build telemetry within our app, is there any API that can tell us

model cost- input, cached and output tokens and other if any
track live cost from Foundry

As for the topic of client-level isolation, we will most likely need one. Our clients expect the service to be available for their use, and a 429 error just because other clients hit the TPM or RPM limit is not acceptable.

An ideal solution for us:

Each client gets their own dedicated service; we can use the cost data from azure's portal to manage the usage. The models can be deployed either geographically or in a specific region depending on the client's request.

Thanks,

Bijaya