zero data retention, security and data privacy in azure open ai

Azure OpenAI models are stateless and do not store prompts or completions inside the model, but true zero data retention (ZDR) is not automatic for the entire service because optional stateful features, abuse monitoring, and your own application logging can persist data.

What Microsoft does—and does not do—with your data

For Azure OpenAI and other Models sold by Azure in Microsoft Foundry:

• Prompts, completions, embeddings, and training data aren't available to other customers.
• They aren't available to OpenAI or other model providers.
• They aren't used to train foundation models without your permission or instruction.
• They aren't used to improve Microsoft or third-party products without your explicit permission or instruction.
• Fine-tuned models are exclusive to your organization.
• The models run in Microsoft's Azure environment and don't call OpenAI services such as ChatGPT or the OpenAI API. ^1^

During ordinary inferencing, the model is stateless: prompts and completions are processed to generate a response but aren't stored in the model. This does not mean no other service component can store them.

Where data can still be retained

1. Stateful features The Responses API, Assistants Threads, Stored completions, Files/vector stores, batch processing, and fine-tuning can create or use persistent data stores. The Responses API specifically stores message history for multi-turn conversations and workflows. Customer-controlled stored data can be deleted, is kept within the resource's geography, and is encrypted at rest. Preview features may have different guarantees. ^1^
2. Abuse monitoring By default, prompts or completions flagged as harmful or part of a potentially abusive pattern may be selected for review. Automated review does not store that content in the abuse-monitoring system. When human review is necessary, only authorized Microsoft employees can access flagged data through Secure Access Workstations with Just-In-Time managerial approval. ^2^
3. Application-side storage API gateways, application telemetry, exception traces, chat databases, caches, vector stores, and Azure Monitor diagnostics are separate from model retention. Ensure these systems don't record prompt or response bodies if ZDR is required.

Closest supported configuration to ZDR

1. Use stateless inference APIs and avoid Responses API persistence, Assistants Threads, Stored completions, Files/vector stores, batch uploads, and fine-tuning unless their storage is explicitly required.
2. Apply for modified abuse monitoring if the workload handles highly sensitive data and your organization meets Limited Access eligibility criteria. After approval, Microsoft doesn't perform the abuse-monitoring storage and human-review process, although real-time automated review can still occur. ^2^
3. Verify the approved setting:
   • Azure portal → Azure OpenAI/Foundry resource → Overview → JSON view.
   • Confirm that ContentLogging appears under capabilities with value false.
   • If the property is absent, storage for abuse monitoring has not been turned off.
   Or use Azure CLI:

      az cognitiveservices account show \
        --name <resource-name> \
        --resource-group <resource-group>
   

   Look for:

      {
        "name": "ContentLogging",
        "value": "false"
      }
   

   Microsoft documents that this property appears only when abuse-monitoring data storage is disabled. ^1^
4. Select the deployment type for residency requirements:
   • Standard deployments process within the customer-specified geography, though processing can occur across regions in that geography.
   • DataZone deployments can process within any geography in the selected zone.
   • Global deployments can process in any geography where the model is deployed.
   • Stored data remains in the customer-designated geography for Global and DataZone deployments. ^1^

Additional security controls

• Use a private endpoint, disable public network access, and restrict traffic to approved virtual networks.
• Use Microsoft Entra managed identities rather than API keys.
• Apply least-privilege Azure RBAC.
• Put Azure API Management in front of the endpoint for schema validation and rate limiting.
• Use input/output filtering and DLP to block or redact secrets, PII, credentials, and regulated data.
• Configure diagnostics carefully: retain operational metadata, but don't log request or response bodies where prohibited. Microsoft recommends private endpoints, managed identity, layered filtering, and controlled diagnostic monitoring. ^3^

Any data persisted by supported features is encrypted at rest with AES-256 by default. Customer-managed keys provide control over key creation, rotation, disabling, revocation, and auditing. ^4^

Practical interpretation

If “ZDR” means no prompts or responses stored by the model or used for training, Azure OpenAI provides that for ordinary stateless inference. If it means no persistence anywhere in Microsoft’s service or your architecture, use stateless APIs, obtain modified abuse-monitoring approval, verify ContentLogging=false, avoid storage-backed features, and remove prompt bodies from application telemetry.

Preview models and features can have different privacy practices, so use GA components for regulated production workloads and validate the complete data path—not only the model endpoint.

References

1. Data, privacy, and security for Models sold by Azure in Microsoft Foundry
2. Abuse monitoring
3. Azure AI security best practices
4. Azure OpenAI in Azure AI Foundry Models encryption of data at rest