Request to Disable Compliance Security Profile for Databricks Dev Workspace

The Compliance Security Profile cannot be manually disabled through the Azure portal, Databricks workspace settings, CLI, REST API, Terraform, or an ARM template after it has been enabled.

The supported outcome depends on the workspace history:

• If the workspace has processed regulated data, the Compliance Security Profile cannot be removed. The workspace must be replaced with a new workspace created without CSP.
• If the workspace has never processed regulated data, the profile can potentially be removed, but there is no self-service procedure for doing so. [learn.microsoft.com]

Before replacing the development workspace, verify whether CSP itself is causing the reported Genie limitation. In CSP-enabled workspaces, Partner-powered AI features are disabled by default, which also disables some Databricks AI capabilities, including Genie Code. A workspace administrator can review this under the workspace administration settings and enable Partner-powered AI features if permitted by the organisation’s security and compliance policy. [learn.microsoft.com]

CSP does not universally disable serverless compute. For example, the current requirements explicitly support serverless environments, including additional requirements for compliance standards that require FIPS encryption. Therefore, validate the specific serverless workload, workspace region, Unity Catalog configuration, entitlement, and network connectivity before concluding that CSP is the blocker. [learn.microsoft.com]

If CSP must be removed and a self-managed resolution is required, the practical approach is:

1. Create a new development workspace without CSP.
2. Attach it to the appropriate Unity Catalog metastore.
3. Recreate workspace-scoped objects such as compute policies, secret scopes, SQL dwh, jobs, permissions and workspace settings.
4. Deploy notebooks and code through databricks git folders or the existing CI/CD process.
5. Test Genie and the required serverless workload before decommissioning the original workspace.

Do not attempt to disable CSP by changing the ARM or Terraform property to Disabled on the existing workspace. Although deployment schemas expose that value, the documented CSP lifecycle rules take precedence once the profile has been enabled. [learn.microsoft.com]

Reference: [Configure enhanced security and compliance settings]

Help make this community better for everyone: if this answer resolved your issue, please accept it or leave an upvote. If not, share more details in a comment so we can continue the discussion and find the right solution