Azure Databricks managed VNet: allow classic compute to a firewall-enabled ADLS Gen2 account in the same region (service endpoint on workers-vnet blocked by deny assignment)

Jerrel den Harder 25 Reputation points
2026-11-05T06:04:04.0166667+00:00

Hello,

We run an Azure Databricks workspace in West Europe that uses the Databricks-managed network (no VNet injection). Classic job compute (DBR 17.3 LTS, Photon, standard access mode) needs to write to an ADLS Gen2 storage account in the same region through a Unity Catalog volume. The storage account firewall has default action Deny, bypass None, and a resource instance rule for the Unity Catalog access connector.

Problem:

Writes fail with HTTP 403, error code AuthorizationFailure ("This request is not authorized to perform this operation"), for example from dbutils.fs.put to a UC volume path. IP rules do not help because the storage account and the workspace are in the same region, and the managed workers-vnet egresses through a NAT gateway.

What we tried:

Adding the Microsoft.Storage service endpoint to public-subnet and private-subnet of workers-vnet (in the managed resource group) so we can add a virtual network rule on the storage account. Saving the subnet fails with: the client has permission for Microsoft.Network/virtualNetworks/subnets/write, but does not have permission to perform 'Microsoft.Network/natGateways/join/action' on the nat-gateway because it is blocked by deny assignments.

Constraint: we do not have a paid Azure support plan, so we cannot open a technical support ticket.

Questions:

  1. Is adding a Microsoft.Storage service endpoint to the managed VNet subnets a supported configuration? If so, how can the deny assignment be handled without Azure Support?
  2. Which approach would you recommend instead: converting the workspace to VNet injection (preview), a private endpoint from workers-vnet to the storage account, or something else? Any pitfalls for each?
  3. Does a resource instance rule for the access connector ever cover classic compute that uses a Unity Catalog vended SAS, or does the compute network always have to be allowed on the storage firewall separately?

Thank you!

Azure Databricks
Azure Databricks

An Apache Spark-based analytics platform optimized for Azure.

0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.