How to connect Databricks to Storage account in Azure when I have no access to app registriation

Zhilin Song 20 Reputation points
2024-10-19T06:17:20.4066667+00:00

How to connect Databricks to Storage account in Azure when I have no access to app registriation. Does this mean I can only go to my administrator to fix this?

Azure Storage
Azure Storage

Globally unique resources that provide access to data management services and serve as the parent namespace for the services.

Azure Databricks
Azure Databricks

An Apache Spark-based analytics platform optimized for Azure.

0 comments No comments

2 answers

Sort by: Most helpful
  1. Vinodh247-1375 44,281 Reputation points Volunteer Moderator
    2024-10-19T11:45:10.4766667+00:00

    Hi Zhilin Song,

    Thanks for reaching out to Microsoft Q&A.

    If you do not have access to app registration and cannot create a service principal for authentication, you can still connect Databricks to your Azure Storage account using other methods, depending on your permissions and setup. Here are some alternatives:

    1. Access Keys:
    • If you have access to the storage account access keys, you can use them directly in Databricks to authenticate.
    • You can mount the storage account in Databricks using the access key as given in the example below. This method does not require app registration but does require access to the storage account keys.

      dbutils.fs.mount( source = "wasbs://<container>@<storage_account>.blob.core.windows.net/", mount_point = "/mnt/<mount_name>", extra_configs = {"fs.azure.account.key.<storage_account>.blob.core.windows.net": "<storage_account_access_key>"} )

    1. SAS Keys:
    • If you cannot use access keys, your administrator can generate a SAS for the storage account, granting limited-time access to the specific resources you need. SAS tokens provide more granular control and do not require app reg.

      dbutils.fs.mount( source = "wasbs://<container>@<storage_account>.blob.core.windows.net/?<sas_token>", mount_point = "/mnt/<mount_name>" )

    1. Azure Managed Identity (If available):
    • If your Databricks workspace is running on a cluster with MI enabled, you can authenticate using the MI without needing app registration.
    • In this case, ensure that the MI is assigned the correct permissions to the Storage Account, such as the Storage Blob Data Contributor role.

      dbutils.fs.mount( source = "abfss://<container>@<storage_account>.dfs.core.windows.net/", mount_point = "/mnt/<mount_name>", extra_configs = { "fs.azure.account.auth.type.<storage_account>.dfs.core.windows.net": "ManagedIdentity", "fs.azure.account.oauth2.msi.service.principal.id": "<managed_identity_client_id>" } )

    If none of the above methods are available or appropriate for your environment, you will need to work with your administrator. They can either...

    • Provide access to the necessary app registration or service principal.
    • Set up access keys or SAS tokens for you.
    • Enable and configure MI for your Databricks environment.

    In summary, if you don't have access to app registration, your administrator can help set up access via Managed Identity, Access Keys, or SAS tokens, or directly assign the necessary permissions.

    Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.

    Was this answer helpful?


  2. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.


    Comments have been turned off. Learn more

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.