Compute can't connect to metastore mysql database

Hi @zmsoft
Your Azure Databricks workspace is deployed inside a custom Virtual Network (VNet) that has no public outbound internet access, while your MySQL metastore database is currently hosted with a public endpoint on the internet. Because Databricks compute clusters cannot access the internet, they are unable to connect to your MySQL database.

To resolve this issue securely and reliably, the Microsoft-recommended best practice is to configure Azure Private Link (Private Endpoint) for your MySQL server. This setup enables your Databricks clusters to connect privately and securely over Azure’s backbone network without requiring internet connectivity.

Here are the few steps:

Create a Private Endpoint for your Azure Database for MySQL in the Azure portal and associate it with the same VNet (or a peered VNet) as your Databricks workspace.

Approve the Private Endpoint connection request in the Private Link connections tab of your MySQL server.

Configure DNS for private name resolution by linking the Azure Private DNS Zone (privatelink.mysql.database.azure.com) to your Databricks workspace VNet. This ensures your MySQL hostname resolves to the private IP address.

Update Network Security Group (NSG) rules to allow outbound traffic on port 3306 from your Databricks subnet to the private IP of the MySQL Private Endpoint.

Test the connection from Databricks using a notebook with the MySQL private DNS name, for example:

import mysql.connector
conn = mysql.connector.connect(
    host='yourserver.privatelink.mysql.database.azure.com',
    user='youruser@yourserver',
    password='yourpassword',
    database='yourdatabase'
)
print(conn.is_connected())  # Should return True if the connection is successful

This solution guarantees that your Databricks compute clusters connect to the MySQL metastore over a private, secure network, avoiding public internet exposure and meeting enterprise security requirements.

If your MySQL database is self-hosted or on another cloud, then consider using VNet Peering between the Databricks VNet and the database VNet, assigning a private IP to the MySQL server, and updating NSG rules accordingly.

Kindly refer to the official Microsoft documentation:

Azure Private Endpoint private DNS zone values,

Network security groups.

Create and manage Private Link for Azure Database for MySQL - Flexible Server using the portal.

If this is helpful, please click Accept Answer and kindly upvote it so that other people who faces similar issue may get benefitted from it.

Let me know if you have any further Queries.