Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
@Yaroslav Kazakov - Thanks for the question and using MS Q&A platform.
We are sorry to hear that you're having trouble deleting your Azure Synapse Analytics Apache Spark pool. Based on the error message you provided, it seems that the pool is stuck in “ResourceInUnupdatableState”, which means that it cannot be modified or deleted at this time.
Why this issue occurs?
This can happen when the spark pool was deleted unsuccessfully in the past. Possibly you had tried to delete the spark pool at a certain point in time unsuccessfully (perhaps due to some dependency) which set the status of the pool to a 'Deleted' state.
Recommended solution:
Unfortunately, the only known possibility is to correctly delete the spark pool and re-create it.
How to delete the Apache spark pool correctly?
Check if there are any triggers, pipelines, notebooks attached to the Spark pool that they are trying to delete. You can identify the issue is due to the exiting reference with the Spark pool.
All dependencies with the Spark pool need to be deleted. Also check if there are any Spark job definitions linked to the Spark pool, those need to be deleted before retrying Spark pool deletion.
On Azure Portal, you can use cloud shell (as it already has Azure CLI, bash, and jq installed). After opening a bash cloud shell, run “az login” to authenticate Azure CLI, your use should have permissions on the workspace level.
To list the pool’s references in notebooks in workspace, run the following command:
az synapse notebook list --workspace-name <workspaceName> | jq '.[] | "Notebook: " + .name + ", Pool: " + .properties.bigDataPool.referenceName' | grep <sparkPoolName>
To list the pool’s references in Spark jobs definitions, run the following command:
az synapse spark-job-definition list --workspace-name <workspaceName> | jq '.[] | "Spark Job: " + .name + ", Pool: " + .properties.targetBigDataPool.referenceName' | grep <sparkPoolName>
To mitigate the issue, we suggest removing these references identified and try deletion again. The spark pool’s reference in notebooks should be removed from all branches. You may need to remove the spark pool’s reference in each branch, publish changes, and delete the spark pool again.
If none of these solutions works for you, I would request to share more details about your workspace, region and details about the Apache spark pool created.
Hope this helps. Do let us know if you any further queries.
If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.