An Azure service for ingesting, preparing, and transforming data at scale.
Securing args of the Python script in Azure Data Factory Custom Azure Batch Activity
Hi,
Imagine running a simple ADF Pipeline that consists of 2 steps:
- Fetching a sensitive value - like Storage Account URL - from Azure Key Vault
- Executing a Python script with argsparse where this value is set as an argument with the command
@{activity('SAURL').output.value
Both Web Activity and Custom Activity have secure input and output values.
THE PROBLEM:
Even though it seems fine from ADF perdpective, when I move to Batch Account -> jobs -> tasks, select Task and move to 'Properties' section, I am able to see the full command line command with the value of my SAURL.
QUESTIONS:
- How bad is it from security perspective?
- What is the proper workaround? My current solution is to move the sensitive data to the code and call them with azure.keyvault. secrets library, but it forces me to hard-code the name of the Key-Vault which is also bad. Also it would require adjusting all scripts (from the argsparse versions to the ones calling AKV).
- Therefore - is there any option to hash the command line command in the task (from the Batch or ADF perspective)?
Azure Data Factory
1 answer
Sort by: Oldest
-
Deleted
This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
1 deleted comment
Comments have been turned off. Learn more