Issues with my Terraform Service Principal not able to read watchlists

John Line 0 Reputation points
2026-11-20T03:33:53.84+00:00

Hi,

I have noticed this issue starting today, this was working a couple of days ago. I deploy Terraform via a Service Principal and have started getting 401 errors at the refresh stage then the stack tries to read the existing watchlists.

The Service Principal has Owner on the subscription, so this should not be an actual permissions issue and nothing has changed with the watchlists.

The only possible change is the preview released for ISOC, but I have not yet activated this or activated the URBAC Sentinel workload.

│ Error: retrieving Watchlist Item (Subscription: "[redacted]"

│ Resource Group Name: "[redacted]"

│ Workspace Name: "[redacted]"

│ Watchlist Alias: "BreakGlassAccounts"

│ Watchlist Item: "ce76ad5b-d628-48e7-aca3-5d1d3eb0886c"): unexpected status 401 (401 Unauthorized) with response: You do not have permission to view this directory or page.

│

│ with module.[redacted],

│ on .terraform/modules/[redacted]/watchlist.tf line 13, in resource "azurerm_sentinel_watchlist_item" "break_glass_accounts":

│ 13: resource "azurerm_sentinel_watchlist_item" "break_glass_accounts" {

│

│ retrieving Watchlist Item (Subscription:

│ "a2b78e69-29f0-4e8e-bb27-e4c74613b63e"

│ Resource Group Name: "[redacted]"

│ Workspace Name: "[redacted]"

│ Watchlist Alias: "BreakGlassAccounts"

│ Watchlist Item: "ce76ad5b-d628-48e7-aca3-5d1d3eb0886c"): unexpected status

│ 401 (401 Unauthorized) with response: You do not have permission to view

│ this directory or page.

Microsoft Security | Microsoft Sentinel
0 comments No comments

1 answer

Sort by: Most helpful
  1. AI answer

    2026-11-20T03:35:46.2033333+00:00

    Because subscription-level Owner already covers broad Azure resource access, first reproduce the watchlist-item GET with the same service principal and compare it with the AzureRM provider version from the last successful run; do not add more roles until that test identifies an authorization gap.

    1. Check for an AzureRM provider change

    A refresh can start failing without a Terraform configuration change if .terraform.lock.hcl changed, was regenerated, or was not preserved between CI runs.

    terraform version
    terraform providers
    git diff -- .terraform.lock.hcl
    git log -p -- .terraform.lock.hcl
    

    Compare the current hashicorp/azurerm version with the last successful pipeline. The azurerm_sentinel_watchlist_item resource remains documented by HashiCorp, so the resource has not generally been removed ^1^.

    If the version changed, temporarily pin the last working version:

    terraform {
      required_providers {
        azurerm = {
          source  = "hashicorp/azurerm"
          version = "= LAST_WORKING_VERSION"
        }
      }
    }
    

    Then run:

    terraform init -upgrade
    terraform plan -refresh-only
    

    Commit the resulting lock file. If the previous lock file is unavailable, use the provider version recorded in the last successful pipeline logs or build artifact. Test candidate versions in a non-production run rather than guessing in the production state.

    2. Test the watchlist-item REST operation directly

    Run this under the service-principal identity used by Terraform. For client-secret authentication locally:

    az login --service-principal \
      --username "$ARM_CLIENT_ID" \
      --password "$ARM_CLIENT_SECRET" \
      --tenant "$ARM_TENANT_ID"
    
    az account set --subscription "$ARM_SUBSCRIPTION_ID"
    az account get-access-token --resource https://management.azure.com/ \
      --query '{tenant:tenant,subscription:subscription,expiresOn:expiresOn}' -o json
    

    Never print or upload the client secret, access token, or unsanitized debug logs.

    Set the ARM resource path and call the item directly:

    ITEM_ID="/subscriptions/[Subscription ID] ID]/resourceGroups/[Resource Group] Group]/providers/Microsoft.OperationalInsights/workspaces/$WORKSPACE/providers/Microsoft.SecurityInsights/watchlists/$WATCHLIST_ALIAS/watchlistItems/$ITEM_ID_GUID"
    
    az rest \
      --method get \
      --url "${ITEM_ID}?api-version=2024-09-01" \
      --output json
    

    2024-09-01 is a documented watchlist-item resource API version ^2^. That page describes Terraform through AzAPI; the HashiCorp page above separately establishes the AzureRM resource.

    Interpret the result diagnostically:

    • REST succeeds, Terraform still returns 401: concentrate on the AzureRM version and the identity actually selected by the pipeline.
    • REST also returns 401 or 403: capture the response, UTC timestamp, subscription, workspace, request/correlation ID, and effective identity for escalation. A 401 often indicates authentication or token context, while RBAC denial more commonly produces 403, but endpoint behavior can vary.
    • REST reaches a different subscription or tenant: correct the pipeline identity context.

    For CI using OIDC/workload identity, do not switch it to a client secret for this test. Run the REST call inside the same federated-authenticated job and inspect only non-secret context. For client-secret-based Terraform, safely display only:

    printf 'ARM_CLIENT_ID=%s\nARM_TENANT_ID=%s\nARM_SUBSCRIPTION_ID=%s\n' \
      "$ARM_CLIENT_ID" "$ARM_TENANT_ID" "$ARM_SUBSCRIPTION_ID"
    

    Terraform uses these values to select the service principal, tenant, and subscription; a pipeline can therefore target a different subscription from an interactive CLI session ^3^.

    3. Separate management failure from watchlist-data availability

    In Sentinel Logs, run:

    _GetWatchlist('BreakGlassAccounts')
    | take 10
    

    If data is returned while Terraform and the direct management call fail, the watchlist still exists and the problem is isolated to the management/API path. Avoid deleting and recreating it. Microsoft specifically notes that query and management access can diverge during transient issues ^4^.

    Also check Azure Service Health for Sentinel impact, especially because this began suddenly with no watchlist change.

    Permissions and ISOC/URBAC

    Sentinel SIEM uses Azure RBAC, and Microsoft lists Microsoft Sentinel Contributor at the Sentinel resource-group scope as the recommended service-principal role for automated management. Owner is broader Azure access, so adding Sentinel Contributor should not be the first response when Owner is already effective ^5^. Watchlist reads use Microsoft.SecurityInsights/Watchlists/read ^6^.

    The Defender XDR unified RBAC preview is documented separately from Sentinel SIEM Azure RBAC. Since it has not been activated here, the available evidence does not establish ISOC/URBAC as the cause.

    If both the direct REST call and Terraform continue returning 401 with the correct tenant, subscription, and unchanged provider, open an Azure support case with the captured request details and the last-success/first-failure times.


    References

    1. terraform-provider-azurerm/website/docs/r/sentinel_watchlist_item.html.markdown at main · hashicorp/terraform-provider-azurerm · GitHub
    2. Microsoft.SecurityInsights watchlists/watchlistItems 2024-09-01 (deployment-language-terraform)
    3. Authenticate to Azure with service principal
    4. Watchlists in Microsoft Sentinel
    5. Roles and permissions in the Microsoft Sentinel platform
    6. Azure permissions for Security
    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.