How to run automated regression testing against an Azure App Service staging slot before swapping to production?

You might want to consider making regression testing an explicit stage between deployment to the staging slot and the production swap. The pipeline would deploy the release to staging, verify that the application is healthy, obtain an access token for Microsoft Entra ID protected endpoints, run automated regression tests against the staging URL, and permit the swap only if every required test succeeds.

The key would be to keep staging isolated from production data and dependencies but, at the same time, ensuring the tests exercise the same application behavior, authentication policies, and deployment configuration that production will use.

1. Authentication: calling the staging API

Use a dedicated Microsoft Entra ID application identity for the regression test runner. A service principal authenticated through workload identity federation from Azure DevOps or GitHub Actions is a suitable choice because it avoids storing a client secret in the pipeline. If the test runner executes on an Azure resource that supports managed identity, a managed identity is also appropriate.

Configure the API to accept application tokens, not just delegated user tokens. Define an application permission, such as RegressionTest.Access, expose it through the API's app registration, grant the test runner that application permission, and require the appropriate app role in the API's authorization policies. Limit this identity to the permissions needed for testing. Do not use an administrator account or a production user's credentials.

The pipeline would request an access token for the API's application ID URI and send it as a bearer token to the staging endpoint. The API would validate the token's issuer, audience, signature, and authorization claims. For example, the test runner can obtain a token using the Azure CLI after authenticating through the pipeline's federated identity:

az login --service-principal \
  --username "$AZURE_CLIENT_ID" \
  --tenant "$AZURE_TENANT_ID" \
  --federated-token "$AZURE_FEDERATED_TOKEN"

az account get-access-token \
  --scope "api://YOUR-API-APP-ID/.default"

The test framework would then attach the returned access token to its HTTP requests. Adapt the login step to the identity mechanism supported by your CI/CD platform.

2. Slot configuration and swap with preview

Mark environment-specific settings as deployment slot settings so they remain associated with the slot during a swap. Configure staging with its own test database connection string, test-service URLs, credentials or managed identity settings, and any other environment-specific values. Production should retain its production configuration.

For Azure SQL, use a separate staging database or isolated test database with synthetic or sanitized data. Store connection strings in Azure Key Vault or App Service configuration using managed identity where applicable. Verify that the staging app's identity has access only to the intended test resources. Avoid relying on a connection string that can silently direct regression tests to production.

Swap with preview does not replace this configuration strategy. During preview, Azure applies the target slot's settings to the source slot and restarts the source app so you can validate it under the configuration it will have after the swap. This is useful for checking production-specific configuration, but it means the source slot may no longer be using its original staging configuration. Do not run tests that write data or call external services during preview unless those tests are explicitly safe against the production-target configuration.

3. Gate the production swap on regression test results

Make the deployment pipeline enforce an explicit dependency - the swap stage should run only after the staging deployment and regression test stage have succeeded. A test failure, timeout, or infrastructure error should fail the pipeline and prevent the swap. Do not implement this as a notification or a manual approval that can be bypassed accidentally.

For Azure DevOps YAML, the basic structure is:

stages:
- stage: Deploy_Staging
  jobs:
  - job: Deploy
    steps:
    # Deploy the application to the staging slot.
    - script: echo "Deploy staging here"

- stage: Regression_Tests
  dependsOn: Deploy_Staging
  condition: succeeded()
  jobs:
  - job: Test
    steps:
    # Run the regression test suite against the staging URL.
    - script: dotnet test tests/RegressionTests.csproj --configuration Release

- stage: Swap_To_Production
  dependsOn: Regression_Tests
  condition: succeeded()
  jobs:
  - deployment: Swap
    environment: production
    strategy:
      runOnce:
        deploy:
          steps:
          # Execute the App Service slot swap here.
          - script: echo "Swap to production here"

The deployment and swap commands above are placeholders, not a complete deployment implementation. Replace them with your existing App Service deployment task or Azure CLI commands, configured for the correct app, resource group, and slot. Ensure the regression job returns a nonzero exit code when any required test fails. If the pipeline uses deployment jobs, multiple dependencies, or custom conditions, review the conditions carefully so no alternative path allows the swap to proceed after a failed test stage. Keep downstream dependencies stable

Regression tests should exercise the real integration paths where practical, but they should not depend on uncontrolled production activity or unreliable external systems. For SQL, use a dedicated staging database with predictable seed data, repeatable setup and cleanup, and a defined schema migration process. Run tests in an isolated transaction or reset test data between runs where feasible. Include tests for database permissions, expected queries, transactions, and compatibility with the deployed schema.

For external services, use dedicated test environments or vendor-provided sandboxes whenever possible. If a service has no reliable sandbox, use a controlled mock or service virtualization for most regression tests, then run a smaller integration suite against the real service. Configure explicit timeouts, bounded retries with backoff, and clear handling of rate limits. Retries should not mask application defects or repeat non-idempotent operations.

Separate application failures from dependency failures in test reporting. A failed test should identify whether the API returned an unexpected response, authentication failed, SQL was unavailable, or an external service timed out. Add a readiness endpoint that verifies the application can start, and use a lightweight smoke test before the full regression suite. Avoid making every readiness probe depend on every downstream service, because a transient dependency failure can otherwise make a healthy deployment appear unavailable.

5. Suggested end-to-end release flow

1. Deploy to staging

Use staging-specific configuration and test resources.

2. Verify readiness and authentication

Check application health and obtain an Entra ID access token.

3. Run regression tests

Execute API, authorization, database, and integration tests.

Tests fail

Fail pipeline, retain production slot, investigate.

Tests pass

Allow the production swap.

4. Swap and verify production

Run post-swap smoke tests and monitor errors, latency, and dependency health.

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin