12,201 questions with Azure Data Factory tags

Sort by: Updated
4 answers

Copy data from Sharepoint using Authorization: Bearer token getting status code 401 Unauthorized

ErrorCode=HttpRequestFailedWithUnauthorizedError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Http request failed with status code 401 Unauthorized, usually this is caused by invalid credentials, please check your activity…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-03T11:20:09.3133333+00:00
Prakash Palanivelu 0 Reputation points
answered 2026-09-20T06:43:09.7066667+00:00
Smaran Thoomu 35,870 Reputation points Microsoft External Staff Moderator
0 answers

Data Factory: HubSpot Connector Out of Date — Errors with Certain Endpoints

Problem description I am experiencing errors when using the Azure Data Factory HubSpot connector 2.0, particularly when accessing certain endpoints like the marketing campaigns table during a source-data update pipeline. The behavior did not meet my…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-16T23:15:08.8+00:00
Noah Daniels 0 Reputation points
commented 2026-09-18T17:23:35.3866667+00:00
Noah Daniels 0 Reputation points
2 answers

Error occurring during the SSH handshake with ADF

I am trying to find a solution on this error when I tried to copy data from a client who uses AWS for SFPT. I am using ADF (Azure Data Factory) to move data from this client's SFTP. Bottom is details of error…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-14T21:28:00.8833333+00:00
Justin Doh 1,000 Reputation points
edited a comment 2026-09-18T16:55:22.13+00:00
Justin Doh 1,000 Reputation points
1 answer

How to bypass locked temporary Office files (~$*) during bulk FileServer-to-ADLS copy in ADF without high cost?

Hi Everyone, I am building an incremental ingestion pipeline in Azure Data Factory (ADF) to copy ~70,000 files (across 47 different file types) from an on-premises FileServer into ADLS Gen2 using a Self-Hosted Integration Runtime (SHIR). The Challenge We…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-18T14:45:46.0133333+00:00
Misbah Bin Hossain 0 Reputation points
answered 2026-09-18T15:26:37.2933333+00:00
Allan Solomon Mejia 9,000 Reputation points
0 answers

Unable to delete data workflows gen2 tasks under fabric workspace

Hi Team, Under fabric workspace, we have created Gen2 dataflow tasks. We would like to delete the Gen2 Workflow tasks, by removing the linages, but we are getting error. Then same is able to delete after some days, either it takes 1 week or 10days. In…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-18T10:37:53.7866667+00:00
Santosh Kumar Joshi 0 Reputation points
1 answer

How to Ingest a table of xml data from on-prem DB2 warehouse to ASDLGen2 using ADF

Hi, I am using ADF to incrementally load a table containing 2 XML columns from on-prem DB2 warehouse to ADSLGen2 in parquet format. In Copy activity, I use DB2 connector at source and parquet dataset at sink; I also use Query with XMLSERIALIZE for these…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-16T14:13:37.9933333+00:00
Nhan, Phuong 0 Reputation points
answered 2026-09-17T19:34:50.6166667+00:00
Nhan, Phuong 0 Reputation points
1 answer One of the answers was accepted by the question author.

Setup ADF authen with personal github

I have error when I try setup repo git integration with new credential it's not working. check in tab network it return 401 but I don't see any step input credential or token???? why it have token in here?? where it come from??

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2024-05-10T05:21:26.4366667+00:00
Duy Nguyen 20 Reputation points
commented 2026-09-17T12:11:24.4233333+00:00
Janne Kujanpää 266 Reputation points
1 answer

Issues Azure data Factory SFTP connection ( SSH client)

HI MS Team, We encounter some issues with connecting a SFTP server of one of our financial counter parties. The issue is due to security level that they do not support rsa-sha  but only the rsa-sha2  version for host key exchange. If i look at the error…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-07T10:30:11.16+00:00
Shivata Tikoo 0 Reputation points
commented 2026-09-17T04:34:40.53+00:00
SIVASANKAR YEDDULA 170 Reputation points Microsoft External Staff Moderator
0 answers

Azure Data Factory Copy Activity fails extracting ZIP files to ADLS Gen2 with BadRequest for Unicode/Vietnamese filenames — possible ZIP filename encoding issue

We are using Azure Data Factory Copy Activity to extract ZIP files and write the extracted files to Azure Data Lake Storage Gen2. The Copy Activity fails for some files/directories inside the ZIP with the following error: The ZIP contains…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-17T04:19:23.85+00:00
Balasuriya R 0 Reputation points
2 answers One of the answers was accepted by the question author.

Recommended technologies for creating a dashboard of ongoing jobs.

I'm looking for recommendations on technologies to use for the following project. I need to create a dashboard of summarized job data from a network of hundreds of users. Our current application is used in the field for inputting and tracking job costs…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2024-05-09T14:00:50.17+00:00
COW Farmer 40 Reputation points
accepted 2026-09-16T15:54:20.8266667+00:00
COW Farmer 40 Reputation points
1 answer

Ingest multiple xml files through ADF which has different schemas

Hi all, I have an Azure blob storage of xml files with at least two different schemas. I would like to ingest all of these files through ADF (probably using Dataflow) and only process one type of schema to sink to a SQL database. How is this done…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2023-05-08T15:25:06.67+00:00
Raymond Mitchell 15 Reputation points
commented 2026-09-16T13:34:05.1933333+00:00
Emilio Molina 0 Reputation points
2 answers

Update of parentaccountid into CRM from ADF

I'm trying to update records from a .csv file through ADF copy activity into CRM. It's working fine for the GUIDs present for parentaccountid and updating accordingly. However, when a blank value is sent for this field, it's not removing the existing…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-16T10:54:54.49+00:00
Vidyashree Dinesh 0 Reputation points
answered 2026-09-16T11:22:10.13+00:00
Radon 0 Reputation points
7 answers One of the answers was accepted by the question author.

How to fix: Request Id: fccd854a-cfc2-43a1-89dd-0403c2493700 Correlation Id: 4fb66761-4ec3-477e-8c3b-d9c2a5c8562e Timestamp: 2024-01-16T19:29:03Z Message: AADSTS900561: The endpoint only accepts POST requests. Received a GET request.

Request Id: fccd854a-cfc2-43a1-89dd-0403c2493700 Correlation Id: 4fb66761-4ec3-477e-8c3b-d9c2a5c8562e Timestamp: 2024-01-16T19:29:03Z Message: AADSTS900561: The endpoint only accepts POST requests. Received a GET request. I can´t login in the web anymore…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2024-01-16T19:32:51.41+00:00
Guillermo Camacho 45 Reputation points
commented 2026-09-10T13:55:31.5166667+00:00
Tiffany Daylight 0 Reputation points
3 answers

Checkpoint Key on Azure Data Factory (ADF)

Hi, We have created Azure Data Flow with Azure SQL Database Managed Instance as source and Azure Data Lake Storage Gen 2 (ADLS) as a sink (parquet format). Change Data Capture is enabled at the source with SQL Server CDC type and "Full on the first…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-02T16:03:08.1133333+00:00
Anil Kestur 5 Reputation points
commented 2026-09-10T07:44:27.26+00:00
SIVASANKAR YEDDULA 170 Reputation points Microsoft External Staff Moderator
2 answers One of the answers was accepted by the question author.

Fabric scan is ingesting Lakehouse schemas but not Warehouse schemas in my environment.

Hi All I am trying to scan my Fabric environment into Purview but I have hit a snag, I want to be able to drill down into the my datawarehouse to extract the Tables and Views, but I am not sure how to do it When I open the Lineage in the scanned…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-09T04:51:25.44+00:00
Chaz Abetz 40 Reputation points
answered 2026-09-09T23:07:45.9666667+00:00
Chaz Abetz 40 Reputation points
2 answers

Trying to filter csv file based on content

I have an existing ADF pipeline that is creating an error due to some bad (or not acceptable data) displayed on the csv file. Basically, instead of having proper header on the first row with certain amount columns (with data), csv file displays data like…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-07-22T21:45:14.68+00:00
Justin Doh 1,000 Reputation points
commented 2026-09-08T21:07:34.2266667+00:00
Justin Doh 1,000 Reputation points
1 answer One of the answers was accepted by the question author.

@microsoft/azure-data-factory-utilities: purpose of the factoryId parameter

Hi, I have recently started using the @microsoft/azure-data-factory-utilities package to generate ARM templates in an Azure DevOps pipeline. It is working, and I am able to generate the ARM templates successfully. :) However, I have a question about the…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-04T07:55:42.2233333+00:00
Bánhidy Attila 40 Reputation points
accepted 2026-09-07T06:19:14.1033333+00:00
Bánhidy Attila 40 Reputation points
2 answers One of the answers was accepted by the question author.

SFTP connection problem from Azure Data Factory to a server that only supports rsa-sha2-256/512

Hello, I am trying to connect from Azure Data Factory (both with Managed IR and Self-Hosted IR) to the SFTP server of a supplier. But i receive the following error: "Failed to connect to Sftp server 'xxxxxx'. The connection was closed by the…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2025-09-11T15:47:40.2966667+00:00
Luigia Costabile 70 Reputation points
edited a comment 2026-09-05T21:45:17.07+00:00
Danny 0 Reputation points
2 answers

enable the access to local machine under self hosted integration runtime

enable the access to local machine under self hosted integration runtime

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-02T06:35:00.2133333+00:00
adula amala 0 Reputation points
answered 2026-09-02T12:03:21.5066667+00:00
Himaja Y 545 Reputation points Microsoft External Staff Moderator
3 answers

How to handle BigQuery mirroring when the source has more than 1,000 tables?

Hi, We are setting up database mirroring in a Microsoft Fabric workspace, with BigQuery as the source. Currently, our BigQuery source contains 1,193 tables, but a single mirrored database supports a maximum of 1,000 tables. Because of this limitation, we…

Azure Data Factory
Azure Data Factory

An Azure service for ingesting, preparing, and transforming data at scale.

12,201 questions
asked 2026-09-02T03:23:14.4133333+00:00
VIGNESH G 40 Reputation points
answered 2026-09-02T11:42:20.5466667+00:00
Himaja Y 545 Reputation points Microsoft External Staff Moderator