Programming language used to interact with SQL Server databases
A lot better place to ask this question is https://community.fabric.microsoft.com/. Over here, we mainly focus on the old and classic SQL Server.
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hello,
I have a question about how a Fabric Warehouse accesses Lakehouse data.
We have a stored procedure running in a Warehouse that references Lakehouse tables using syntax similar to the example below:
SELECT DISTINCT
end_reason_test
FROM lh_Silver.test.test
WHERE end_reason_test IS NOT NULL;
Could you clarify how the Warehouse retrieves this data?
Specifically:
The reason I'm asking is that we occasionally notice delays between data being written to the Lakehouse and the data appearing to be available through Warehouse queries. I understand some delay may be expected, but I'd like to better understand the architecture and data access path so we can design appropriately.
Thank you.
Programming language used to interact with SQL Server databases
A lot better place to ask this question is https://community.fabric.microsoft.com/. Over here, we mainly focus on the old and classic SQL Server.
AI answer
A Fabric Warehouse can read Lakehouse data by referencing Delta Lake folders that are automatically represented as tables in the Lakehouse SQL analytics endpoint when they are stored in the Lakehouse /Tables folder. The documented pattern is to reference that data by three-part name notation using the Lakehouse name, schema, and table, for example MyLakehouse.dbo.TaxiTrips.
Based on the documented behavior:
For the delay being observed after writes to the Lakehouse, the supported conclusion from the documentation is that some propagation delay is consistent with Fabric’s Delta-based architecture and autogenerated SQL representations. The context does not provide a documented SLA, cache duration, or a detailed internal execution path beyond that.
For design guidance from the documented material:
A documented example of materializing Lakehouse data into Warehouse is:
CREATE TABLE dbo.TaxiTrips_2023
AS
SELECT *
FROM MyLakehouse.dbo.TaxiTrips
WHERE DATEPART(YEAR, tpep_pickup_datetime) = '2023';
That pattern is the clearest documented best practice in the provided material for reducing repeated cross-item reads and improving predictability.