Problems with running Fabric · Learn Microsoft Fabric

Problems with running Fabric

Hi Guys,

I have noticed a problem with a solution I have deployed using Fabric.

I am using a trial capacity (FT1).

I am using a medallion architecture with a data pipeline to orchestrate this process.

On the bronze layer, I have a dataflow that pulls the latest data from a web source as a raw table and store this data in a lakehouse (this seems to always work).

However there is another dataflow that pulls the data from the bronze lakehouse and then performs cleansing on this source. This is where the problems start as some of the time this works just fine and the latest raw data from the bronze lakehouse is cleansed into my silver lakehouse. However there are times where it doesn't bring in the latest data pulled from the bronze lakehouse.

This is the M-code to access the bronze lakehouse data:

let

Source = Lakehouse.Contents([]),

#"Navigation 1" = Source{[workspaceId = "c79766a3-4f30-43d3-942c-d1fa4e84b64d"]}[Data],

#"Navigation 2" = #"Navigation 1"{[lakehouseId = "72ab087e-c6f4-40dd-8149-95bd9c8e5bb5"]}[Data],

#"Navigation 3" = #"Navigation 2"{[Id = "Files", ItemKind = "Folder"]}[Data],

It is very frustrating as it makes no sense.

1 comment