Incremental ingestion in Fabric Notebook

20d (edited) • Technical

I had question - how to pass and save multiple parameter values to fabric notebook.

For example - for the below code how to pass 7 values for table in {Table} parameter sequentially and after every run need to save the last update date/time (updatedate) column values as variables and use these in the next run to get incremental values for all 7 tables.

Notebook-1

-- 1st run

query = f"SELECT * FROM {Table}"

spark.sql (query)

--2nd run

query-updatedate = f"SELECT * FROM {Table} where updatedate > {updatedate}"

spark.sql (query-updatedate)

2 comments