Activity
Mon
Wed
Fri
Sun
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
What is this?
Less
More
Memberships
Learn Microsoft Fabric
12.7k members • Free
6 contributions to Learn Microsoft Fabric
Apr 30 •
Non-Default Lakehouse Schema Problems
Hi everyone, I have the following problem: I have two lakehouse with schemas enabled. I'd like to load data from LH-A, do some transformations and then load the data into LH-B using notebooks. I have big struggles reading data from LH-A as LH-B is my default Lakehouse. It seems that reading and writing from schema Lakehouses works fine if it's your default LH, however when it's not I can't seem to read tables with three-level-namespace. Can anyone help me here with a code-snippet or is this a general limitation? All the examples I find refer to lakehouses that are set as default.
0 likes • May 2
Thanks a lot for your fast reply Will, I will try!
Apr 22 •
Accessing data via gateway in notebooks
Hello everyone, I would like to develop a flow via pyspark-notebooks in Microsoft Fabric. More specifically, I want to extract tables from SAP and store the data in a Lakehouse after a few transformations. I need an on-premise data gateway for the connection. Furthermore, I would like to avoid the following if possible: SAP->Lakehouse->Notebooks->Lakehouse Does anybody know if I can use the on-premise data gateway directly in Notebooks so that I don't have to stage the data anywhere? Or do I have to use DataPipeline Copyjobs or Dataflows when it comes to using the DataGateway?
May '24 •
Future of Lakehouse vs Warehouse
Hey All, looking into in this release plan, it sparked a question in me if you see the highlighted features related to lakehouse and warehouse in the attached screenshot. Warehouse: To get Spark support. Lakehouse: To get schema support and additional security features like OLS. I mean, till now all the discussions I have seen, Lakehouse was chosen for Spark and Warehouse for Security. Now if both gets both, would be an interesting discussion to start on how the future of lakehouse vs warehouse do you see? When would you pick what, or will they even get merged (too far I know😂) ?
0 likes • Jul '24
@Vinayak K Got your point. However, I still find it difficult to derive a "best practice" as to when which variant (DLH or DWH) is the better choice, apart from the obvious decision criteria such as whether I prefer Python or SQL, whether I have structured or unstructured data, etc. Let's assume I can handle both languages and only work with structured data whose schema does not change, why shouldn't I still use a lakehouse instead of a warehouse? Because I would then deprive myself of the opportunity to work with stored procedures and have no access to system tables? I'm just thinking out loud here, I'm not sure.
1 like • Jul '24
@Vinayak K Thanks for sharing your thoughts here!
Jul '24 •
Create/Update Tables in Lakehouse with TSQL?
Hello, I know it is (currently) not possible to create/update tables in the Lakehouse using TSQL as the endpoint is read only. However, I think I read somewhere, that this is a feature that will be implemented in the future, but I can't find the source anymore. Can someone confirm this or am I wrong?
Apr '24 •
Notebook to DWH not possible?
Hi everyone, does anyone know why it's (technically) not possible to use a notebook to store a table into a DWH? I have read this several times and don't understand the technical obstacle to do so.
1 like • Apr '24
Understood! Makes perfect sense to me. Thanks a lot! :)
1 like • Apr '24
Thanks a lot for your answer and giving a clearer picture with some more background knowledge here! :) Btw: Looks like the "communication" between those two is already planned on the MF Roadmap and soon to beil added: https://learn.microsoft.com/de-de/fabric/release-plan/data-engineering
1-6 of 6
Active 39d ago
Joined Mar 31, 2024
Powered by