Synapse Data Engineering in Microsoft Fabric is a powerful tool for building and managing, data pipelines at scale. It allows users to ingest, transform, and orchestrate data from multiple sources in a seamless and efficient manner. With capabilities like Apache Spark integration, data engineers can seamlessly build and optimize robust ETL workflows. Here, you will find hands-on tutorials, end-to-end guides, and best practices for using Synapse Data Engineering in Microsoft Fabric.
Introduction In Microsoft Fabric, a workspace is essential for the creation and collection of components. To enhance the organization of these components, folders can be created within a workspace. In this tutorial we will demonstrate how to create a folder in a workspace. Goal A folder should be created in...
Introduction In this tutorial, we will explore how to read a Delta table into a PySpark DataFrame. Goal A Delta table stored in a lakehouse should be read into a PySpark DataFrame. Prequestisies ☑️ Notebook created We have already created the notebook "dlnerds_notebook". If you want to know...
Introduction In this tutorial, we will walk through the steps required to write a PySpark DataFrame to a Delta table in Microsoft Fabric. We'll start by creating an example DataFrame, then demonstrate how to write it to a Delta table. Goal An example PySpark DataFrame should be created...
Introduction In Microsoft Fabric, notebooks can interact very closely with lakehouses. In this tutorial, we will explain step-by-step how to connect a lakehouse and a notebook in Microsoft Fabric. Goal A lakehouse and a notebook should be connected. Prequestisies ☑️ Notebook created We have already created the notebook "dlnerds_notebook&...
Introduction One key component of Microsoft Fabric is the Lakehouse. In a lakehouse, custom schemas can be created to group tables together. In this tutorial, we will explain step-by-step how to create a lakehouse schema. Goal A new lakehouse schema should be created. Step 1: Sign in to Fabric First,...
Introduction In Microsoft Fabric, you can schedule a data pipeline by defining specific times and frequencies for your data processing tasks. This automation helps ensure that your data is constantly updated and available at the right time. In this tutorial, we will explain step-by-step how to schedule a data pipeline...
Introduction In this tutorial, we will explain step-by-step how to query data from a Fabric lakehouse with T-SQL using an SQL analytics endpoint. Goal Data should be queried from a delta table in the lakehouse with T-SQL using an SQL analytics endpoint. Data We consider the delta table student that...
Introduction In Microsoft Fabric, there are several activities that can be added to a data pipeline. One import activity is the Copy activity. In this tutorial, we will explain step-by-step how to add a Copy activity to a data pipeline using the Data Factory user interface. Goal A CSV File...
Introduction In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to use a notebook. In this tutorial, we will explain step-by-step how to ingest data into a Fabric lakehouse using a notebook....