How to ingest Data into a Fabric Lakehouse using a Notebook

Introduction

In order to work with data in Microsoft Fabric, you first need to get data into it. There are several ways to do this. One way is to use a notebook. In this tutorial, we will explain step-by-step how to ingest data into a Fabric lakehouse using a notebook.

Goal

Source

The source data is a CSV file with the name student.csv stored under "Files" in the lakehouse.

Target

The target is a delta table in the lakehouse.

Step 1: Open Data Engineering Experience

We have already signed in into Fabric and opened the Data Engineering Experience. We have selected the workspace "dlnerds_workspace".

Here you can see the Data Engineering Experience.

Step 2: Create Notebook

We have already created the notebook "ingest_student". Check out the following post if you want to know how to create a notebook.

How to create a Notebook in Microsoft Fabric: A Step-by-Step Guide

Introduction Microsoft Fabric is a powerful All-in-One Data Platform (SaaS) in the Azure Cloud that combines various Azure components to cover the fields of Data Integration, Data Engineering, Data Science and Business Intelligence. One key component of the Microsoft Fabric architecture is the Notebook. In this tutorial, we will explain

Deep Learning Nerds | The ultimate Learning Platform for AI and Data ScienceDevOps Engineer

The notebook looks as follows:

Step 3: Connect Lakehouse

First, we need to connect the lakehouse to the notebook. To do this, the lakehouse must be added to the notebook using the explorer on the left-side bar.

Click on "Lakehouses".

Click on the "Add Lakehouse" button.

Select "Existing Lakehouse" and click on the "Add" button.

Navigate to the lakehouse that should be added to the notebook. In our case, we have selected the lakehouse "dlnerds_lakehouse".

Once connected, the lakehouse will be visible in the explorer with its "Tables" and "Files".

Step 4: Add Code to Notebook

Now, let's add the code cells to the notebook. We use PySpark for ingesting the raw data into the lakehouse.