Data Engineer

Data Engineer

As a Data Engineer, I collect, extract and transform raw data in order to provide clean, reliable and usable data.

76 posts
How to group Data using a Dataflow in Microsoft Fabric
Academy Membership Microsoft FabricAzure

How to group Data using a Dataflow in Microsoft Fabric

Introduction One fundamental part of Microsoft Fabric is transforming data. Whether filtering, joining, merging or grouping data, there are several options available in Fabric to perform these operations. In this tutorial, we will explain step-by-step how to group data and apply an aggregation function using a dataflow. Goal A delta...

PySpark - Group and Concatenate Strings in a DataFrame
Academy Membership PySparkPython

PySpark - Group and Concatenate Strings in a DataFrame

Introduction In this tutorial, we will show you how to group and concatenate strings in a PySpark DataFrame. In order to do this, we will use the groupBy() method in combination with the functions concat_ws(), collect_list() and array_distinct() of PySpark. Import Libraries First, we import the following...

PySpark - How to use Pandas User Defined Function (UDF)
Academy Membership PySparkPython

PySpark - How to use Pandas User Defined Function (UDF)

Introduction In the realm of big data processing, PySpark has emerged as a powerful tool for handling large-scale datasets. Its distributed computing framework allows for efficient processing of massive volumes of data. However, despite its capabilities, performing certain data transformations in PySpark can sometimes be cumbersome and complex. That'...

You’ve successfully subscribed to Deep Learning Nerds | The ultimate Learning Platform for AI and Data Science
Welcome back! You’ve successfully signed in.
Great! You’ve successfully signed up.
Success! Your email is updated.
Your link has expired
Success! Check your email for magic link to sign-in.