As a Data Engineer, I collect, extract and transform raw data in order to provide clean, reliable and usable data.
Introduction In dbt, one of the most essential commands is dbt run. In this tutorial, we’ll dive deep into the dbt run command, exploring its syntax, functionality, and practical usage. This tutorial is also a valuable resource for those preparing for the dbt Analytics Engineering Certification Exam. 💡Explanation of...
Introduction In data processing, it's common to find timestamp fields as strings. Converting these string representations into proper timestamp formats is crucial for accurate data analysis and processing. In this tutorial, we will explore how to convert a string to a timestamp column in a PySpark DataFrame. Import...
Introduction In this tutorial, we’ll dive into the concept of Materializations in dbt and explore the different types available. Understanding materializations is key to optimizing how data is stored, queried, and updated within your dbt projects. Whether you're just starting with dbt or preparing for the dbt...
Introduction In data processing, it's common to find date fields as strings. Converting these string representations into proper date formats is crucial for accurate data analysis and processing. In this tutorial, we will explore how to convert a string to a date column in a PySpark DataFrame. Import...
Introduction dbt comes in two versions - dbt Core and dbt Cloud. While both provide the core functionality for data transformation, they serve different purposes and are suited to different requirements. In this tutorial, we’ll dive into the features of dbt Core and dbt Cloud, highlighting the key differences...
Introduction dbt empowers data professionals to transform raw data into clean, analytics-ready datasets. Mastering its essential commands is key to unlocking its full potential. In this tutorial, we’ll explore the key dbt commands, providing both the syntax and a short description for each command. This tutorial is also a...
Introduction When dealing with large datasets in PySpark, it's common to encounter situations where you need to manipulate string data within your DataFrame columns. One such common operation is extracting a portion of a string—also known as a substring—from a column. In this tutorial, we will...
Introduction FastAPI is a modern web framework that empowers developers to build web APIs quickly and efficiently using Python. A POST endpoint in FastAPI allows clients to submit data to the server, often for creating new resources. One part of a POST request ist the Request Body, which contains the...
Introduction FastAPI is a modern web framework that empowers developers to build web APIs quickly and efficiently using Python. A GET endpoint allows clients to retrieve data from a server. In a GET request, query parameters allow passing additional information to an endpoint via the URL's query string,...