As a Data Engineer, I collect, extract and transform raw data in order to provide clean, reliable and usable data.
📘 Introduction One of DuckDB’s most useful features is the ability to query CSV files directly—no need to load them into a database first. This tutorial will guide you through running SQL queries on a CSV file using Python. ✅ Prerequisites Before you begin, make sure you have: 🐍☑️ Installed Python...
📘 Introduction If you enjoy working with pandas but wish you could use clean, powerful SQL at any time, then DuckDB is the right tool for you. With DuckDB, you can query your DataFrames instantly without having to set up a database, run a server, or change your workflow. ✅ Prerequisites Before...
📘Introduction In this blog post, we’ll walk through how to install DuckDB, a fast and lightweight analytical database engine designed for modern data workflows. Whether you're working on data analysis, ETL pipelines, or experimenting with in-process SQL queries, DuckDB is incredibly easy to set up and use....
📘 Introduction In many real-world datasets, the same type of information can appear in more than one column. A customer may provide an email address, a phone number, or a backup contact, and different systems may populate different fields. When you want to select the first available non-null value from several...
📘Introduction In this blog post, we’ll explore DuckDB—a lightweight analytics engine that’s transforming the way data professionals and enthusiasts analyze data locally. You’ll learn what it is, why it’s gaining popularity, and how it can make your data workflows faster and simpler. By the end,...
📘Introduction In this hands-on dbt tutorial, you'll learn how to use pre-hooks to automate tasks such as creating backup tables before a model runs. Pre-hooks allow you to execute SQL before your dbt model builds, which is useful for auditing, data quality checks, or preparing the environment. 🎓 Preparing...
📘 Introduction Modern systems produce endless streams of real-time data — from app events and online purchases to sensor readings and transactions. To handle this flow efficiently, applications need a fast, reliable way to move data between services as it happens. That’s where Apache Kafka comes in. Kafka is a distributed...
📘Introduction In this hands-on dbt tutorial, you’ll learn how to configure your sources.yml file to handle multiple environments (DEV and PROD) dynamically using dbt’s powerful target variable. This will ensure that your models always connect to the correct database, based on the environment you are currently working...
📘 Introduction Modern applications generate massive amounts of data — clicks, payments, sensor readings, and more — all happening in real time. To handle this flow efficiently, systems need a reliable way to move data between components instantly. That’s where Apache Kafka comes in. Kafka is a distributed data streaming platform built...