Understanding _dlt_id and _dlt_parent_id in dlt
Learn what _dlt_id and _dlt_parent_id mean in dlt, why they appear in normalized tables, and how to use them in SQL joins.
Data Engineering
135 posts
How dlt Loads Nested JSON into Tables in DuckDB
Learn how dlt normalizes nested JSON into parent and child tables in DuckDB using a simple students and courses example.
When to Use replace, append, or merge in a dlt Pipeline
Learn when to use replace, append, or merge in a dlt pipeline with practical DuckDB examples and beginner-friendly explanations.
How to Load JSON Data into DuckDB using dlt
Learn how to load a local JSON file into DuckDB with dlt, inspect the result with SQL, and understand a simple data pipeline pattern.
PySpark Structured Streaming Explained for Beginners: Build a Real-Time Data Pipeline
Learn PySpark Structured Streaming step by step by building a beginner-friendly real-time data pipeline with JSON files, readStream, writeStream, output modes, triggers, and checkpoints.
How to Load CSV Data into DuckDB using dlt
Learn how to load a local CSV file into DuckDB with dlt, inspect the result with SQL, and understand the basic pipeline pattern.
dbt Snapshots Explained: Track Historical Changes in Your Data Warehouse
Learn what dbt snapshots are, why they matter, and how to track historical changes in your data warehouse with a beginner-friendly customer status example.
dbt Exposures Explained for Beginners: Connect Models to Dashboards, Reports, and ML Models
Learn what dbt exposures are, why they matter, and how to connect dbt models to dashboards, reports, notebooks, and machine learning workflows.
How to Load API Data into DuckDB with dlt
📘 Introduction In this hands-on tutorial, you will learn how to load data from a REST API into DuckDB using dlt. This is a great first local data pipeline because you do not need a cloud warehouse, a complex setup, or production credentials. We will use dlt to fetch data...