Skip to content
Home
Programming
Data Engineering

Data Engineering

Data engineering guides covering data pipelines, ETL/ELT, data warehousing, streaming, Spark, Airflow, and building reliable data infrastructure.

Learning Path — 18 articles

1 Data Engineering: Building Reliable Data Pipelines at Scale Learn data engineering foundations — ETL vs ELT, batch and stream processing, data warehousing, orchestration, quality … 8 min Start Here 2 ETL Pipelines: Building Reliable Data Workflows at Scale ETL Pipelines: Building Reliable Data Workflows at Scale. An accessible introduction to etl pipelines guide, exploring … 8 min Start Here 3 Streaming Data with Apache Kafka: Real-Time Pipelines Guide Streaming Data with Apache Kafka: Real-Time Pipelines Guide. A practical guide to streaming data kafka guide, covering … 8 min Start Here 4 Data Warehousing: Architecture, Design, and Cloud Options Data Warehousing: Architecture, Design, and Cloud Options. A practical guide to data warehousing guide, covering key … 8 min 5 Data Orchestration with Apache Airflow: Complete Guide Data Orchestration with Apache Airflow: Complete Guide. A practical guide to data orchestration airflow guide, covering … 8 min 6 Analytical SQL: Window Functions, CTEs, and Query Optimization Analytical SQL: Window Functions, CTEs, and Query Optimization. A practical guide to sql analytical processing guide, … 8 min 7 Data Quality Testing: Ensuring Reliable Data Pipelines Data Quality Testing: Ensuring Reliable Data Pipelines. Build your understanding of data quality testing guide with … 8 min 8 Modern Data Stack and Lakehouse Architecture Explained Modern Data Stack and Lakehouse Architecture Explained. Build your understanding of modern data stack guide with clear … 8 min 9 Apache Spark Guide: Distributed Data Processing at Scale Apache Spark Guide: Distributed Data Processing at Scale. A straightforward look at apache spark guide, covering what … 10 min 10 dbt Guide: Analytics Engineering for Modern Data Teams dbt Guide: Analytics Engineering for Modern Data Teams. Build your understanding of dbt guide with clear explanations … 8 min 11 Data Modeling Guide: Dimensional Design for Analytics Data Modeling Guide: Dimensional Design for Analytics. Learn what you need to know about data modeling guide: core … 9 min 12 Data Lakes: Architecture, Governance, and Best Practices Data Lakes: Architecture, Governance, and Best Practices. A practical guide to data lakes guide, covering key concepts, … 8 min 13 Data Pipeline Monitoring and Observability Learn data pipeline monitoring — observability pillars, metrics, monitoring tools, alerting, lineage tracking, and … 7 min 14 Data Security and Governance Guide for Data Engineers Data Security and Governance Guide for Data Engineers. A practical guide to data security governance guide, covering key … 9 min 15 Data Ingestion Guide Learn data ingestion — batch vs streaming ingestion, API ingest, database connectors, change data capture, file … 7 min 16 Data Transformation Guide Learn data transformation — SQL vs Python transformations, dbt models, data cleansing, feature engineering, and … 8 min Advanced 17 Data Lakehouse Architecture Guide Learn data lakehouse architecture — combining data lake and warehouse, Delta Lake, Apache Iceberg, ACID transactions, … 8 min Advanced 18 Data Catalog and Discovery Guide Learn data catalog and discovery — metadata management, data lineage, data discovery tools, cataloging best practices, … 8 min Advanced