Data Engineering: Zero to Expert

A complete path from zero to expert: Linux, Python, SQL, data modeling, warehouses, Spark, Airflow, Kafka, lakehouses, cloud and architecture. Free resources first, with paid courses and books listed separately.

13 phases69 topics171 free resources27 books

Press H for shortcuts.

Every topic lists free resources first; paid courses and books are listed separately. Many paid courses can be audited for free.

Showing 69 of 69 topics

By the end of this phase

You can explain what a data engineer builds, how data flows from source to dashboard, and which skills you need.

The Data Engineer Role & Data Lifecycle

Data engineers build the systems that collect, store, clean and serve data so analysts, scientists and apps can use it. Learn the lifecycle: generation, ingestion, storage, transformation and serving.

Free3

Paid1

Books1

Data Engineer vs Analyst vs Scientist vs ML Engineer

These roles overlap but differ: analysts answer questions, scientists build models, ML engineers deploy them, and data engineers make all of it possible with reliable data. Know where you fit.

Free2

ETL vs ELT & Batch vs Streaming

ETL transforms data before loading it; ELT loads raw data first and transforms it inside the warehouse. Batch moves data in chunks on a schedule; streaming moves it continuously. Most design choices come back to these two ideas.

Free2

The Modern Data Stack — Big Picture

Sources → ingestion → storage/warehouse → transformation → BI and ML. See where Airflow, dbt, Spark, Kafka, Snowflake and friends each fit before diving into them one at a time.

Free2

Career Paths & How Teams Work

Data engineer, analytics engineer, platform engineer, ML engineer — different companies mean different jobs. Learn how data teams are organized and how engineers work with stakeholders.

Free2

Start Here1–2 weeksPhase 1 — Orientation: What Data Engineers DoUnderstand the job, the data lifecycle and the modern data stack before learning any tool.0 / 5 topics

Foundations5–7 weeksPhase 2 — Core Foundations: Linux, Git, Python & SQLThe four tools used every day: the terminal, version control, Python and SQL.0 / 6 topics

SQL + Modeling5–6 weeksPhase 3 — Advanced SQL & Data ModelingMove from writing queries to designing the tables they run on.0 / 5 topics

Storage4–5 weeksPhase 4 — Databases & StorageWhere data lives: relational and NoSQL databases, file formats and object storage.0 / 5 topics

Warehouse5–6 weeksPhase 5 — Data Warehousing & Analytics EngineeringCloud warehouses, ELT with dbt, and turning raw tables into trusted analytics models.0 / 5 topics

Pipelines7–9 weeksPhase 6 — Batch Processing & OrchestrationMove and transform data on a schedule, at scale, and keep the whole thing running reliably.0 / 7 topics

Streaming5–6 weeksPhase 7 — Streaming & Real-Time DataProcess events as they happen: messaging, stream processing and change data capture.0 / 5 topics

Lakehouse3–4 weeksPhase 8 — Data Lakes & the LakehouseOpen table formats that bring warehouse reliability to cheap object storage.0 / 4 topics

Cloud6–8 weeksPhase 9 — Cloud & InfrastructureRun your pipelines on real infrastructure: cloud data services, containers, infrastructure as code and CI/CD.0 / 6 topics

Trust4–5 weeksPhase 10 — Data Quality, Governance & ReliabilityMake data trustworthy: test it, monitor it, document it, secure it and respect privacy.0 / 5 topics

ArchitectureOngoing (6+ weeks)Phase 11 — System Design & Data ArchitectureThink like an architect: scale, trade-offs and how the pieces fit into a whole platform.0 / 4 topics

AI Data4–6 weeksPhase 12 — Data for ML & AIAI is only as good as its data. Build the pipelines that feed machine learning and LLM applications.0 / 4 topics

BuildOngoing (start in Phase 6)Phase 13 — Portfolio, Career & Staying CurrentProve your skills with real pipelines, prepare for interviews and keep up with a fast-moving field.0 / 8 topics