Data Engineering Zoomcamp

This project is an open-source educational curriculum designed to provide comprehensive training in data engineering. It focuses on building scalable data pipelines and managing cloud-native infrastructure through a structured, self-paced program that combines technical explanations with hands-on practical exercises.

The curriculum distinguishes itself by emphasizing industry-standard methodologies, specifically teaching students how to implement infrastructure as code and manage data workflows through orchestration tools. By utilizing container-based environment isolation and declarative configuration, the program ensures that learners gain experience with reproducible deployments and consistent development environments across distributed systems.

The training covers a broad range of technical topics, including the design of automated data processing tasks and the configuration of cloud resources. The materials are organized into modular, progressive units that build foundational knowledge before advancing to complex engineering workflows.

The course materials are hosted in a centralized repository, which facilitates community-supported updates and collaborative improvements to the educational assets.

Features

Data Engineering Curricula - A comprehensive technical syllabus focused on building scalable data pipelines, managing cloud infrastructure, and mastering modern distributed computing workflows.

Data Engineering - Focuses on building scalable data pipelines and storage systems using modern cloud infrastructure.

Data Pipeline Architectures - Designing and managing automated workflows that handle the movement, transformation, and scheduling of data across complex distributed systems.

Cloud Infrastructure Courses - A practical guide to provisioning and managing cloud resources using declarative configuration files and containerized execution environments for data-intensive applications.

Technical Training - Provides structured guides and hands-on practice to build professional data engineering skills.

Data Pipeline Orchestrators - "Teaches the use of automated workflow tools to schedule, monitor, and manage the execution of complex data processing tasks."

Infrastructure as Code - Automates cloud resource provisioning using declarative configuration files to ensure reproducible deployments.

Open-Source Learning Programs - A structured collection of learning materials and practical exercises designed to teach technical skills through a self-paced, community-supported program.

CI/CD and Orchestration - Data engineering course covering dbt.

Data Engineering - Course on data engineering fundamentals.

Infrastructure as Code Practices - Automating the provisioning and management of cloud resources through declarative configuration files to ensure consistent and reproducible deployments.

Curricula - Organizes complex technical topics into progressive, modular learning units.

Containerization - Teaches the use of container images to ensure consistent execution environments across development and production.

Development Environments - Ensures consistent execution environments by isolating dependencies within portable containers.

Curriculum Modules - Module 1: Containerization and Infrastructure as Code - Introduction to GCP - Docker and Docker Compose - Running PostgreSQL with Docker - Infrastructure setup with Terraform - Homework #### Module 2: Workflow Orche

Professional Development - Provides hands-on training in industry-standard tools for real-world engineering tasks.

DataTalksClubdata-engineering-zoomcamp

Data Engineering Zoomcamp

Features

Star history