ETL Pipeline Projects
An ETL pipeline extracts data from a source, transforms it, and loads it into a target. These are real ETL pipelines you build and debug in a live cloud workspace - each one a portfolio piece.
Get started - freeLive cloud workspace in your browser · nothing to install
What makes an ETL pipeline production-ready
Moving rows once is easy. A production pipeline loads only what changed, gives the same result when it runs twice, rejects bad rows without losing them, and catches data that arrives late. Those are the properties these projects practice.
12 projects to start with
Backfill a Column Safely (Batched + Verified)Data · SeniorHandle Late-Arriving Data in an Incremental LoadData · SeniorSwitch Batch ETL to Streaming on AWS KinesisData · SeniorFix a Pipeline Silently Dropping RecordsData · MidSchedule a Pipeline as an Airflow DAGData · MidStream a Kafka Topic to ParquetData · MidBackfill 90 Days of Events With AirflowData · MidMake an ETL Pipeline IdempotentData · JuniorValidate Data Before Loading ItData · JuniorHandle Orphan Records in an ETL JoinData · JuniorImplement Incremental Loading in an ETLData · JuniorLoad Data Into AWS Redshift With dbtData · Junior
Pick by the skill to show
Reliability? The idempotent and dead-letter ones. Scale? Incremental loads, bulk loads and backfills. Design? Late-arriving data and event sourcing.
Do real production work, free. Live cloud workspaces in your browser - every project you complete lands on your portfolio.
Get started - free