D

datalake

A production-ready data pipeline that ingests data from public API endpoints, validates and processes it through a medallion architecture (Raw → Bronze → Gold) using Apache Airflow orchestration, AWS Glue transformations, and S3 Data Lake storage. Features robust error handling, circuit breaker patterns, and schema evolution management.