D

data-engineering

Arquitectura Medallón (Bronze/Silver/Gold) sobre The Simpsons API: ingesta cruda inmutable, contratos Pydantic con cuarentena, carga idempotente vía staging + UPSERT, y búsqueda semántica con FAISS. Dockerizado, testeado y auditable desde UI.

Bitcoin UTXO reconstruction pipeline with independent read-only auditor.

OpenBirth Platform is an open source demographic data system designed to track, organize, and analyze birth statistics across the world. The platform provides modular tools for collecting, validating, visualizing, and researching birth trends from national regions to local communities, supporting transparent demographic research through open data. https://roxanneardary.com/openbirth-platform/

NaturePharmacopeia is an open-source, all-natural database documenting the medicinal, coloring, and material properties of plants, minerals, and fibers. It integrates historical and traditional knowledge with modern AI-friendly search tools, allowing users to explore natural compounds, pigments, fibers, and materials, and discover safe, sustainable, and fully natural workflows for medicines, dyes, textiles, and other materials.

The project emphasizes 100% natural sources, safety, cultural context, and detailed preparation methods, providing a comprehensive reference for researchers, artisans, herbalists, and educators alike. https://roxanneardary.com/nature-pharmacopeia/

Pipeline híbrido (Batch + Streaming) para análise da alfabetização infantil no Brasil. Arquitetura Medalhão em GCP. Tech Challenge Fase 2 - POSTECH AI Scientist.

AI and data engineering portfolio ? explainable products, vector search, analytics and production-minded delivery.

Fundamental theory and practice in Data Science (DS). 🧮

Automated Urban Heat Island (UHI) data pipeline using ERA5 climate data and OpenStreetMap, orchestrated with Apache Airflow, processed with GeoPandas/xarray, stored in PostGIS, and visualized with Folium.

Real-time stock market data lakehouse using Kafka, Bronze/Silver/Gold architecture, and Postgres feature serving.

End-to-end IoT data platform on Kubernetes (kind) using FastAPI, MongoDB, and a background processor with a telemetry simulator. Includes Terraform IaC example.

End-to-end AWS data lake pipeline for fleet telemetry data using S3, Spark, and Athena. Includes partitioned Parquet ETL, vehicle safety analytics, and SQL queries for overspeed and harsh braking detection.

Real-time connected vehicle telemetry pipeline using Kafka, MongoDB, FastAPI, and Grafana with anomaly detection and fleet monitoring dashboards.

Distributed geospatial data pipeline using Apache Spark and Apache Sedona to analyze NYC taxi demand hotspots from millions of trip records.

Multi-source geospatial ETL pipeline integrating bike lanes, traffic sensors, and GTFS data to analyze urban mobility infrastructure coverage in Berlin.

Solución end-to-end para la migración y análisis de datos utilizando Python, FastAPI, Kafka y PostgreSQL. Implementa un pipeline de datos asíncrono y una API RESTful para analíticas, todo completamente containerizado con Docker Compose para un despliegue fácil y reproducible.

End-to-end design of a Hadoop-based ecosystem for healthcare data at scale (50 TB, IoT streams, medical imaging). Proposed a 10-node cluster architecture integrating HDFS, Spark, Hive, NiFi, Kafka, and Docker with HIPAA-compliant security (Kerberos, TLS, Apache Ranger). Delivered a proof-of-concept Docker deployment and professional proposal document.

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

See the documentation at: https://airflow-dbt-python.readthedocs.io/