K

kafka

A living error knowledge base for the Nielsen Fulfillment Center (NFC) — Realtime bugs logged, diagnosed, and fixed across the full stack (Java Spring Boot, React, MAF-CLI, PostgreSQL, Kafka). Built on the 0xDEBUG universal debugging taxonomy: a multiple-file Obsidian knowledge graph and Gemini AI assistant that classifies any error into one of the failure patterns across variousebugging knowledge-base obsidian gemini ai java spring-boot react typescript postgresql kafka liquibase hikari docker nielsen nfc devex developer-tools error-logging inner-source tech domains.

Sistema modular, orientado a eventos, para catálogo de conciertos, proceso de reserva con caducidad y pagos con Stripe. Construido con Java 21 y Spring Boot 3.5.x siguiendo DDD y Arquitectura Hexagonal. Actualizaciones al navegador mediante SSE.

Modular, event-driven system for concert catalog, reservation process with expiration, and payments with Stripe. Built with Java 21 and Spring Boot 3.5.x following DDD and Hexagonal Architecture. Browser updates via SSE.

Krun reads events from a Kafka topic and run a system command passing the read event to the stdin of the command. The command can be a bash script or anything else that can manage a stdin.

1000+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..

Change data capture on Oracle databases + transfer of change events to Kafka

Hybrid Filtering with Spark MLlib

Streaming data pipeline for European Day-Ahead electricity market — Cats Effect 3 + fs2 + http4s + Kafka

WebSocket proxy that allows clients to produce and consume messages from topics in Apache Kafka. Documentation can be found at https://kpmeen.gitlab.io/kafka-websocket-proxy/

Learning Spark in java/scala

Lab CDC event-driven : PostgreSQL → Debezium → Kafka → consumers Python (notifications, audit, risk scoring). Illustre WAL, replication slots, outbox pattern et eventual consistency sur un cas bancaire réel.

Projet de référence d'une architecture Lakehouse moderne appliquée à la détection de fraude bancaire.

Simule un environnement de production avec trois sources de données hétérogènes (fichiers CSV, base PostgreSQL, streaming Kafka/Redpanda) ingérées en continu vers un stockage objet S3-compatible (MinIO).

Stack technique :

Ingestion batch : Apache Spark (PySpark) + Delta Lake Ingestion streaming : Spark Structured Streaming + Redpanda (Kafka) Orchestration : Apache Airflow Transformation : dbt (DuckDB) Stockage : MinIO (S3), Delta Lake (Bronze/Silver), Parquet (Gold) Exploration : DuckDB / DBeaver

Architecture en médaillon (Medallion Architecture) :

Bronze : données brutes, sources séparées Silver : données nettoyées, déduplication inter-sources Gold : agrégats métier (fraude par heure)

L'ensemble de la stack tourne en local via Docker Compose.

Production-style real-time crypto streaming pipeline (Binance → Kafka → Polars Parquet → DuckDB → dbt) built locally with Docker and Python.

Real-time stock market data lakehouse using Kafka, Bronze/Silver/Gold architecture, and Postgres feature serving.

Real-time connected vehicle telemetry pipeline using Kafka, MongoDB, FastAPI, and Grafana with anomaly detection and fleet monitoring dashboards.

-> Contenu -Un seul topic order-events sur lequel sont publiés trois types d’événements : OrderCreated, PaymentValidated, OrderShipped. -Producer (REST API) : publie les événements ; Consumer : écoute le topic avec trois consumer groups (groupIds distincts) et délègue au domaine via des handlers. -Architecture : hexagonale + template method ; contrats partagés dans le module kafka-contracts (OpenAPI). -> Stack: -Java 21, Spring Boot 3.2, Spring Kafka -Kafka (Zookeeper), Kafka UI -Maven multi-module : kafka-contracts, order-event-producer, order-event-consumer -Docker / Docker Compose pour l’infra et les apps -CI/CD GitLab : build Maven, tests unitaires, build & push des images sur Docker Hub, déploiement par SSH + docker compose sur le serveur. -> Dépôts d’images -Images poussées sur Docker Hub par la CI : kafka-kata-producer, kafka-kata-consumer.

Sistema event-driven con Kafka que transforma documentos no estructurados en especificaciones de software completas. Extrae texto con OCR, procesa NER con transformers, clasifica oraciones y generar SRS en múltiples formatos.

E-commerce backend built with Java 17 + Spring Boot 3, Clean Architecture, Kafka, Elasticsearch, MySQL, JWT auth and observability (Filebeat + Elasticsearch).

Solución end-to-end para la migración y análisis de datos utilizando Python, FastAPI, Kafka y PostgreSQL. Implementa un pipeline de datos asíncrono y una API RESTful para analíticas, todo completamente containerizado con Docker Compose para un despliegue fácil y reproducible.

FastAPI that reads rss feeds from magazines, consolidates and broadcast them to pub/sub and clients.