nfc-data-ingestion-service

Project information

A self-service data ingestion platform that lets teams connect to 30+ databases (PostgreSQL, ClickHouse, Sybase, Snowflake, and more), upload CSVs with a visual column mapper, and execute bulk INSERT/UPSERT operations with real-time execution analytics.

Built on a FastAPI backend with Google OAuth, encrypted credential storage, and a full audit trail — paired with a React/TypeScript frontend that includes an AI-powered query risk analyzer before execution.

Designed for data engineering workflows where traceability and security matter: every operation is hashed into an immutable audit chain, and all inputs are validated against SQL injection at the API layer.

Python React CSS Docker TypeScript GitLab fastapi data-ingestion sqlalchemy open-ai psutil vite ETL upsert