Elasticsearch

Beats - Lightweight shippers for Elasticsearch & Logstash

Logstash is part of the Elastic Stack along with Beats, Elasticsearch and Kibana. Logstash is a server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite ...

Simple elastic csv loader/downloader command line

https://elasticcsv.readthedocs.io/en/latest/

Elasticsearch-like dissect pattern implementation in Python. https://dissec.touhey.pro/

Elasticsearch ingest pipeline validation. https://estceque.touhey.pro/

Indexation des nomenclatures (csv) dans ElasticSearch pour offrir un moteur de recherche

Elasticsearch beat to save messages received with the MQTT protocol. https://github.com/nathan-K-/mqttbeat

Elasticsearch Recon Ingestion Scripts (ERIS) 🔎

Tools for handling ElasticSearch queries in various languages. https://kaquel.touhey.pro/

English: Python project to extract PDF data from the current directory.

One script transforms the PDF data into a pandas dataframe for eventual conversion to Excel. The other script stores this data in JSON for indexing in a non-relational Elasticsearch database.

pt-BR:Projeto em Python para extração de dados de pdf do diretório atual.

Um script transforma os dados do pdf em pandas dataframe para eventual conversão para excel. O outro script armazena esses dados em JSON para indexar em um banco de dados não relacional Elasticsearch.

Transpiler for data processing pipelines. https://mustash.touhey.pro/

O projeto tem como objetivo coletar dados de qualidade do ar provenientes de estações distribúidas pelo Brasil, realizar o processamento e enriquencimento dos dados e os indexar no ElasticSearch.

Os dados são provenientes de API's com índices de qualidade do ar vindos de estações ao longo do Continente. Estes dados são filtrados para retornar apenas do Brasil, processados e enriquencidos com bibliotecas da linguagem Python e indexados no banco de dados não relacional ElasticSearch.

English: The project collects API data on traffic, users, holidays and weather, processes it and indexes it in ElasticSearch. It also performs geoqueries to update existing documents with climate data and other information. The project features a docker-compose to upload scripts as services to be run on a server.

Pt-BR: O projeto realiza coleta de dados de API de trânsito, usuários, feriados e clima, realiza o processamento dos mesmos e os indexa no ElasticSearch. Também realiza geoqueries para atualizar os documentos existentes com dados climáticos e outras informações. O projeto apresenta um docker-compose para subir os scripts como serviços a serem executados em um servidor.

The Fiscal ELK project provisions a server to provide a graphical interactive frontend to California's Open FI$Cal data and the City of Sacramento's budget data.

Python Flask app query Vietnamese online newspapers crawl by Selenium. Use Elasticsearch cloud engine (combine full-text search and cosine similarity score) online in Azure. Apply GitLab CI/CD pipelines.

Text search for a test task

A bot fetching front page of french newspaper L'Equipe every day.

A rudimentary search engine for movies. For that, a big amount of imdb movies were scraped and stored into Elasticsearch. Then, four types of queries were performed and evaluated as PoC. The API is written in Python Flask and fully dockerized.

This project deploys an elasticsearch demo with two web servers feeding logs to a single node elasticsearch instance.

Tutoriels en français sur la stack ELK : https://youtu.be/FIIWGs65Gi0