Awesome GitHub RepositoriesMessage Reconstructors

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

Buf is a toolchain for managing the full lifecycle of Protocol Buffers schemas. It provides a set of tools for schema governance, including linting to enforce style guides, a breaking change detector to ensure backward compatibility, and a system for producing language-specific source code via local or remote plugins. The project distinguishes itself through a remote schema registry that centralizes the hosting, versioning, and distribution of Protocol Buffers modules. This registry allows for server-side enforcement of governance policies, such as blocking updates that introduce backward-inc

Rerun is a multimodal data visualizer and robotics data logger designed for rendering synchronized streams of 3D spatial data, images, and time-series metrics. It functions as a tool for capturing high-frequency sensor data and AI outputs into a queryable columnar format, providing a dedicated interface for viewing MCAP recording files and analyzing physical environments. The project distinguishes itself as a machine learning dataset streamer, capable of feeding logged recordings directly into GPU buffers and PyTorch training pipelines without intermediate exports. It supports a high-performa

tdl is a command-line tool for Telegram account automation, media management, and data archiving. It provides a programmatic interface for downloading and uploading files, forwarding messages, and exporting chat history, member lists, and media into structured JSON files. The project distinguishes itself through a session management system that isolates multiple account identities using unique namespaces. It features expression-based routing for messages and uploads, allowing users to direct content to specific destination chats using custom logic and dynamic caption generation. The tool cov

Kafdrop ist eine webbasierte Schnittstelle zur Überwachung und Verwaltung von Apache Kafka-Clustern, Topics, Brokern und Consumer-Groups. Es fungiert als Cluster-Monitor und Topic-Manager, der eine visuelle Darstellung des Broker-Status, der Partitionszuweisungen und des Consumer-Group-Lags bietet. Das System enthält einen Message-Browser, der Nachrichten aus Kafka-Topics lesen, dekodieren und veröffentlichen kann, wobei Schema-Registries oder Deskriptordateien verwendet werden. Es bietet zudem eine Metadaten-API, die Cluster-Informationen über JSON-Endpunkte für die Integration in externe Monitoring-Tools bereitstellt. Die Plattform deckt administrative Bereiche wie das Topic-Lifecycle-Management, Sicherheitsaudits von Access Control Lists (ACLs) und den Aufbau verschlüsselter Broker-Verbindungen mittels TLS-Zertifikaten und SASL-Anmeldedaten ab. Die Observability-Funktionen umfassen die Verfolgung von Offsets pro Partition und die Berechnung des Consumer-Group-Lags.

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer