12 repositorios
Systems designed for the high-volume ingestion, processing, and streaming of real-time sensor and event data.
Distinguishing note: Focuses on high-throughput ingestion and streaming pipelines for telemetry, distinct from general-purpose database management.
Explore 12 awesome GitHub repositories matching data & databases · Telemetry Data Pipelines. Refine with filters or upvote what's useful.
ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring. The platform distinguishes itself through ad
Processes millions of events per second from streaming sources to enable real-time analytics and monitoring.
SkyWalking is an application performance monitoring system and observability platform designed to collect and analyze metrics, traces, and logs from distributed microservices. It functions as a distributed tracing platform and a telemetry data pipeline that ingests and aggregates observability data from various language agents. The project features an AI-powered anomaly detector that uses machine learning to calculate metric baselines and identify irregular URI patterns. It includes an eBPF performance profiler for diagnosing CPU and network bottlenecks at the kernel level and generates inter
Implements a high-performance engine for the ingestion, processing, and aggregation of multi-source observability data.
OpenObserve is a unified observability data platform designed to ingest, store, and analyze logs, metrics, and traces. It functions as a cloud-native monitoring tool that centralizes telemetry from diverse sources, including standard collectors and cloud service providers, into a single, scalable system. By utilizing a columnar storage engine backed by object storage, the platform enables efficient long-term data retention and high-performance analytical querying. The platform distinguishes itself through deep integration with artificial intelligence, allowing users to query data using natura
Ingests, transforms, and routes telemetry data streams from cloud services and standard collectors.
Telegraf is a modular, cross-platform telemetry pipeline designed to collect, process, and route metrics from diverse infrastructure, applications, and hardware. It functions as a server-side middleware that normalizes heterogeneous data into a unified format, enabling consistent monitoring across complex environments. By utilizing a plugin-driven architecture, the agent manages the entire lifecycle of telemetry data from initial ingestion to final transmission. The project distinguishes itself through a declarative, configuration-driven execution model that allows users to define complex dat
Ingests metrics from various sources, applies transformations, and routes them to multiple configured output destinations.
This is a reference implementation library providing a collection of code samples, Transact-SQL scripts, and schemas for SQL Server, Azure SQL, and Azure Synapse. It focuses on providing standardized implementation patterns and reference code for building relational databases and cloud data warehouses. The library distinguishes itself by offering specialized guides and examples for deploying database instances within containerized environments and Azure cloud services. It includes specific reference databases and language extensions for integrating machine learning services and advanced analy
Provides specialized ingestion pipelines for streaming real-time sensor data into relational tables.
This project is a collection of utilities designed for machine learning experiment tracking, data versioning, and the observability of large language model applications. It provides a client for recording hyperparameters and metrics during training to visualize performance trends and compare different model versions. The tool includes a model evaluation framework that uses custom scorers and automated judges to assess the quality of generated text outputs. It also provides observability tools to monitor and debug the execution flow and runtime behavior of language model applications. The sys
Ships a telemetry data pipeline using REST-based HTTP requests to transmit model performance data to a centralized backend.
This project is a JavaScript error tracking SDK and application performance monitoring tool. It captures runtime exceptions and crashes across web browsers, server-side environments, and edge computing contexts. The SDK includes a session replay tool that records visual user interactions to reproduce bugs. To ensure telemetry delivery, it provides a tunneling proxy that routes monitoring data through custom endpoints to bypass browser-level ad blockers. The toolkit also features a source map processor that translates minified stack traces back into original source code. Additionally, it cove
Implements a telemetry pipeline to collect errors and performance spans for asynchronous transmission.
Podman Desktop is a graphical user interface for building, managing, and deploying containers and Kubernetes clusters from a local workstation. It serves as a container engine manager and a Kubernetes cluster dashboard, providing a visual environment for tasks typically handled via the command line. The project includes a container extension framework that allows users to integrate additional tools and capabilities into the management environment through a plugin system and extension catalog. The software covers the full container lifecycle, including image building and pushing to registries
Collects anonymous usage data and error reports asynchronously to monitor application stability without blocking the UI.
Side-Menu.Android es un componente de interfaz de usuario reutilizable para aplicaciones Android que proporciona un cajón de navegación deslizable. Está diseñado para ayudar a los desarrolladores a organizar secciones de la aplicación y opciones de usuario en un panel estructurado y oculto que mantiene una interfaz limpia para el área de contenido principal. El componente se distingue por su presentación visual, que sigue las directrices de Material Design para asegurar una experiencia de usuario consistente e intuitiva. Cuenta con una jerarquía de menú basada en datos que permite la agrupación lógica de elementos de navegación, e incorpora animaciones de revelación circular fluidas para proporcionar transiciones visuales pulidas cuando el menú se abre o se cierra. Al encapsular la lógica compleja de diseño e interacción en una sola clase modular, la librería simplifica la implementación de la navegación en múltiples pantallas. Soporta transiciones orientadas a eventos, permitiendo a los desarrolladores desacoplar las interacciones del menú de las actualizaciones de contenido para mantener una arquitectura de aplicación limpia y receptiva.
Ingests and processes high-volume telemetry data through streaming architectures for real-time analytics.
Diffusion Policy is a robot learning framework that uses diffusion models to map visual observations to precise action trajectories. It functions as an imitation learning toolkit and visuomotor policy learner, providing a system to train neural networks that replicate human behavior by generating robotic movements based on image and sensor data. The framework employs a conditional denoising process to sample sequences of robotic movements, allowing it to handle multimodal action distributions where multiple valid trajectories may exist for a single state. It utilizes score-based action modeli
Provides a telemetry data pipeline for batching and streaming high-resolution image and sensor data for training.
Riemann is a Clojure-based event stream processor and real-time analytics engine. It functions as a network telemetry pipeline and extensible event router that ingests, transforms, and routes event data from distributed systems. The system uses a domain-specific language to compute metrics and statistical patterns over continuous streams, enabling network trend analysis and real-time alerting. It supports dynamic plugin loading from the classpath and allows for live configuration reloading without interrupting active event streams. Capabilities include centralized telemetry aggregation, even
Implements a high-volume data pipeline for the ingestion, processing, and forwarding of real-time network telemetry.
This project provides a collection of extensions for the OpenTelemetry Go software development kit, serving as a toolkit for automated instrumentation, context propagation, and telemetry pipeline management. It enables the capture of traces, metrics, and logs across distributed systems by providing unified interfaces that decouple data collection from specific backend storage or analysis platforms. The project distinguishes itself through its ability to inject telemetry collection into popular frameworks and libraries without requiring manual source code modifications. It maintains request co
Routes telemetry signals through a series of configurable processors that filter, sample, and transform data before final export.