Open-source alternatives to Heka

Rudder Server is a customer data platform and event routing pipeline designed to collect, transform, and route customer event data from various sources to data warehouses and business tools. It functions as a customer identity resolver, linking identifiers from multiple sources to build a unified identity graph and comprehensive behavioral customer profiles. The system differentiates itself through reverse ETL capabilities, which push processed customer segments and audiences from data warehouses back into operational third-party applications. It also provides a containerized data plane for K

Secor is a service implementing Kafka log persistence

Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.

Build platforms that flexibly mix SQL, batch, and stream processing paradigms

ingestr is a command-line tool for copying and syncing data between different database engines and third-party platforms without writing custom code. It functions as an ETL pipeline utility that extracts data from diverse sources and loads it into destinations. The tool features a schema-agnostic data loader that maps source fields to destination columns dynamically, removing the need for predefined static table definitions. It also operates as an incremental data synchronizer, updating destination tables by appending new records or merging changes to maintain current datasets. The system pr

Scribe is a distributed log aggregation system designed to collect and route real-time log data from numerous servers to centralized storage or analysis tools. It functions as a log data pipeline and scalable collector that gathers streaming data and writes it to local disks or remote endpoints. The system employs a log routing server model that organizes incoming streams into specific buckets based on predefined configuration mappings. It supports multi-hop log forwarding, allowing data to be routed through a chain of intermediate servers to centralize logs from diverse network segments. Re

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

A probabilistic data structure service and storage

DocId set compression and set operation library

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

Iggy is a distributed message streaming platform and multi-protocol message broker that functions as a persistent distributed log store. It provides infrastructure for publishing and consuming binary messages using an append-only log, ensuring high availability and data consistency across nodes through Viewstamped Replication. The platform is distinguished by its specialized LLM streaming infrastructure, which uses a server protocol to connect large language models to streaming data and system controls. This includes standardized protocols for context management and data bridging via HTTP or

RemoteWeather is a professional weather monitoring system that collects data from your weather station, stores it for historical analysis, and shares it through multiple channels including a beautiful live website, weather services, and amateur radio networks.

Gatling is a load testing framework and traffic generation engine used to measure response times and error rates under heavy load. It functions as an as-code testing library, allowing users to define high-volume traffic simulations and performance tests through programming languages rather than graphical interfaces. The system enables multi-language load simulation and the ability to model concurrent user traffic to identify infrastructure bottlenecks and stability limits. It supports a test-as-code workflow, where version-controlled scripts are integrated into build pipelines as performance

Open source part of the centreon monorepoc

crankshaftd

This program interfaces ham radio APRS-IS servers and saves packet data into an influxdb database. aprs2influxdb handles the connection, parsing, and saving of data into an influxdb database from APRS-IS using line protocol formatted strings. Periodically, a status message is also sent to the…

aggregateD

Charmander Scheduler Lab

statsd-jvm-profiler is a JVM agent profiler that sends profiling data to StatsD. Inspired by riemann-jvm-profiler, it was primarily built for profiling Hadoop jobs, but can be used with any JVM process.

Live import all your Google Sheets to your data warehouse

The Apache Kafka C/C++ library

Kafkacat is a suite of command-line utilities for interacting with Apache Kafka clusters. It provides a non-JVM binary for producing and consuming messages, inspecting cluster metadata, and debugging the Kafka protocol via the terminal. The tool functions as a producer and consumer capable of pushing data from files or standard input and reading messages from specific topics and partitions. It includes a metadata inspector to retrieve cluster state and partition configurations in plain text or JSON, as well as a protocol debugger for inspecting message offsets, timestamps, and binary payloads

cAdvisor is a container resource monitoring agent and performance analyzer that collects and exports CPU, memory, network, and disk usage statistics from running containers. It functions as a telemetry tool for discovering containers across various runtimes and serves as a Prometheus-compatible metrics exporter. The agent distinguishes itself by analyzing Linux control groups to provide visibility into resource consumption and limits. It utilizes kernel perf events and NUMA statistics for low-level hardware performance tracking and diagnostics, and it can identify out-of-memory kill events th

Data ingestion engine — Singer taps to Parquet in cloud buckets