Benthos is a declarative stream processor and data integration pipeline used to route, transform, and filter information between disparate services. It functions as an at-least-once message broker and change data capture engine, using a transaction model to guarantee message delivery despite system crashes or server faults.
الميزات الرئيسية لـ jeffail/benthos هي: Data Streaming, Change Data Capture, Data Pipeline Configurations, Data Stream Processors, Message Brokers, Stream Transformations, At-Least-Once Delivery Guarantees, Data Mapping Languages.
تشمل البدائل مفتوحة المصدر لـ jeffail/benthos: redpanda-data/connect — Connect is a Kafka data integration platform and stream processing engine used to build declarative pipelines that… benthosdev/benthos — Benthos is a stream processing engine and data integration pipeline used for routing, transforming, and connecting… vectordotdev/vector — Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and… vonng/ddia — This project serves as a comprehensive technical reference for the architecture and design of data-intensive… uptrace/uptrace — Uptrace is an OpenTelemetry-based observability platform designed to collect, store, and analyze distributed traces,… risingwavelabs/risingwave — RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process…
Connect is a Kafka data integration platform and stream processing engine used to build declarative pipelines that move and transform messages between Kafka topics and external sources. It functions as a Kafka Connect framework and a change data capture tool, streaming real-time database modifications to synchronize data across distributed environments. The project differentiates itself through a dedicated mapping language for mutating and reshaping message payloads and the ability to execute custom processing logic within a sandboxed WebAssembly runtime. It also provides an observability pip
Benthos is a stream processing engine and data integration pipeline used for routing, transforming, and connecting data streams between diverse sources and sinks. It functions as event routing middleware and a change data capture tool, streaming real-time database modifications as discrete events for downstream processing. The system utilizes a declarative pipeline configuration, where data flow and processing logic are defined in a single static file. It features a specialized domain-specific language for mapping, filtering, and enriching data payloads, allowing for complex transformations w
Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network
This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi