Benthos is a stream processing engine and data integration pipeline used for routing, transforming, and connecting data streams between diverse sources and sinks. It functions as event routing middleware and a change data capture tool, streaming real-time database modifications as discrete events for downstream processing.
The system utilizes a declarative pipeline configuration, where data flow and processing logic are defined in a single static file. It features a specialized domain-specific language for mapping, filtering, and enriching data payloads, allowing for complex transformations without custom code.
The platform provides an observability-driven data plane with integrated telemetry, performance metrics, and message flow tracing. Reliability is managed through a transaction model that ensures at-least-once delivery guarantees to prevent data loss during system crashes.
The engine is extensible through a plugin architecture that supports loading external binary modules to add new source and sink connectors.