Awesome GitHub RepositoriesStreaming Libraries

MediaPipe is a cross-platform machine learning framework designed for building and deploying pipelines that process live and streaming media. It provides a system for connecting processing components into custom machine learning chains to analyze real-time audio and video streams. The framework includes a suite of pre-trained models for tasks such as hand, face, and pose tracking, along with tools for retraining and customizing these models with specific datasets. It also features a dedicated benchmarker for measuring the execution speed and accuracy of machine learning models directly within

Kafka is a distributed event streaming platform designed for capturing, storing, and processing real-time data streams across interconnected nodes. It functions as a distributed commit log, providing a fault-tolerant storage mechanism that records state changes sequentially to ensure data consistency and durability across distributed environments. The platform distinguishes itself through a partitioned commit log architecture that enables horizontal scaling and parallel processing of data streams. It integrates a stream processing engine for continuous transformations and aggregations, while

The Reactive Extensions for JavaScript

Akka is an actor model framework and distributed systems platform used to build concurrent and distributed applications. It provides a toolkit for managing multi-threaded state and behavior through asynchronous message passing, allowing developers to create concurrent applications without manual locks or synchronization. The system functions as a cluster management and event sourcing framework, automating the scaling and coordination of high-availability clusters. It enables the deployment of elastic services that coordinate workloads across multiple network nodes and ensures fault tolerance

Benthos is a declarative stream processor and data integration pipeline used to route, transform, and filter information between disparate services. It functions as an at-least-once message broker and change data capture engine, using a transaction model to guarantee message delivery despite system crashes or server faults. The system is defined by an observability-first approach, featuring built-in HTTP health probes, performance metrics export, and distributed request flow tracing. It utilizes a plugin architecture that allows the core engine to be extended with custom binaries for new inpu

FastStream is an asynchronous Python framework designed for building event-driven microservices. It provides a unified abstraction layer for interacting with various message brokers, enabling developers to manage event production and consumption through a consistent interface while maintaining access to native provider-specific features. The framework centers on a decorator-based routing model that binds application logic directly to broker topics, supported by a built-in dependency injection container that resolves resources at runtime. The framework distinguishes itself through its deep int

RxGo is a functional reactive programming library and an implementation of ReactiveX for the Go language. It serves as an asynchronous stream processing toolkit designed to coordinate event-based programs and data flows using the observable pattern. The library enables the construction of asynchronous processing pipelines that transform, filter, and combine event sequences. It distinguishes itself through the use of functional operators to compose these pipelines and provides mechanisms for managing concurrent execution. The toolkit covers a broad range of stream orchestration capabilities,

StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.

Compositional, streaming I/O library for Scala

Asynchronous, Reactive Programming for Scala and Scala.js.

🦖 Serverless AI Agent Framework with Geo-distributed Edge AI Infra.

Python Streaming DataFrames for Kafka

Real-time stream processing for python

.NET Stream Processing Library for Apache Kafka 🚀

Streaming reactive and dataflow graphs in Python

Substation is a toolkit for routing, normalizing, and enriching security event and audit logs.

A lightweight Reactive Streams Infrastructure Toolkit for Scala.

StreamLine - Streaming Analytics

Daggy - Data Aggregation Utility and C/C++ developer library for data streams catching

Stream Ops is a fully embeddable data streaming engine and stream processing API for Java.