Awesome GitHub RepositoriesBackpressure Controllers

RxJava is a reactive stream processing framework and JVM reactive extensions library. It serves as an asynchronous dataflow orchestrator used to compose event-based programs by transforming, combining, and consuming real-time data flows on the Java Virtual Machine. The project distinguishes itself through integrated backpressure flow control, which manages the emission rate between producers and consumers to prevent memory exhaustion. It further provides mechanisms for concurrent thread management and parallel data processing to offload blocking operations and maintain application responsiven

RxJS is a library for reactive programming that provides a framework for composing asynchronous and event-based programs. It utilizes observable sequences to model data flows, allowing developers to manage complex sequences of events through a declarative programming interface. The library implements the observer pattern to facilitate decoupled communication between data producers and subscribers. By employing a lazy execution model, streams remain dormant until a consumer explicitly subscribes, at which point data production is triggered. This approach enables the construction of predictable

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

xterm.js is a high-performance terminal emulator library designed for web applications. It provides a core rendering engine and a modular interface that allows developers to embed fully functional, interactive command-line interfaces directly into browser environments. By processing standard terminal data streams and managing internal buffer states, the library enables the creation of rich, text-based user interfaces that support standard terminal protocols. The project distinguishes itself through a highly extensible architecture that allows for deep customization of terminal behavior. Devel

The Reactive Extensions for JavaScript

uWebSockets is a high-performance networking engine providing an HTTP web server and a WebSocket server framework. It implements a multi-threaded event loop architecture to deploy isolated application instances across multiple CPU cores and includes an SSL/TLS network layer for secure, encrypted communication. The project features a dedicated WebSocket pub/sub engine for distributing messages to specific groups of connected clients. It optimizes network throughput through syscall corking to reduce kernel overhead and employs payload compression to minimize data transfer sizes. The system cov

Quarkus is a Kubernetes-native Java framework designed for building high-performance, memory-efficient applications. It utilizes ahead-of-time native compilation to transform Java code into standalone, optimized binaries that eliminate the need for a virtual machine, enabling rapid startup and reduced memory consumption. By performing code augmentation during the build phase, it shifts heavy processing tasks away from runtime, ensuring that applications are optimized for cloud-native environments. The framework distinguishes itself through a unified approach to reactive and imperative program

Apache Pulsar is a cloud-native distributed pub-sub messaging system designed for high-performance data ingestion. It functions as a geo-replicated data streamer and a multi-tenant event streaming platform, providing a serverless stream processing engine and a tiered storage messaging broker. The system distinguishes itself by separating serving layers from storage layers to allow independent scaling of compute and data retention. It features native geo-replication to synchronize messages across different geographical regions and employs a multi-layered tenant isolation model using authentica

Beats is a collection of lightweight, modular agents designed to gather, process, and forward operational telemetry from distributed infrastructure to centralized storage and analysis platforms. These agents function as a distributed data transport layer, decoupling the collection of logs, metrics, and network events from their final delivery destination. By maintaining local state and managing data flow, the system ensures reliable transmission of information across heterogeneous environments. The project distinguishes itself through a modular pipeline architecture that allows for the assemb

This project provides educational materials and courseware focused on the theoretical and practical foundations of distributed systems design. It serves as a comprehensive curriculum covering the disciplines of consensus, data consistency, reliability engineering, and scalability. The instructional content focuses on achieving cluster agreement through consensus algorithms and managing system-wide state via coordination frameworks. It includes a dedicated guide to data theory, exploring replication strategies, consistency models, and data convergence. The courseware covers a broad capability

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

CAP is a .NET distributed transaction framework and event bus designed to manage asynchronous communication in microservices. It implements the outbox pattern to ensure eventual consistency and reliable message delivery by persisting messages in local database tables until transactions commit. The framework includes a distributed message monitor and web dashboard for tracking the status of sent and received messages. It provides tools for event traffic visualization, distributed request tracing, and the ability to manually trigger retries for failed delivery attempts. The system supports var

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

Highly scalable realtime pub/sub and RPC framework

Reactor Core es un kit de herramientas de programación reactiva y una base no bloqueante para componer pipelines de datos asíncronos en la JVM. Sirve como framework de procesamiento de flujos asíncronos y sistema de gestión de contrapresión (backpressure), permitiendo a los desarrolladores transformar, filtrar y combinar secuencias de eventos mientras regulan el flujo de datos entre productores y consumidores para evitar el agotamiento de recursos. La biblioteca se diferencia por un sofisticado sistema de planificación de concurrencia y control de flujo basado en la demanda. Desacopla el procesamiento de señales de hilos específicos utilizando un registro de planificadores y proporciona mecanismos para la propagación de metadatos inmutables conscientes del contexto a través de límites asíncronos. También cuenta con herramientas especializadas para la captura de trazas en tiempo de ensamblaje y planificación de tiempo virtual para facilitar la prueba de operadores basados en el tiempo. El proyecto cubre una amplia gama de capacidades, incluyendo procesamiento funcional de datos para agregación y ventanas de secuencias, una variedad de estrategias de recuperación de errores como reintentos con retroceso exponencial y utilidades para conectar API de callback heredadas o síncronas en flujos reactivos. Además, proporciona instrumentación para el monitoreo de pipelines y un conjunto de herramientas de prueba para verificar secuencias de señales.

This is a cross-platform media processing library that reads, writes, encodes, and decodes media in both browser and server environments. It supports common container formats including ISOBMFF, Matroska, Ogg, MPEG-TS, and HLS, and handles codec operations through a combination of WebCodecs API and WebAssembly-based encoders. Media is processed in streaming pipelines that maintain constant memory usage and automatically apply backpressure from output speed to all upstream components. The library distinguishes itself through a plugin-based codec registration system that allows extending support

Piscina is a Node.js worker thread pool that runs CPU-intensive JavaScript functions across multiple threads for parallel execution. It manages a dynamic pool of worker threads with configurable size, handling task submission, cancellation, and lifecycle management through a promise-based interface. The pool supports AbortController-based task cancellation, enabling clean termination of submitted or running tasks without disrupting other work. It enforces per-worker memory limits through V8 resource caps and applies backpressure with a configurable maximum queue size that emits a drain event

Este proyecto proporciona una especificación formal y un conjunto de interfaces estándar de Java para el procesamiento de flujos asíncronos. Define un protocolo estandarizado para pasar secuencias de elementos entre editores (publishers) y suscriptores a través de diferentes hilos, centrándose en una especificación de flujos reactivos para la JVM. El proyecto se centra en la interoperabilidad al proporcionar una API común que permite que diferentes bibliotecas de streaming asíncrono trabajen juntas. Esto se logra mediante un conjunto estándar de interfaces y mecanismos de puente que traducen entre especificaciones de streaming incompatibles. La especificación cubre un protocolo de contrapresión (backpressure) sin bloqueo para regular el flujo de datos y evitar la sobrecarga del sistema, requiriendo que los suscriptores señalen la demanda. También define el ciclo de vida de los flujos, incluyendo la gestión de suscripciones, el procesamiento de elementos y la terminación basada en señales para la limpieza de recursos. El proyecto incluye un framework para verificar el comportamiento del flujo y validar la lógica de procesamiento frente a las reglas de contrapresión y eventos asíncronos.

Este proyecto es una colección de directrices y manuales curados para escribir código limpio, idiomático y mantenible en Scala. Sirve como una guía completa para los estándares de codificación de Scala, diseño de programación funcional y arquitectura de software empresarial. El repositorio proporciona estrategias específicas para la gestión de la concurrencia, incluyendo patrones para actores, futures y pools de hilos para garantizar la seguridad de los hilos. También contiene un manual de optimización del rendimiento centrado en reducir las asignaciones de memoria y gestionar la presión del recolector de basura para mejorar la eficiencia en tiempo de ejecución. Las guías cubren una amplia gama de capacidades, incluyendo arquitectura de aplicaciones, manejo de errores con seguridad de tipos y el uso de estructuras de datos inmutables. También aborda el aseguramiento de la calidad del software mediante convenciones de nomenclatura estandarizadas, diseño modular de traits y la implementación de contrapresión (back-pressure) y señalización de demanda.

Reactor is a reactive streams library and framework for building asynchronous data pipelines. It provides a system for coordinating execution contexts via an asynchronous event-loop manager, alongside a set of reactive abstractions for implementing high-performance TCP, UDP, QUIC, and HTTP services. The project includes a specialized testing tool for verifying the timing and order of asynchronous data flows and a bill of materials to synchronize compatible versions of core reactive libraries and networking add-ons. Its capability surface covers non-blocking network services, demand-based bac