Awesome GitHub RepositoriesProcess Scaling

Huginn is a self-hosted automation platform that functions as an event-driven workflow engine. It allows users to build autonomous agents that monitor web services, scrape data, and execute complex tasks by propagating events through a directed graph. By running on your own server infrastructure, it provides a private environment for orchestrating workflows without relying on third-party automation services. The platform distinguishes itself through a modular, plugin-based architecture that enables the development of custom agents to handle specific data processing needs. Each agent maintains

This project is a Node.js process manager, runtime environment, and production deployment orchestrator. It provides the foundational system components required to run, monitor, and restart applications in the background to ensure continuous service availability. The system distinguishes itself through a built-in load balancer that distributes network traffic across multiple process instances to utilize all available CPU cores. It includes a real-time process monitor with a terminal-based dashboard for tracking server health, CPU and memory usage, and aggregated logs. The tool covers a broad

Discord.js is a Node.js library and framework for interacting with the Discord API. It provides a comprehensive set of wrappers for REST and WebSocket connections, enabling the development of automated server accounts and real-time chat applications. The project distinguishes itself through a distributed bot sharding system that splits a single bot instance across multiple processes to handle high server counts and large-scale workloads. It also includes a specialized voice API wrapper for managing audio streams and voice channel connectivity. The library covers broad capability areas includ

This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

ip2region is an offline IP geolocation library and framework designed to resolve IPv4 and IPv6 addresses to city-level regional information using local binary data files. It functions as a binary IP database compiler and a cross-language search client, allowing for regional lookups without relying on external APIs. The project distinguishes itself through a specialized binary format that supports high-performance query optimization. It employs adjacent-segment IP merging and deduplicated region storage to minimize the database footprint, while utilizing memory-mapped file caching and vector-i

uWebSockets is a high-performance networking engine providing an HTTP web server and a WebSocket server framework. It implements a multi-threaded event loop architecture to deploy isolated application instances across multiple CPU cores and includes an SSL/TLS network layer for secure, encrypted communication. The project features a dedicated WebSocket pub/sub engine for distributing messages to specific groups of connected clients. It optimizes network throughput through syscall corking to reduce kernel overhead and employs payload compression to minimize data transfer sizes. The system cov

MMKV is a high-performance, cross-platform key-value storage framework designed for mobile platforms and POSIX environments, including Android, iOS, macOS, and Windows. It provides a persistence layer that utilizes memory-mapped files and binary serialization to achieve low-latency data access. The project distinguishes itself through native support for multi-process synchronization, allowing concurrent read and write operations across different application processes. It also implements security via AES encryption for data at rest, featuring symmetric encryption and key rotation to protect st

Bull is a Node.js library for managing distributed jobs and message queues using Redis as the primary data store. It functions as a distributed task worker, job scheduler, and priority queue manager designed to handle asynchronous workloads across multiple processes. The project distinguishes itself by providing a persistent communication channel that decouples servers through the exchange of serializable data objects. It ensures distributed system reliability by detecting stalled tasks and recovering from process crashes to ensure every queued job is completed. The system covers a broad ran

Muduo is a C++11 event-driven network library and framework designed for building high-concurrency Linux servers. It provides a toolkit for implementing scalable network services and socket communication using non-blocking I/O and asynchronous event processing. The framework implements a multi-threaded TCP server architecture that distributes connection handling across multiple CPU threads to maximize server throughput. This is achieved through a one-loop-per-thread model and a reactor pattern implementation, which dispatch network events from a central demultiplexer to registered handler fun

NodeBB is a real-time, self-hosted community forum platform built on Node.js. It is designed to support scalable discussion environments by utilizing a document-oriented database for content storage and an in-memory engine for high-speed data retrieval and session management. The platform provides a comprehensive administrative interface for managing user groups, forum settings, and system health. What distinguishes the platform is its native support for federated social networking via the ActivityPub protocol, allowing forums to exchange content, synchronize discussions, and interact with de

Memcached is a high-performance, distributed, in-memory key-value storage and request routing engine. It functions as a volatile data store designed to accelerate dynamic applications by caching objects in RAM, thereby reducing backend database load and providing sub-millisecond response times. The system utilizes a specialized architecture that organizes memory into fixed-size slabs to minimize fragmentation and maximize throughput for high-concurrency workloads. The project distinguishes itself through a multi-threaded, lock-friendly design that scales across CPU cores and supports complex

Coturn is a network server that facilitates peer-to-peer media traffic for real-time communication applications. It functions as a relay platform for voice, video, and data transmission, enabling direct connections between clients located behind restrictive firewalls and network address translators. The server implements standard network traversal protocols to manage media packet exchange and client authentication. It utilizes a multi-threaded architecture and event-driven polling to handle high-throughput traffic, while employing hash-based message authentication codes to verify client ident

Fluentd is a unified logging layer and distributed event router that collects, parses, and routes log data from diverse sources to various storage backends. It functions as a log forwarding agent and pipeline orchestrator, transforming raw unstructured log strings into formatted objects using structured log parsing. The project utilizes a plugin-based pipeline architecture to route data through independent input, filter, and output stages. It differentiates itself through tag-based event routing, which uses regular expression patterns to direct specific data streams to their intended destinat

KeyDB is a multithreaded in-memory key-value store and distributed cache. It functions as a NoSQL database utilizing multi-version concurrency control to execute non-blocking queries and scans. The project is a multithreaded fork of Redis that maintains protocol compatibility while utilizing a multithreaded architecture to scale across multi-core hardware. It distinguishes itself with flash-tiered storage, allowing the system to offload data from primary RAM to SSD or flash storage to increase total capacity. The system supports high availability through active-active mesh replication and mu

DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono

Encore is a distributed systems framework designed to unify backend development, infrastructure provisioning, and observability. It functions as an infrastructure-as-code platform that allows developers to define cloud resources, databases, and messaging topics directly within their application code. By analyzing these declarations at compile-time, the system automatically manages the deployment of cloud resources and security policies, ensuring parity between local development and production environments. The platform distinguishes itself through its integrated development experience, which

FrankenPHP is a Go-based PHP runtime and application server that integrates a web server and PHP interpreter to host applications without requiring a separate process manager. It functions as a worker mode server that keeps applications in memory across requests to eliminate bootstrap overhead and a static binary bundler that packages applications and the server into a single self-contained executable. The project distinguishes itself by allowing the embedding of a PHP runtime directly into Go programs and enabling the development of PHP extensions using the Go language. It also includes a bu

Vue Storefront is a composable commerce platform designed to decouple the presentation layer from backend systems. By providing a headless frontend framework, it enables developers to build high-performance, mobile-first digital storefronts that remain independent of specific commerce engines, payment providers, or content management systems. The platform distinguishes itself through a modular architecture that uses standardized integration adapters to aggregate data from disparate services into a unified layer. This approach allows businesses to modernize legacy infrastructure or manage comp

ThreadPool is a C++ thread management library designed to execute asynchronous tasks using a fixed number of background worker threads. It functions as a concurrent task executor that reduces the overhead associated with the repeated creation and destruction of threads. The library utilizes a synchronized queue to distribute workloads across multiple CPU cores and employs variadic templates to accept any callable function with arbitrary arguments. The system manages concurrency through mutex-protected shared state and condition-variable signaling to wake idle threads. It also provides a mech