Awesome GitHub RepositoriesLoad Balancing Architectures

This project is a comprehensive educational resource focused on the principles, patterns, and trade-offs required to design scalable, reliable, and high-performance distributed systems. It provides a structured curriculum that covers the fundamental architectural strategies necessary for building modern software infrastructure, ranging from high-level system decomposition to low-level networking and data management. The repository distinguishes itself by offering deep dives into complex architectural patterns, such as microservices-based decomposition, event-driven communication, and command-

JavaFamily is a curated set of learning paths and reference guides for backend engineering, distributed systems, and virtual machine internals. It provides a structured curriculum covering the Java language, operating system concepts, and network protocols. The project features detailed study guides for the Java virtual machine architecture, including memory management and garbage collection. It also includes a comprehensive reference for distributed systems, covering microservices, remote procedure call frameworks, and scalable system design. The collection covers a broad range of technical

xxl-job is a distributed task scheduling platform and job orchestrator designed to manage and trigger timed jobs across a cluster of remote executor nodes. It provides a centralized system for scheduling tasks, linking dependent jobs, and managing complex execution lifecycles through a relational database that persists configurations and logs. The platform distinguishes itself through a web-based interface for cron job management, allowing users to create and update scheduled tasks without modifying source code. It supports cross-language task execution by triggering logic on third-party exec

TiKV is a cloud-native distributed transactional key-value store and storage engine. It provides a distributed database designed for horizontal scalability and strong consistency across a cluster of physical nodes. The system uses a Raft-based consensus mechanism to maintain data availability and state synchronization. It ensures ACID compliance for distributed transactions through a two-phase commit workflow and manages data distribution via multi-Raft sharding. The engine handles massive datasets using automated range splitting and cluster load balancing to distribute data across different

Eureka is a service registry and discovery server that serves as a coordination tool for microservices. It provides a distributed service registry where instances record their network locations and health status to enable dynamic communication without the use of hardcoded addresses. The system functions as a client-side load balancing framework, providing location data to consumers so they can distribute network traffic across available instances. It employs a heartbeat-based health monitor to track instance availability and automatically removes unresponsive nodes from the registry to manage

Buildkit is a programmable container build toolkit and OCI container image builder that converts build definitions into concurrent dependency graphs for image construction. It functions as an OCI image distribution engine, capable of generating container images and exporting artifacts to local storage or remote registries. The project is distinguished by its use of a low-level binary intermediate representation to decouple high-level build languages from the execution engine. It supports multi-platform image builds through user-mode architecture emulation and provides a distributed build cach

ET is a C# game server framework and distributed actor model runtime designed for large-scale multiplayer environments. It provides a comprehensive toolkit for building distributed game backends, incorporating a multiplayer network transport layer and a specialized suite for game AI and pathfinding. The framework is distinguished by its use of a distributed actor model to scale processing across multiple threads and servers, utilizing isolated actors for state management and messaging. It features a unified codebase architecture that allows shared logic between the server and client, enabling

Finagle is a distributed service mesh and fault-tolerant remote procedure call framework. It provides a protocol-agnostic network library that implements a consistent interface for different network standards, including HTTP and Thrift. The project distinguishes itself by integrating a fault tolerance library that prevents cascading failures through circuit breaking and timeout management. It also implements a distributed tracing system to track requests across network boundaries and visualize call graphs. The framework covers several core capability areas, including dynamic service discover

Storm is a distributed stream processing framework and fault-tolerant compute engine designed for executing real-time continuous computations across a cluster of machines. It functions as a stateful stream processor and cluster topology manager, enabling the deployment and monitoring of distributed data flow configurations. The system ensures exactly-once semantics by utilizing transactional state management to guarantee that every message in a data stream is processed exactly one time. It further operates as a distributed RPC system, allowing for the integration of non-native languages throu

rpcx is a high-performance remote procedure call framework for building scalable microservices in Go. It functions as a binary protocol RPC system and a service mesh, providing the necessary infrastructure for low-latency inter-service communication and distributed cloud environments. The project features a cross-language service gateway that provides an HTTP entry point, allowing clients written in any programming language to invoke Go remote services via protocol translation. It also includes a specialized RPC traffic analyzer for capturing and analyzing binary packets to debug network comm

dockerlabs is a collection of educational labs and technical tutorials designed to teach the fundamentals of containerization and microservice architecture. It provides instructional material and hands-on exercises covering image optimization, security training, infrastructure setup, and cluster orchestration. The project features specific courses and guides focused on reducing image size through multi-stage builds, securing workloads via vulnerability scanning and encrypted networks, and deploying multi-node clusters with high availability using Swarm orchestration. The materials cover a br

Quartz.NET is a job scheduler for .NET applications designed to schedule and execute programmatic tasks. It functions as a distributed task orchestrator and enterprise task orchestrator, capable of managing recurring jobs with concurrency limits and complex intervals. The system provides high availability through a clustered execution model that balances loads and provides fail-over redundancy across multiple server instances. It utilizes a relational database job store to persist job and trigger states, ensuring that scheduled tasks survive application restarts. The framework includes capab

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

DataX Web is a web-based management platform for scheduling, building, executing, and monitoring distributed data synchronization jobs powered by DataX. It provides a visual console for creating and managing DataX tasks without manual JSON configuration, with a distributed executor cluster that auto-registers worker nodes and supports configurable routing and blocking strategies for task distribution. The platform offers cron-based task scheduling with dynamic start, stop, and immediate status changes, along with incremental sync capabilities that pass dynamic parameters to extract only new o

protoactor-go is a framework for building concurrent and distributed systems in Go using the actor model. It provides a distributed actor system that enables isolated entities to communicate via asynchronous messaging and share state across a cluster. The framework implements a multi-language actor protocol, allowing interoperability between actors written in Go, C#, and Java. It further supports a virtual actor implementation, where actors are automatically instantiated across a network based on a unique identity. The system includes a supervision model for managing actor lifecycles and fau

SparkInternals est une référence technique et un guide d'architecture détaillant la conception interne et l'implémentation du moteur de calcul distribué Apache Spark. Il sert d'étude sur l'analyse des moteurs de big data, en se concentrant sur la gestion de l'exécution en cluster et l'interaction entre les nœuds drivers, les exécuteurs et les workers. Le projet fournit une décomposition détaillée de la manière dont les plans logiques sont convertis en étapes d'exécution physiques. Il analyse spécifiquement la mécanique des opérations de shuffle, la gestion de la mémoire et la coordination de la planification des jobs distribués. La documentation couvre un large éventail de capacités de calcul distribué, incluant la planification de l'exécution des requêtes, la gestion des dépendances de données et les stratégies de mise en cache en mémoire. Elle examine également la distribution des tâches, l'exécution parallèle et les processus utilisés pour la reprise sur erreur et la persistance des données.

Akka.NET is an actor model framework used for building concurrent and distributed applications. It functions as a distributed computing platform and state manager that enables isolated actors to communicate via asynchronous message passing, ensuring thread-safe state management without manual locks. The project is distinguished by its decentralized coordination capabilities, including a distributed state manager that uses sharding and dynamic rebalancing to maintain high availability. It incorporates an event sourcing engine that persists state as a sequence of events in an append-only log an

YDB est une base de données SQL distribuée et un moteur analytique conçu pour la scalabilité horizontale et une forte cohérence. Il fonctionne comme un système multi-modèle qui prend en charge les charges de travail transactionnelles et analytiques via une architecture distribuée fournissant des transactions ACID sérialisables. Le système se distingue par sa large compatibilité de protocole, implémentant le protocole wire PostgreSQL pour les pilotes SQL standard et le protocole Kafka pour la messagerie et le streaming. Il sert en outre de base de données vectorielle, prenant en charge les index vectoriels et les recherches de voisins les plus proches approximatifs pour la recherche sémantique et les embeddings. La plateforme gère les données en utilisant un modèle de stockage hybride avec des formats orientés lignes et orientés colonnes, utilisant l'exécution de requêtes vectorisées pour des analyses à l'échelle du pétaoctet. Sa surface opérationnelle inclut le streaming de capture de données modifiées (CDC), des files d'attente persistantes avec garantie d'exécution unique (exactly-once) et une haute disponibilité multi-zone. Le déploiement et la gestion du cycle de vie sont pris en charge via un opérateur Kubernetes et le provisionnement d'infrastructure as code.

InternetArchitect est une collection éducative de documents et de code source conçue comme un cours d'architecture à haute concurrence. Il sert de guide d'implémentation de systèmes distribués, fournissant des patterns techniques et des exemples pratiques pour concevoir des architectures internet scalables qui maintiennent la stabilité sous de lourdes charges de trafic. Le projet se concentre sur l'optimisation des bases de données haute performance et les patterns de conception de microservices. Il couvre des stratégies pour réduire la latence et augmenter le débit via le sharding de base de données et les couches de proxy, ainsi que la coordination de l'état global à travers des clusters distribués. La portée architecturale inclut des stratégies de mise en cache multi-niveaux pour accélérer la récupération des données et l'implémentation de frameworks de découverte de services pour gérer la communication entre les microservices découplés. Il aborde également la coordination d'état distribué et l'utilisation de maillages d'équilibrage de charge pour distribuer le trafic réseau à travers les serveurs backend.

Mgmt est un système de gestion de configuration distribué qui maintient l'état souhaité des clusters en utilisant l'automatisation pilotée par les événements et le feedback en boucle fermée. Il fonctionne comme un moteur d'automatisation d'infrastructure qui déclenche des corrections d'état système en temps réel basées sur la surveillance des ressources et des spécifications prédéfinies. Le système inclut un sélecteur de nœuds de cluster distribué pour élire des sous-ensembles d'hôtes basés sur des stratégies et contraintes spécifiques afin de distribuer les charges de travail. Il dispose également d'un gestionnaire d'infrastructure cloud pour contrôler le cycle de vie des instances de machine virtuelle, incluant le déploiement d'images, la sélection de région et les scripts de démarrage, aux côtés d'un orchestrateur de firmware matériel pour installer et vérifier les binaires sur les contrôleurs de gestion de carte mère. Des capacités supplémentaires couvrent l'exécution de tâches en parallèle, la modification de fichiers de configuration basée sur des lentilles et la récupération de métadonnées système. Le toolkit fournit également des utilitaires pour la transformation de types de données, le décodage de format et l'inspection d'environnement pour déterminer les états actuels du système.