Awesome GitHub RepositoriesLoad Balancing

Bagisto is a modular e-commerce platform designed for building and managing scalable online storefronts. It functions as an enterprise-grade toolkit that provides a structured environment for handling complex inventory, secure payment processing, and shipping logistics. Built on a flexible architecture, the platform allows developers to extend core business logic through custom packages and specialized product types. The platform distinguishes itself through a highly modular design that utilizes service-provider-based dependency injection and a repository-pattern data access layer to decouple

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

Qbot is a multi-purpose platform designed to support automated recruitment, quantitative trading, and distributed service orchestration. It functions as a comprehensive framework that integrates artificial intelligence into specialized workflows, enabling users to build and deploy systems for candidate screening, financial strategy execution, and context-aware knowledge retrieval. The platform distinguishes itself through a modular architecture that combines high-performance distributed communication with domain-specific automation. It provides a robust foundation for managing microservices t

kops is a Kubernetes cluster provisioner and lifecycle manager designed to automate the creation, maintenance, and destruction of production-grade clusters on cloud infrastructure. It functions as a declarative infrastructure manager, synchronizing the live state of a cluster with versioned manifests stored in remote object storage to ensure idempotent operations. The project distinguishes itself by offering comprehensive automation for the entire cluster lifecycle, including high-availability control plane deployment, incremental rolling updates, and automated version upgrades. It also serve

This project is a web-based user interface and multi-model API gateway for interacting with various large language model providers and local inference services. It functions as a retrieval-augmented generation chatbot for private document questioning, a manager for model fine-tuning, and an autonomous agent framework. The system distinguishes itself by integrating an autonomous assistant mode that uses web search and external tools to solve complex, multi-step tasks without manual prompting. It also features an API gateway capable of rotating multiple authentication keys to balance usage and

Thanos is a CNCF cloud native monitoring tool that provides a highly available and scalable extension to the Prometheus ecosystem. It functions as a global query engine, a long-term storage system, and a metric downsampler. The project enables a unified interface to aggregate and query metrics across multiple distributed clusters from a single view. It maintains historical data beyond local retention limits by persisting time-series metrics in object storage and eliminates data gaps by merging metrics from redundant server pairs. The system includes capabilities for reducing the resolution o

Dask est un framework de calcul parallèle et un planificateur de tâches distribué conçu pour mettre à l'échelle les flux de travail de science des données Python, des machines uniques aux grands clusters. Il fonctionne comme un gestionnaire de ressources de cluster qui orchestre la logique computationnelle en représentant les tâches et leurs dépendances sous forme de graphes acycliques dirigés. Cette architecture permet au système d'automatiser la distribution des charges de travail sur le matériel disponible tout en gérant des exigences d'exécution complexes. Le projet se distingue par un moteur d'évaluation paresseuse qui diffère les opérations sur les données jusqu'à ce qu'elles soient explicitement demandées, permettant une optimisation globale du graphe et une allocation efficace des ressources. Il intègre le déversement de données conscient de la mémoire pour éviter les plantages du système lors du traitement de jeux de données dépassant la mémoire disponible, et il utilise la fusion de graphes de tâches pour combiner des séquences d'opérations en étapes d'exécution uniques, minimisant la surcharge de planification et la communication entre nœuds. La plateforme fournit une surface de capacités complète pour l'analyse de données à grande échelle, incluant le support pour l'apprentissage automatique distribué, l'intégration du calcul haute performance et le traitement de données parallèle. Elle offre des outils étendus pour la gestion du cycle de vie des clusters, le profilage des performances et la surveillance en temps réel de l'exécution des tâches. Les utilisateurs peuvent déployer ces environnements sur diverses infrastructures, incluant le matériel local, les fournisseurs cloud, les systèmes conteneurisés et les clusters de calcul haute performance.

Rayon is a data parallelism library for Rust that provides a framework for converting sequential computations into parallel operations. It enables the transformation of standard data structures and loops into parallel iterators, allowing workloads to be distributed across multiple processor cores. By utilizing a work-stealing scheduler, the library dynamically balances tasks to maximize throughput and minimize execution time. The library distinguishes itself through its focus on safe, scoped task synchronization, which ensures that all spawned operations complete before a scope exits to preve

Taskflow is a C++ task-parallel framework designed to build high-performance parallel workflows and complex dependency graphs. It provides a programming model that organizes computational work into directed acyclic graphs, enabling developers to manage concurrency, resource scheduling, and task dependencies across multi-core CPUs and GPU accelerators. The framework distinguishes itself through its ability to orchestrate heterogeneous systems, allowing for the integration of hardware-accelerated kernels and memory operations into unified execution pipelines. It supports dynamic runtime subflow

Kafka Manager is a web-based management interface and monitoring tool for Apache Kafka clusters. It serves as a central control plane for topic administration, consumer monitoring, and cluster health inspection. The project provides specialized utilities for data rebalancing and partition reassignment to distribute workloads across brokers. It also includes tools to optimize partition leadership by electing preferred replicas. The platform covers a broad range of administrative capabilities, including the creation and configuration of message topics, tracking of consumer offsets, and the col

BunkerWeb is a containerized suite of infrastructure tools that functions as a cloud-native web application firewall and Nginx reverse proxy. It provides a security layer for web applications, combining traffic routing with automated SSL certificate management and a web-based security dashboard for monitoring and configuration. The project distinguishes itself through its deep integration with container orchestrators, serving as a Kubernetes ingress controller that automates security settings and service discovery via container labels. It features a plugin-based extension model and a manageme

KEDA is a Kubernetes event-driven autoscaler and cloud event scaling engine. It functions as a custom metrics provider that monitors external event sources—including message brokers, databases, and cloud metrics—to dynamically adjust the replica counts of containerized workloads. The project is distinguished by its scale-to-zero workflow, which reduces workloads to zero replicas during inactivity and automatically restarts them when new events are detected. It operates as a multi-cloud event trigger system, using a pluggable scaler interface to integrate with a wide array of third-party servi

AutoMQ is a cloud-native streaming platform and Apache Kafka distribution that implements a decoupled compute and storage architecture. It functions as an S3-backed message queue, using object storage as the primary log repository to eliminate dependencies on local disks. The platform utilizes a stateless broker architecture to enable dynamic compute scaling and automated partition balancing. This design allows the system to adjust the number of brokers in seconds and distribute network traffic without requiring manual data migration or partition reassignment. The system provides multi-avail

AutoMQ is a cloud-native streaming platform and Kafka-compatible message broker. It implements the Kafka protocol to provide integration with existing clients and ecosystems while functioning as a message queue that persists data directly to cloud object storage. The system decouples compute from storage, allowing processing power and storage capacity to scale independently. It utilizes a shared-log architecture and object-storage-based persistence to remove dependencies on local disks, which reduces operational costs and eliminates manual disk management. The platform includes mechanisms fo

OceanBase is a distributed SQL database designed for high availability and strong consistency across multiple nodes and regions. It functions as a hybrid transactional and analytical processing engine, allowing real-time analytics and transactions to execute on a single data copy. The system also serves as a vector database engine for indexing and querying vector data to power semantic search and recommendation systems. The platform features native compatibility layers for MySQL and Oracle, enabling the migration of legacy workloads without rewriting SQL code. It utilizes a Paxos-based distri

Yarp is a .NET HTTP reverse proxy toolkit and framework used to build applications that route network traffic to backend servers. It provides a customizable routing engine for directing incoming requests to specific destination targets based on defined matching patterns. The project enables the implementation of API gateways and reverse proxies by providing tools for dynamic request routing and custom load balancing across pools of backend servers. The toolkit covers traffic management capabilities including cluster-based load balancing, configuration-driven route matching, and support for H

FastDFS is a distributed file system and object store designed as a high-capacity file server. It functions as a cluster storage manager that saves, syncs, and accesses large volumes of unstructured data across a network of distributed servers. The system uses unique identifiers for file retrieval and indexing instead of traditional hierarchical naming to avoid metadata bottlenecks. It manages file attributes through key-value metadata mapping and employs a distributed replication model to ensure high availability and data redundancy across storage groups. The project provides capabilities f

xmall is a distributed e-commerce platform based on a service-oriented architecture. It separates business logic into independent services that communicate over a network to ensure scalability and fault tolerance, utilizing a decoupled storefront interface for customer transactions. The platform employs a distributed architecture using Dubbo for service orchestration and Zookeeper for cluster coordination and service discovery. It integrates a specialized set of components including an asynchronous message broker for background tasks, an indexed search system for product catalogs, and a centr

oneAPI Threading Building Blocks (oneTBB)

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis