Awesome GitHub RepositoriesShared Caching Topologies

This project is a comprehensive educational resource focused on the principles, patterns, and trade-offs required to design scalable, reliable, and high-performance distributed systems. It provides a structured curriculum that covers the fundamental architectural strategies necessary for building modern software infrastructure, ranging from high-level system decomposition to low-level networking and data management. The repository distinguishes itself by offering deep dives into complex architectural patterns, such as microservices-based decomposition, event-driven communication, and command-

This project is a build orchestration engine and development toolkit designed for managing large-scale monorepos. It provides a unified workspace environment that maps project relationships and dependencies, enabling the system to perform intelligent impact analysis and execute only the tasks affected by specific code changes. The system distinguishes itself through a persistent daemon that monitors file changes for near-instant feedback and a content-addressable caching mechanism that stores task outputs to prevent redundant computation across local and remote environments. It further suppor

Fasthttp is a high-performance networking framework for Go, designed to maximize throughput and minimize memory overhead in demanding web applications. It functions as a specialized HTTP server and client library that prioritizes efficient resource management, allowing developers to build scalable services capable of handling massive concurrent traffic with minimal garbage collection pressure. The library distinguishes itself through a focus on zero-allocation processing and low-level optimization. It achieves this by recycling temporary request and response objects through managed pools and

Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

Vitess is a distributed MySQL orchestrator and clustering system designed for horizontal database scaling. It functions as sharding middleware that distributes data and load across multiple MySQL instances to handle growth beyond the capacity of a single machine. The system provides a proxy layer that abstracts data distribution, allowing applications to query a cluster as a single logical database without knowing the physical location of the data. This is achieved through a routing mechanism that intercepts queries and directs them to the appropriate shards based on keyspace mappings. The p

This project is a collection of Python scripts and tools designed for web scraping, browser automation, and large-scale data extraction. It provides a set of implementations for retrieving information from websites and private APIs, including tools for multimedia downloading and social media data archiving. The toolset includes specialized mechanisms for bypassing anti-scraping measures through IP proxy pool rotation and multi-threaded crawlers. It also features capabilities for simulating browser sessions to handle authentication, intercepting session cookies, and decrypting network payloads

Iceberg is an open table format and big data table manager designed for huge analytic datasets in cloud storage. It provides a specification for tracking large-scale datasets to maintain transactional consistency and structural integrity. The project utilizes a standardized REST catalog interface to manage table metadata, ensuring interoperability between different compute engines. This allows diverse query engines to connect to a single table interface and maintain consistency across different processing frameworks. Its core capabilities include managing large-scale analytic tables, coordin

Delta is a lakehouse table format that brings ACID transactions and data warehouse consistency to large scale data lakes on cloud object storage. It serves as an ACID transaction manager, coordinating atomic commits and serializable isolation for concurrent reads and writes across distributed compute engines. The project provides a multi-engine interoperability layer that uses format translation to allow diverse SQL engines and processing frameworks to read and write the same tables. It functions as a data versioning system, utilizing a transaction log to enable time travel, historical snapsh

Noms is a distributed version control database and content-addressable data store. It identifies data by cryptographic hashes to ensure integrity and deduplication, while tracking dataset state changes through a sequence of immutable commits to enable branching, forking, and historical recovery. The system functions as a peer-to-peer data synchronizer, reconciling state between disconnected database instances to ensure all nodes converge on the same data. It distinguishes itself as a schema-flexible document store that supports self-describing types, allowing schemas to evolve and widen as ne

LMCache is a distributed key-value cache manager and tiering system designed to accelerate large language model inference. It functions as a tiered storage layer that offloads tensors from GPU memory to CPU RAM, local disks, or remote object stores, enabling the reuse of cached prefixes across different inference sessions and serving engines. The system differentiates itself through a disaggregated prefill-decode model, which separates prompt processing from token generation by transferring caches between distributed compute nodes. It utilizes peer-to-peer orchestration to share and retrieve

goleveldb هي قاعدة بيانات تخزين مفتاح-قيمة مدمجة للغة Go. توفر استمرارية البيانات المحلية والفهرسة، مما يسمح للتطبيقات بتخزين واسترجاع المعلومات باستخدام مفاتيح فريدة دون الحاجة إلى خادم منفصل. تنظم قاعدة البيانات البيانات باستخدام شجرة دمج منظمة بالسجل (log-structured merge-tree) وفهرسة دائمة بترتيب معجمي. يدعم هذا الهيكل عمليات مسح النطاق الفعالة وعمليات البحث القائمة على البادئة. يتضمن النظام قدرات لعمليات الكتابة المجمعة الذرية لضمان اتساق البيانات وتجنب التحديثات الجزئية. تتم إدارة الأداء من خلال تسجيل الكتابة المسبقة (write-ahead logging)، والتخزين المؤقت القائم على الذاكرة، وفلاتر bloom لتقليل إدخال وإخراج القرص غير الضروري.

Cerbos is an open-source authorization service that provides a centralized, language-agnostic engine for managing access control. It functions as a policy-as-code platform, allowing teams to define, test, and distribute authorization rules using declarative YAML or JSON configurations. By decoupling access logic from application code, it enables consistent permission enforcement across diverse service stacks. The project distinguishes itself through its ability to translate high-level authorization policies into native database query filters. This capability allows applications to enforce sec

This is a Java client library for interacting with the Kubernetes API server. It provides a programmatic interface and a set of typed models to manage cluster resources, orchestrate state reconciliation, and administer pods and namespaces within a Kubernetes environment. The library enables the development of custom controllers by providing frameworks for implementing automated control loops that reconcile the actual state of resources with a desired target state. It supports the generation of strongly-typed Java classes from OpenAPI specifications and custom resource definitions to ensure ty

pgdog is a PostgreSQL sharding proxy, distributed SQL router, and connection pooler. It is designed to enable horizontal data distribution by splitting tables and indices across multiple independent servers to scale storage and processing capacity. The project distinguishes itself through online resharding capabilities, using logical replication to move data between shards without application downtime. It supports multiple routing strategies, including hash, list, and range-based query routing, and manages distributed atomic transactions using a two-phase commit process to ensure consistency