Awesome GitHub RepositoriesMulti-level Caching

This project is a comprehensive Java backend engineering guide and technical reference focused on high-concurrency design, distributed systems, and microservices architecture. It provides detailed strategies for decomposing monolithic applications, managing service discovery, and implementing the architectural patterns required for scalable backend environments. The repository distinguishes itself through an extensive collection of big data algorithmic references and database scaling strategies. It covers memory-efficient techniques for analyzing massive datasets, such as Top-K element extrac

RocksDB is a high-performance, embeddable persistent key-value library and storage engine based on Log-Structured Merge-trees. It is designed to provide durable storage for large-scale datasets, integrating directly into applications to manage data on flash and RAM-based hardware. The engine is distinguished by its focus on minimizing read and write amplification through multi-threaded compaction and custom memory allocators. It features specialized optimizations for flash storage, including support for zoned block devices, and provides the ability to extend store behavior via external plugin

AISystem is a comprehensive AI full-stack infrastructure project covering the entire pipeline from AI chip architecture to high-level training frameworks. It encompasses the development of AI compiler frameworks, inference engines, and distributed training orchestrators designed to coordinate workloads across a heterogeneous compute stack of CPUs, GPUs, and NPUs. The project focuses on the deep integration of software and hardware, employing software-hardware co-design to align tensor layouts with physical memory structures. It provides specialized capabilities for accelerating Transformer mo

Open-IM-Server is an instant messaging server and cross-platform messaging backend designed for real-time text and media communication. It provides a distributed chat architecture that coordinates user sessions and group communications across multiple server nodes to support millions of concurrent users. The system features a scalable message distribution model that utilizes asynchronous message pipelines and distributed queueing to maintain reliability during traffic spikes. It employs multi-level data caching across local memory and distributed stores to reduce latency and uses a document-b

Doctrine ORM is a PHP object-relational mapper that connects application objects to relational database tables. It uses the data mapper and identity map patterns to decouple the in-memory object model from the database schema, allowing developers to manage data persistence without writing manual SQL. The project features a dedicated object-oriented query language and programmatic builder for retrieving data based on entities rather than tables. It implements a unit-of-work system to track object changes during a request and synchronize them via atomic transactions. The capability surface inc

XiangShan is a high-performance RISC-V processor core and a hardware description language framework. It provides a construction-based system for designing, simulating, and verifying complex processor micro-architectures and peripheral devices. The project includes a high-performance CPU simulator used for architectural exploration and functional verification of processor execution. The project implements a superscalar out-of-order CPU architecture that uses renaming and reorder buffers to execute instructions in parallel. It generates synthesizable Verilog files from hardware descriptions to

Hibernate ORM 是一个 Java 对象关系映射器，也是 Jakarta Persistence API 的完整实现。它作为 SQL 数据库抽象层，将 Java 对象模型转换为关系数据库模式，以管理数据持久化和生命周期。 该框架以其多租户数据隔离框架而著称，可在单个数据库实例中分离客户数据。它还具有数据库模式生成器，可根据实体映射自动生成和更新关系结构。 该系统涵盖了广泛的功能领域，包括事务管理、并发锁定控制以及用于审计日志的时间数据跟踪。它通过实体图策略提供数据获取优化工具，并支持向量数据和国家化字符等高级数据类型。 该项目包含一套全面的持久层测试工具，包括数据库方言过滤和规范合规性测试。

Upspin is a decentralized naming and storage system that provides an end-to-end encrypted file system. It assigns every user a unique identity and organizes files within a global, permissioned namespace where data is encrypted on the client before transmission. The system separates identity resolution from data storage using a public-key identity provider and a key-server architecture. This allows for decentralized identity management and the resolution of usernames to specific directory and storage server addresses. The project includes a hierarchical access control system that manages read

Pebble is an embedded key-value storage engine written in Go, designed as a library that provides durable, write-optimized data persistence directly within applications. It organizes data using a log-structured merge-tree (LSM-tree) structure, where writes are first buffered in an in-memory skiplist memtable and persisted to a write-ahead log before being flushed to block-based SSTable files on disk. The engine supports atomic batch commits, configurable write synchronization, and automatic background compaction that merges and rewrites sorted runs to reclaim space and maintain read performanc

HotChocolate is a GraphQL server implementation for .NET that handles queries, mutations, and subscriptions over HTTP. It provides code-first schema generation directly from C# classes and attributes, keeping the schema and business logic in sync without manual type definitions. The project also includes a GraphQL gateway that composes multiple GraphQL and REST endpoints into a single unified schema, along with a GraphQL IDE for exploring, testing, and debugging GraphQL APIs interactively. The project distinguishes itself through its comprehensive tooling ecosystem, which includes a GraphQL c

SpringSide 4 is an enterprise Java reference architecture and utility library built on the Spring Framework. It provides a pragmatic, best-practice application stack for building RESTful web services, web applications, and data access layers, along with a curated collection of high-performance utility classes for common operations like text, date, collection, reflection, concurrency, and I/O handling. The project distinguishes itself by combining a complete reference application scaffold with production-oriented infrastructure. It includes a JPA-based data access layer that automatically tran

JetCache 是一个 Java 缓存框架，提供了一个用于管理应用数据的统一接口。它作为一个二级缓存管理器，协调本地内存存储和远程分布式缓存，以减少网络延迟并提高检索速度。 该项目的特色在于其声明式缓存库，使用注解来自动化方法结果的存储、过期和失效。它包含一个分布式缓存同步器，通过使集群实例间的本地条目失效来保持一致性，并提供一个分布式锁工具来协调对共享资源的访问。 该框架涵盖了广泛的性能和稳定性功能，包括非阻塞异步检索、自动后台缓存刷新，以及防止缓存穿透的请求合并。它还提供程序化缓存管理、可插拔序列化策略，以及诸如白名单反序列化过滤等安全特性。

OpenTSDB 是一个分布式时间序列数据库和指标引擎，专为存储和管理海量高基数系统指标而设计。它作为一个数据存储和分析平台，支持跨分布式集群的大规模指标摄取和基础设施性能监控。 该系统以其支持 HBase、Cassandra 和 Google Bigtable 等多个后端的分布式存储抽象而著称。它利用分层指标树来组织时间序列，并采用数字标识符索引来减少存储占用并加速标记指标的查找。 该项目涵盖了广泛的能力领域，包括具有分布式百分位数计算和降采样功能的时间序列数据分析，以及全面的元数据管理。它提供用于数据摄取和查询的 API 集成、用于性能优化的堆外缓存，以及用于数据完整性审计和异常分析的工具。 该系统通过用于数据库管理和指标树同步的命令行界面进行管理。

该项目为异步流处理提供了正式规范和一套标准 Java 接口。它定义了一个标准化的协议，用于在不同线程的发布者和订阅者之间传递元素序列，核心是 JVM 的响应式流（Reactive Streams）规范。 该项目通过提供一个通用 API 来实现互操作性，允许不同的异步流处理库协同工作。这是通过一组标准接口和桥接机制实现的，这些机制可以在不兼容的流规范之间进行转换。 该规范涵盖了非阻塞背压（Backpressure）协议，通过要求订阅者发出需求信号来调节数据流并防止系统过载。它还定义了流的生命周期，包括订阅管理、元素处理以及用于资源清理的基于信号的终止。 该项目包括一个用于验证流行为的框架，以根据背压和异步事件规则验证处理逻辑。

Unbound 是一个 DNS 解析器，作为缓存、递归和验证服务器运行。它通过在互联网上执行独立查找或将查询转发给外部解析器来将域名转换为 IP 地址。 该项目的特色在于作为可编程 DNS 服务器，提供 Python 脚本接口和动态共享库加载以自定义解析流水线。它还作为 DNS64 NAT64 网关，从 IPv4 记录合成 IPv6 地址，以实现 IPv6 客户端的通信。 该系统包含用于 DNSSEC 加密验证和 DNS 流量加密的安全框架，以防止缓存中毒和窃听。其功能扩展到基于策略的响应过滤、具有外部数据库后端的二级缓存以及本地权威区域数据的交付。 管理控制通过用于向运行中的服务器发送加密命令的安全远程接口进行管理。

InternetArchitect 是一个教育性文档和源代码合集，旨在作为高并发架构课程。它作为一个分布式系统实现指南，提供技术模式和实践示例，用于设计在重负载下保持稳定性的可扩展互联网架构。 该项目专注于高性能数据库优化和微服务设计模式。它涵盖了通过数据库分片和代理层减少延迟并提高吞吐量的策略，以及在分布式集群中协调全局状态的方法。 架构范围包括用于加速数据检索的多级缓存策略，以及用于管理解耦微服务之间通信的服务发现框架。它还解决了分布式状态协调问题，并使用负载均衡网格在后端服务器之间分配网络流量。

MiniOB is an open-source educational relational database kernel designed for learning the internals of database systems. It implements a dual-engine storage architecture combining B+ Tree and LSM-Tree, supports SQL parsing and query execution, and provides transactional processing with multi-version concurrency control. The system communicates with clients using the MySQL wire protocol and includes a vector database extension for storing and querying high-dimensional vectors. The project distinguishes itself through its comprehensive coverage of core database concepts in a single, learnable c

Rainloop Webmail 是一个基于浏览器的自托管电子邮件应用程序，通过 IMAP 和 SMTP 协议连接到外部邮件服务器。它作为一个无状态客户端运行，直接从服务器获取数据，无需本地数据库来存储邮件。 该系统支持多账户邮件管理，允许用户将多个电子邮件身份链接到单个会话，以便在不同的浏览器标签页中同时访问。它包含一个基于插件的扩展系统，以及用于用户界面自定义和品牌化的工具。 该应用程序提供了邮件工作流自动化功能，包括收件过滤和自动休假回复。它还集成了外部云存储提供商以方便邮件附件处理，并利用多级响应缓存来优化数据检索。

这是一个教育框架和工具包，专为开发和测试操作系统内核而设计。它提供了一个结构化的环境，用于使用 Rust 和 C 实现基础系统原语，包括虚拟内存管理、抢占式进程调度和文件系统组织。该框架专门面向 RISC-V 和 x86 架构，作为学习如何从零开始构建核心系统软件的实验室。 该项目的独特之处在于同时支持裸机部署和硬件仿真，允许开发者在物理硬件或模拟环境中验证内核逻辑。它包含一个全面的构建工具链，可处理跨架构编译和固件初始化，确保内核能够跨不同目标平台正确引导和执行。 该系统涵盖了广泛的底层功能，包括将系统逻辑与特定处理器设计解耦的硬件抽象层，以及管理用户空间应用与内核服务之间边界的系统调用接口。它还结合了运行时日志记录和执行监控等可观测性功能，以协助在开发过程中分析系统行为。

Mini-LSM 是一个教育性存储引擎和键值数据库库，旨在演示日志结构合并树（LSM-tree）架构的实现。它作为教学资源，用于理解如何从零开始构建高性能存储系统，重点关注持久化数据结构和基于磁盘的存储机制。 该项目提供了一个功能框架，用于通过内存到磁盘的刷新和多版本并发控制（MVCC）来管理数据。它通过实现基于快照的隔离来脱颖而出，允许在并发操作期间查看数据库状态的一致视图，并利用分层压缩策略来组织数据文件并优化读取性能。 该引擎涵盖了广泛的存储管理能力，包括用于崩溃恢复的预写日志（WAL）和用于最小化磁盘 I/O 的概率索引。它还包含一套验证工具和测试工具，旨在验证存储操作的完整性并协助诊断并发相关问题。