Awesome GitHub RepositoriesDisk-Backed Data Structures

Beanstalkd is a distributed work queue server designed for asynchronous job processing. It functions as a standalone server that distributes background tasks between producers and consumers to improve application responsiveness and throughput. The system organizes tasks using numeric priority levels to ensure critical work is processed first. It manages the job lifecycle through discrete states and uses a simple line-based text protocol over TCP for communication. To ensure reliability, the server persists job data to a sequential disk log, allowing the queue state to be recovered after a sy

This is an open-source, crowd-sourced wiki textbook that teaches Linux system programming in C. It covers the core operating system concepts of process management through the fork-exec-wait model, dynamic memory allocation using implicit free list heap allocators, inode-based file systems, inter-process communication via pipes and shared memory, POSIX threads with synchronization primitives, signal-based asynchronous notification, virtual memory with page table translation, and runtime diagnostics using Valgrind and GDB. The textbook distinguishes itself by providing practical, implementation

MapDB 是一个嵌入式数据库引擎和磁盘支持的集合库，将 Java 集合存储在磁盘或堆外内存中。它作为一个本地数据处理引擎，旨在处理超过可用物理内存（RAM）的数据集。 该项目利用堆外数据存储来消除垃圾回收开销，并采用磁盘溢出缓存来平衡内存和磁盘使用。它为在单台机器上过滤和分析海量本地数据提供了专门的实用程序。 该系统通过符合 ACID 的事务和多版本并发控制（MVCC）确保数据完整性。它支持使用 B 树索引和用于持久性的顺序事务日志的持久化映射、集合和队列。 系统行为和内部引擎操作可以通过可配置的日志级别和输出格式进行监控。

Miniflare 是一个用于 Cloudflare Workers 的本地模拟器和仿真器。它在本地机器上提供了一个无服务器（serverless）运行时环境，允许在不部署到生产云环境的情况下开发和测试函数及边缘 API。 该项目实现了边缘存储原语的本地模拟，包括键值存储、对象存储和持久对象（Durable Objects）。它进一步启用了本地集成测试平台，其中无服务器函数在隔离的沙盒中执行。 该工具集涵盖了广泛的开发面，包括通过文件系统监控进行热重载、存储绑定的本地状态持久化，以及环境密钥管理。它还支持使用状态隔离容器进行自动化集成测试，以防止测试用例之间的数据泄漏。

This project is a disk-backed key-value store and persistent data structure library for Python. It provides a mechanism for persisting mappings, sets, and queues to the local filesystem to bypass memory limitations and cache expensive function results across threads and processes. The system serves as a cross-process synchronization tool, offering distributed locks, semaphores, and barriers to coordinate shared resource access. It implements advanced caching strategies such as probabilistic stampede prevention, sharded data partitioning to increase throughput, and least-recently-used eviction