Awesome GitHub RepositoriesDisk-Backed Data Structures

Beanstalkd is a distributed work queue server designed for asynchronous job processing. It functions as a standalone server that distributes background tasks between producers and consumers to improve application responsiveness and throughput. The system organizes tasks using numeric priority levels to ensure critical work is processed first. It manages the job lifecycle through discrete states and uses a simple line-based text protocol over TCP for communication. To ensure reliability, the server persists job data to a sequential disk log, allowing the queue state to be recovered after a sy

This is an open-source, crowd-sourced wiki textbook that teaches Linux system programming in C. It covers the core operating system concepts of process management through the fork-exec-wait model, dynamic memory allocation using implicit free list heap allocators, inode-based file systems, inter-process communication via pipes and shared memory, POSIX threads with synchronization primitives, signal-based asynchronous notification, virtual memory with page table translation, and runtime diagnostics using Valgrind and GDB. The textbook distinguishes itself by providing practical, implementation

MapDB es un motor de base de datos embebido y una biblioteca de colecciones respaldada por disco que almacena colecciones de Java en disco o fuera del heap (off-heap). Funciona como un motor de procesamiento de datos local diseñado para manejar conjuntos de datos que exceden la RAM física disponible. El proyecto utiliza almacenamiento de datos off-heap para eliminar la sobrecarga de la recolección de basura y emplea caché de desbordamiento en disco para equilibrar el uso de memoria y disco. Proporciona utilidades especializadas para filtrar y analizar grandes volúmenes de datos locales en una sola máquina. El sistema garantiza la integridad de los datos mediante transacciones compatibles con ACID y control de concurrencia multiversión. Soporta mapas, conjuntos y colas persistentes utilizando indexación B-tree y un registro de transacciones secuencial para la durabilidad. El comportamiento del sistema y las operaciones internas del motor pueden ser monitorizados a través de niveles de registro y formatos de salida configurables.

Miniflare is a local simulator and emulator for Cloudflare Workers. It provides a serverless runtime environment on a local machine, allowing for the development and testing of functions and edge APIs without deploying to a production cloud environment. The project implements a local simulation of edge storage primitives, including key-value stores, object storage, and durable objects. It further enables a local integration test bed where serverless functions are executed within isolated sandboxes. The toolset covers a broad development surface, including hot-reloading via file-system monito

This project is a disk-backed key-value store and persistent data structure library for Python. It provides a mechanism for persisting mappings, sets, and queues to the local filesystem to bypass memory limitations and cache expensive function results across threads and processes. The system serves as a cross-process synchronization tool, offering distributed locks, semaphores, and barriers to coordinate shared resource access. It implements advanced caching strategies such as probabilistic stampede prevention, sharded data partitioning to increase throughput, and least-recently-used eviction