Awesome GitHub RepositoriesFile-Based Storage Systems

This project is a comprehensive dataset and archive of classical Chinese poetry, prose, and Confucian classics. It serves as a digital humanities corpus, providing machine-readable access to hundreds of thousands of poems and detailed poet biographies, specifically spanning the Tang and Song dynasties. The collection is distinguished by its scholarly depth, incorporating textual variation annotations to track disputed characters across different source editions. It also includes tonal pattern mapping to describe the rhythmic and phonetic structures of the verse, alongside a popularity ranking

LevelDB is an embedded database library and persistent storage engine that provides a sorted key-value store. It uses a log-structured merge-tree architecture to map byte arrays to values, running directly within a process to provide storage without the need for a separate server process. The system is distinguished by its use of custom comparison functions to define key ordering, enabling efficient range scans and sequenced lookups. It ensures data reliability through atomic batch execution, consistent snapshot generation, and log-based recovery after failures. The engine covers broad capab

RocksDB is a high-performance, embeddable persistent key-value library and storage engine based on Log-Structured Merge-trees. It is designed to provide durable storage for large-scale datasets, integrating directly into applications to manage data on flash and RAM-based hardware. The engine is distinguished by its focus on minimizing read and write amplification through multi-threaded compaction and custom memory allocators. It features specialized optimizations for flash storage, including support for zoned block devices, and provides the ability to extend store behavior via external plugin

ArchiveBox is a self-hosted archiving tool designed for personal digital preservation and research data management. It functions as an automated web preservation engine that monitors URL inputs from bookmarks, browser history, or manual entries to capture and store permanent, offline copies of web content. By utilizing headless browser automation, the system renders dynamic web pages to ensure that captured snapshots, PDFs, and media assets remain accurate and accessible even if the original source disappears. The project distinguishes itself through a modular extractor pipeline and a task-qu

NativeScript is a cross-platform mobile development framework that enables the creation of native iOS and Android applications using JavaScript or TypeScript. It provides a direct bridge to native platform APIs, allowing developers to invoke native classes, methods, and properties directly from script code while maintaining full access to underlying mobile operating system features. The framework distinguishes itself through its direct native UI rendering, which maps declarative markup components to actual platform-native widgets rather than web-based views. This architecture is supported by

RocketMQ is a cloud-native distributed messaging platform and streaming engine. It functions as a distributed transactional queue that ensures atomicity between local transactions and message delivery, and serves as an MQTT IoT message broker to bridge lightweight device traffic into high-performance data streams. The system is distinguished by a Kubernetes-native architecture that decouples compute from storage to allow independent scaling of traffic and data retention. It utilizes a tiered storage model to offload older data to remote storage and employs quorum-based replication and automat

LibSQL is a high-performance, distributed SQL database engine that extends SQLite to support remote network access, edge computing, and real-time synchronization. It functions as an embedded database library that integrates directly into application processes while providing the infrastructure to maintain consistency across multiple geographic regions. The platform distinguishes itself by enabling database interaction over standard HTTP protocols, allowing applications to query remote data sources in serverless and edge environments without requiring local filesystem access. It includes nativ

Badger is an embeddable key-value store written in Go that provides persistent data storage for byte keys and values. It is a persistent database that utilizes a tiered LSM tree storage model to optimize disk storage and retrieval efficiency. The system features an ACID transaction engine that ensures data integrity through serializable snapshot isolation and multi-version concurrency control. It also provides an encrypted key-value store with data-at-rest encryption and a managed encrypted key registry to secure stored information. The engine covers a broad set of capabilities including hig

Hermes-webui is a self-hosted AI orchestrator and web interface for managing autonomous agents. It serves as a multi-provider gateway that connects cloud and local large language models, providing a central hub to execute scheduled background jobs, run shell commands, and manage agent memory on private hardware. The system distinguishes itself through a persistent memory manager that utilizes knowledge graphs and markdown files for long-term context across sessions. It features a model context protocol host for extending agent capabilities with standardized tools and supports the orchestratio

Bolt is a single-file embedded key-value store for Go applications. It is an ACID transactional database that organizes data in B+trees on disk to provide efficient sorted key retrieval and range scans. The system uses a memory-mapped model to map the database file directly into the process address space for fast random-access reads. The project distinguishes itself through a multi-version concurrency control architecture that allows multiple simultaneous readers to access a consistent snapshot of data without blocking a writer. It employs a single-writer multi-reader locking model and uses a

Dolphin is a software environment that simulates GameCube and Wii console hardware, allowing users to execute game discs and digital software on modern computing platforms. It functions by translating original console processor instructions into native host machine code at runtime and intercepting low-level graphics commands to render them through modern graphics APIs. The project distinguishes itself through a comprehensive netplay engine that enables online multiplayer for local games by synchronizing game states and input timing across remote instances. This system enforces deterministic e

This project is a markdown knowledge base used to maintain a curated collection of concise technical notes and write-ups across various programming languages and tools. It serves as a searchable personal reference library for documenting technical discoveries and software development patterns. The system implements a learning in public workflow, transforming markdown-based content storage into a static site. It utilizes directory-based routing to map folder structures to URL paths and employs schema-driven type generation to ensure data consistency across the knowledge base. The codebase cov

daedalOS is a browser-based operating system simulator that provides a virtualized desktop experience entirely within a web browser. It functions as a JavaScript desktop environment, utilizing a collection of web components to replicate the visual layout, windowing, file management, and taskbar functionality of a traditional desktop operating system. The project distinguishes itself by implementing a client-side process simulation that manages concurrent application instances and an event-driven window manager to coordinate overlapping graphical elements. It employs a virtualized file system

Phabricator is a software development suite consisting of a collection of integrated web applications designed to manage the full software engineering lifecycle. It serves as a project management platform, issue tracking system, and code review tool. The suite provides capabilities for bug tracking and coordination, allowing teams to report and manage software defects and feature requests. It also facilitates peer code review workflows to manage proposed changes before they are merged into a repository. The platform includes tools for project task organization and general software developmen

Multer is a multipart form parser and file upload manager designed as middleware for Node.js and Express.js applications. It extracts text fields and uploaded files from incoming requests to populate the request body and file objects for server-side use. The project utilizes pluggable storage engines to abstract file persistence, allowing developers to switch between saving files to a physical disk or holding them as buffers in system memory. It supports customizable file naming strategies and a custom storage engine interface for defining how files are saved and deleted. The middleware incl

SQLite is a serverless relational database engine and C-based library that stores data in a single local disk file. It functions as an embedded SQL database, integrating directly into applications without the need for a separate server process. The engine includes specialized capabilities for full-text search indexing and spatial data querying using R-Tree structures for geographic or geometric coordinate ranges. The system provides broad support for SQL data manipulation, database recovery and repair, and change tracking for synchronizing modifications between databases. It also features a

This project is a collection of bash automation scripts and command-line utilities designed to automate common tasks and retrieve information from web APIs. It serves as a toolkit for developer references, network diagnostics, and media access directly from the terminal. A central component of the suite is a GitHub Gist manager that handles the full lifecycle of code snippets. This includes cloning and syncing Gists into local directories for offline access, organizing content through hashtags, and searching through snippet descriptions and file contents using regular expressions. The toolki

jupyter-themes is a Jupyter Notebook theme manager and CSS interface customizer. It provides a command line tool to apply custom color schemes, fonts, and layout styles to notebook environments. The project includes a data visualization styling tool that synchronizes the aesthetic properties and color schemes of plotting libraries with the active interface theme. This ensures that data charts and figures remain visually consistent with the overall workspace theme.

Lets-chat is a self-hosted team communication platform and XMPP chat server designed for private messaging. It provides a containerized communication environment for small teams to exchange messages and files, featuring a programmable REST API for automating conversations and managing messages from external tools. The platform functions as an XMPP gateway and server, ensuring interoperability with other compliant messaging clients. It distinguishes itself by supporting enterprise identity management, allowing administrators to verify user identities through local accounts or external director

Cassandra is a distributed NoSQL database and wide-column store designed for high availability and linear scalability. It functions as a fault-tolerant distributed system that utilizes an LSM-tree storage engine to optimize write throughput and manage massive datasets. The system is a CQL-compliant database, using a structured query language to manage and retrieve tabular data stored across multiple nodes. It organizes information into rows and columns based on a flexible schema and primary keys. The project provides capabilities for horizontal database scaling, distributed data partitioning