Awesome GitHub RepositoriesData Storage Architectures

Redis is an in-memory, key-value database designed to provide sub-millisecond latency for read and write operations. It functions as a versatile data platform, serving as a distributed cache, a message broker, a NoSQL document store, and a vector database. The system utilizes an event-driven, single-threaded loop to process requests efficiently, while maintaining data durability through append-only persistence logs and asynchronous snapshotting mechanisms. What distinguishes Redis is its ability to handle complex data structures—including strings, hashes, lists, sets, and sorted sets—alongsid

AFFiNE is a collaborative knowledge base and productivity suite designed as a private-first, local-first platform. It provides an integrated workspace that combines structured documents with an infinite digital canvas, allowing users to organize complex information through a block-based model. By prioritizing local data persistence, the platform ensures immediate responsiveness and data sovereignty while maintaining a distributed state for real-time synchronization across multiple devices. The platform distinguishes itself through a canvas-integrated database engine that enables transitions b

Prometheus is a comprehensive monitoring and alerting platform designed to track infrastructure health and application performance. It functions as a time series database that ingests, indexes, and queries high-frequency numerical data points. By utilizing a pull-based model, the system periodically collects multi-dimensional metrics from monitored targets, storing them in an optimized block storage format that supports high-throughput ingestion and efficient historical analysis. The platform distinguishes itself through a specialized query engine that enables real-time analysis of performanc

Meilisearch is a Rust-based search engine providing typo-tolerant full-text and vector-based semantic search with real-time conversational capabilities.

Open-notebook is a collaborative workspace designed for knowledge management and structured data workflows. It functions as a centralized repository where users can document, refine, and retrieve information while interacting with artificial intelligence models to generate content and process complex data. The platform distinguishes itself through a local-first data persistence model that ensures offline availability and performance, paired with state-synchronized collaborative editing for real-time team sessions. It utilizes a virtualized rendering engine to maintain interface responsiveness

This project is an educational platform and tutorial series designed to teach the Go programming language through the practice of test-driven development. It provides a structured path for developers to master language fundamentals, concurrency, and standard library usage by building functional applications in small, verifiable increments. The core methodology centers on the test-driven development cycle, where failing tests are written before implementation to define requirements and ensure code correctness. This approach is applied across a wide range of practical scenarios, including the c

simdjson is a high-performance, header-only C++ library designed for parsing, querying, and serializing JSON data with minimal memory overhead. It functions as a hardware-aware data processing engine that leverages vector instructions to achieve gigabyte-per-second parsing speeds. By detecting host processor capabilities at runtime, the library automatically selects the most efficient instruction sets to accelerate structural analysis and validation. The library distinguishes itself through a focus on extreme efficiency and resource management. It utilizes memory mapping and padded buffer ali

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

This project is a themed technical art collection and visual archive of stylized illustrations depicting computer science literature. It serves as a technical book illustration gallery that pairs anime-style characters with specific programming and technical textbooks. The repository functions as a static technical asset library, utilizing a JSON-driven image gallery to map visual assets to their corresponding metadata. This structured approach allows for the sourcing of theme-consistent artwork for use in developer blogs, presentations, and technical documentation. The system employs a resp

Cube is a semantic data layer that provides a unified framework for defining business metrics, dimensions, and relationships across diverse data sources. By acting as a headless business intelligence engine, it transforms raw data into a governed model that can be queried via SQL, REST, and GraphQL interfaces. This architecture ensures consistent data definitions and logic across all downstream analytical applications and reporting tools. The platform distinguishes itself through its integrated conversational AI capabilities, which allow users to explore data using natural language. It orches

Super-productivity is a cross-platform productivity tool that integrates task management, time tracking, and data synchronization. It functions as a task management application for organizing daily work through todo lists and timeboxing, while serving as time tracking software to record activity durations and generate worklogs. The project features a synchronization layer that imports tickets and tasks from external project management tools, specifically Jira and GitHub, into a centralized local workflow. Data consistency across multiple devices is maintained through cloud storage synchroniza

NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t

Candle is a minimalist machine learning framework and deep learning inference engine designed for the Rust programming language. It functions as a low-level tensor computation library, providing the necessary primitives for multi-dimensional array operations and mathematical transformations required to execute pre-trained neural network models. The framework distinguishes itself through a focus on memory efficiency and hardware utilization. It employs static-typed tensor operations to enforce shape validation and memory safety at compile time, while utilizing a lazy-loaded computational graph

Yaak is a cross-platform desktop client and command-line utility designed for developing, testing, and debugging API endpoints. It supports multi-protocol request execution for REST, GraphQL, and gRPC services, providing a unified environment for managing network interactions, authentication credentials, and automated testing workflows. The tool distinguishes itself through a local-first architecture that stores all workspace configurations and request definitions directly on the filesystem. This design enables native integration with version control systems like Git, allowing teams to track

This project is a local-first task manager and time tracking tool designed to consolidate work items from multiple external project management platforms into a single, unified interface. By prioritizing local data sovereignty, it ensures that all task lists, time logs, and application states remain on the user's device, providing full functionality in offline environments while maintaining privacy. The application distinguishes itself through a focus on deep work and structured productivity rituals. It integrates distraction-free modes, configurable focus timers, and automated time tracking t

This project is a cross-platform graphics and compute framework that provides a unified, hardware-agnostic abstraction layer for rendering and parallel processing. It enables developers to build high-performance applications that execute consistently across diverse operating systems and hardware backends, including Vulkan, Metal, and DirectX. By mapping high-level graphics commands to native APIs, it serves as a portable foundation for both real-time 3D rendering and general-purpose GPU computing. The framework distinguishes itself through a robust architecture that supports both native deskt

This repository serves as a public archive for the raw datasets and analytical code used to support journalistic reporting. It functions as a platform for reproducible research, providing the necessary materials for users to verify published findings and conduct independent statistical analysis. The collection utilizes a versioned storage model to track historical changes to both data and processing scripts. By organizing information into a structured directory hierarchy, the repository maps specific journalistic projects to their corresponding inputs and outputs, ensuring that the methodolog

This is a mobile object database and NoSQL local data store that replaces relational tables with a schema-based model. It functions as a reactive data store, using live object observations and change notifications to trigger automatic user interface refreshes. The system provides built-in mobile cloud data synchronization to keep local datasets consistent with a remote server across multiple devices. It also includes security features for encrypted local storage, protecting sensitive on-disk data using at-rest encryption keys and fine-grained access control. Broad capabilities include object

Realm-Cocoa is a NoSQL mobile database engine and reactive object database designed for local data storage on mobile devices. It serves as a non-relational alternative to Core Data and SQLite, storing data as objects rather than tables. The system functions as an encrypted local store that protects sensitive application data using encryption. It provides reactive data synchronization, allowing application objects and user interfaces to update automatically when the underlying database changes.

Ceph is a unified, software-defined storage platform designed to provide object, block, and file storage services from a single distributed cluster. By decoupling data management from physical hardware, it enables elastic scaling across commodity hardware, allowing organizations to build large-scale storage infrastructure without reliance on proprietary vendor equipment. The system distinguishes itself through a shared-nothing, distributed architecture that utilizes deterministic hashing for data placement. This approach eliminates centralized metadata bottlenecks, allowing the cluster to sca