Awesome GitHub RepositoriesPersistent Storage Providers

Puter is a browser-based desktop environment and cloud-native development platform that provides a virtualized graphical workspace. It enables developers to build and deploy full-stack web applications by integrating cloud storage, authentication, and serverless backend logic directly into the browser, eliminating the need for traditional server infrastructure. The platform distinguishes itself through a unified cloud storage layer and a distributed network runtime that facilitates peer-to-peer communication and cross-origin resource fetching. It features a sophisticated cross-window orchestr

This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

Automa is a browser-based automation platform that enables users to build, schedule, and execute repetitive web tasks through a visual, no-code interface. By operating as a browser extension, it provides a canvas-based environment where users construct workflows by connecting functional blocks to interact with web elements, manage browser state, and process data. The platform distinguishes itself through its deep integration with the browser environment, allowing for complex orchestration such as event-driven triggers, cross-origin request handling, and the ability to package workflows as sta

NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t

Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

Memori is an AI agent memory middleware platform designed to provide persistent, context-aware recall for language models. It functions as a non-intrusive layer that intercepts outbound model requests to automatically capture interaction history and execution traces, ensuring that agents maintain continuity across sessions without requiring modifications to existing application logic. The platform distinguishes itself through a dual-model storage architecture that maintains information as both structured relational primitives for precise fact retrieval and rolling narrative summaries for situ

Annoy is a C++ library designed for approximate nearest neighbor search in high-dimensional vector spaces. It functions as a vector similarity search engine that constructs static, disk-based data structures to facilitate fast lookups. By mapping identifiers to vector data and persisting these structures to disk, the library enables efficient, memory-mapped access to large datasets. The project distinguishes itself through the use of random projection trees and distance-metric-based partitioning, which organize data into hierarchical binary trees to balance search precision against computatio

Dask هو إطار عمل للحوسبة المتوازية وجدول مهام موزع مصمم لتوسيع نطاق سير عمل علوم البيانات في Python من أجهزة فردية إلى مجموعات (clusters) كبيرة. يعمل كمدير موارد للمجموعة يقوم بتنسيق المنطق الحسابي من خلال تمثيل المهام وتبعياتها كرسوم بيانية موجهة غير دورية. تسمح هذه البنية للنظام بأتمتة توزيع أعباء العمل عبر الأجهزة المتاحة مع إدارة متطلبات التنفيذ المعقدة. يتميز المشروع بمحرك تقييم كسول يؤجل عمليات البيانات حتى يتم طلبها صراحة، مما يتيح تحسين الرسم البياني العالمي وتخصيص الموارد بكفاءة. يتضمن خاصية تسريب البيانات الواعية بالذاكرة لمنع تعطل النظام عند معالجة مجموعات البيانات التي تتجاوز الذاكرة المتاحة، ويستخدم دمج الرسم البياني للمهام لدمج تسلسلات العمليات في خطوات تنفيذ واحدة، مما يقلل من عبء الجدولة والاتصال بين العقد. توفر المنصة سطح قدرات شاملاً لتحليلات البيانات واسعة النطاق، بما في ذلك دعم التعلم الآلي الموزع، وتكامل الحوسبة عالية الأداء، ومعالجة البيانات المتوازية. توفر أدوات واسعة النطاق لإدارة دورة حياة المجموعة، وتوصيف الأداء، والمراقبة في الوقت الفعلي لتنفيذ المهام. يمكن للمستخدمين نشر هذه البيئات عبر بنية تحتية متنوعة، بما في ذلك الأجهزة المحلية، ومزودي السحابة، والأنظمة الحاوية، ومجموعات الحوسبة عالية الأداء.

Redux Persist is a state management utility designed to maintain application data across sessions by synchronizing state with local storage. It functions as a middleware-based engine that automatically saves and restores application state, ensuring that user sessions remain intact after page refreshes or application restarts. The library distinguishes itself through granular control over the persistence lifecycle and data structure. It supports recursive state tree partitioning, allowing developers to apply distinct storage configurations and filtering rules to specific branches of the applic

OpenRefine is a data cleaning tool and wrangling platform used to transform raw, messy datasets into consistent and structured formats. It operates as a Java-based data processor that runs a local server and provides a web browser interface for managing and manipulating data. The platform includes a data reconciliation engine for matching local entries against external knowledge bases to standardize entities. It also functions as a web data augmentation tool, allowing users to fetch and integrate information from external web sources to enrich their datasets. The system provides a transforma

GPTCache is a semantic caching layer and response optimizer for large language models. It functions as pluggable middleware for orchestration frameworks, utilizing vector database caching to store and retrieve model responses based on the semantic similarity of prompts rather than exact text matches. The system uses embeddings to determine cache hits by comparing the distance between new queries and stored vectors. It employs a hybrid storage model that persists original prompts in relational databases while maintaining high-dimensional embeddings in vector stores. The project covers a broad

Riverpod is a reactive state management framework for Dart that manages application state through providers which automatically track dependencies and rebuild dependents when values change. It functions as an asynchronous data caching library, caching results of async operations and exposing loading, error, and data states to widgets without manual future handling. The framework includes a provider composition system that combines multiple providers into derived values that recompute automatically when any dependency changes, along with code generation for state providers that reduces boilerpl

FASTER is a high-throughput key-value store that combines an in-memory data store with a hybrid memory-disk storage engine, enabling datasets larger than available RAM. It uses a latch-free, cache-optimized index for concurrent point lookups and heavy updates, and records all mutations to a persistent append-only log on disk with checksum validation and group-commit checkpointing for crash recovery. The system supports multi-key transactional workloads through atomic multi-key locking, ensuring transactional consistency without coarse-grained contention. It exposes the key-value store to remo

Materialize is a streaming SQL database that continuously ingests live data from sources such as Kafka, Redpanda, PostgreSQL, and MySQL, and incrementally maintains materialized views. It provides a PostgreSQL-compatible query engine that accepts standard SQL over the PostgreSQL wire protocol, enabling any existing SQL client or BI tool to query real-time data. The system also includes a Model Context Protocol (MCP) server that exposes live materialized view data to AI agents, providing fresh context without polling. Materialize distinguishes itself through its ability to offer configurable c

PikiwiDB هو قاعدة بيانات NoSQL موزعة ومخزن مفتاح-قيمة يعتمد على القرص يعمل كخادم بروتوكول متوافق مع Redis. تم تصميمه للتعامل مع مجموعات بيانات أكبر من ذاكرة النظام المتاحة من خلال استخدام محرك استمرارية يخزن مجموعة البيانات الكاملة على القرص. يستخدم النظام نموذج تخزين متدرج، حيث يقوم بتخزين البيانات الساخنة التي يتم الوصول إليها بشكل متكرر في الذاكرة مع الحفاظ على الحجم الأساسي على القرص. يضمن توفراً عالياً من خلال بنية مخزن بيانات مكرر، باستخدام سجلات ثنائية غير متزامنة لمزامنة البيانات بين العقد الأساسية والثانوية. يدعم المشروع توسيع قاعدة البيانات الموزعة عبر تقسيم البيانات القائم على المجموعة (sharding) وينظم البيانات في مجموعات لتوسيع طبقة التخزين. تشمل قدراته التشغيلية مراقبة أداء النظام لتتبع استخدام الموارد ودعم النشر المعبأ في حاويات.

SparkInternals is a technical reference and architecture guide detailing the internal design and implementation of the Apache Spark distributed computing engine. It serves as a study of big data engine analysis, focusing on how the system manages cluster execution and the interaction between driver nodes, executors, and workers. The project provides a detailed breakdown of how logical plans are converted into physical execution stages. It specifically analyzes the mechanics of data shuffle operations, memory management, and the coordination of distributed job scheduling. The documentation co

هذا المشروع عبارة عن حزمة هوية .NET وإطار عمل خادم يُستخدم لبناء مزودي هوية OpenID Connect و OAuth 2.0 معتمدين. يوفر المنطق الأساسي المطلوب لإصدار والتحقق من رموز الأمان وإدارة مصادقة المستخدم عبر أنواع المنح وتدفقات البروتوكول المختلفة. يتضمن إطار العمل طبقة ترجمة بروتوكول تربط OpenID Connect و SAML لتمكين التشغيل البيني بين مزودي الهوية المختلفين. كما يدعم وضع التشغيل عديم الحالة، والذي يزيل التحقق والتخزين المدمجين للسماح بالتحكم اليدوي في التحقق من الرمز والعميل. يغطي النظام مجموعة واسعة من القدرات، بما في ذلك التحكم في الوصول إلى API من خلال وسيط التحقق من الرمز وتكامل الهوية عبر الأنظمة الأساسية لتطبيقات سطح المكتب والجوال. يتميز بخط أنابيب معالجة طلب قابل للتخصيص مع مزودي استمرارية قابلين للتوصيل لتخزين بيانات الهوية وتخزين طلبات التفويض مؤقتاً.

android-common is a collection of shared utility components and framework libraries for Android development. It provides specialized toolkits for reverse engineering, system utility management, data caching, and high-performance user interface components. The project includes a reverse engineering toolkit for inspecting application internals through package decompilation and manifest data extraction. It also features a system utility toolkit for managing file operations and executing shell commands within the Android operating system. The library covers several capability areas, including da

MiniOB is an open-source educational relational database kernel designed for learning the internals of database systems. It implements a dual-engine storage architecture combining B+ Tree and LSM-Tree, supports SQL parsing and query execution, and provides transactional processing with multi-version concurrency control. The system communicates with clients using the MySQL wire protocol and includes a vector database extension for storing and querying high-dimensional vectors. The project distinguishes itself through its comprehensive coverage of core database concepts in a single, learnable c