161 Repos
Systems that store and manage transient application data directly in system memory for high-performance access.
Distinguishing note: Focuses on volatile, process-local or shared-memory state management rather than persistent database engines.
Explore 161 awesome GitHub repositories matching data & databases · In-Memory Data Stores. Refine with filters or upvote what's useful.
Dokploy is a self-hosted platform-as-a-service designed to simplify the deployment and management of containerized applications and databases. It provides a centralized control plane that decouples administrative management from application workloads, allowing users to oversee infrastructure across multiple server nodes through a unified web interface or a command-line tool. The platform distinguishes itself through an extensive library of pre-configured application templates, enabling the rapid deployment of databases, identity providers, and various productivity or development tools. It sup
Delivers high-performance data storage using in-memory architecture.
Next-auth is an authentication library and identity framework used to manage user sign-in and session state across web applications. It provides a system for handling user identity through OAuth, OpenID Connect, and passwordless sign-in flows. The project features a multi-provider framework that integrates third-party identity services and custom directory backends. It supports passwordless authentication via email magic links or hardware keys and utilizes a database-agnostic storage layer to persist authentication states across different database types or in-memory. Security is managed thro
Provides a flexible storage layer for persisting auth states across different database types or in-memory.
Kotaemon is an orchestration framework designed for building modular, agentic workflows that integrate document processing, retrieval-augmented generation, and multi-step reasoning. It provides a comprehensive platform for developing document-based question answering systems, allowing users to chain language models, prompt templates, and external tools into complex, automated pipelines. The system distinguishes itself through a highly modular architecture that emphasizes component-based composition and schema-driven data exchange. It supports autonomous agents capable of decomposing complex q
Stores document collections in volatile memory for fast access during the application lifecycle.
Valkey is an in-memory, NoSQL database server designed for high-performance data storage and real-time state management. It operates as a distributed key-value store, maintaining datasets entirely within system memory to facilitate sub-millisecond response times for read and write operations. The system distinguishes itself through a single-threaded event loop that utilizes asynchronous I/O multiplexing to ensure high throughput. It supports high availability via master-replica replication and provides a decoupled communication model through a built-in publish-subscribe messaging pattern. To
Provides sub-millisecond latency for real-time applications via memory-resident storage.
gbrain is an agent framework and retrieval-augmented generation system that combines a durable task queue, a git-synced vector store, and a knowledge graph engine. It provides a foundation for building AI agents that interact with structured knowledge bases using the Model Context Protocol. The system synchronizes markdown files from a git repository into a database for high-performance semantic retrieval and creates typed edges between data pages by extracting entity references and wikilinks. It uses a database-backed queue to execute persistent background jobs and tool loops, ensuring relia
Synchronizes markdown files from a Git repository into a database for fast semantic retrieval.
Lowdb is a lightweight local data persistence library that manages application state as a JSON document. It functions as a database utility that stores information directly in local files or browser storage, removing the requirement for a dedicated database server or complex infrastructure. By treating stored data as standard objects and arrays, it provides a straightforward interface for managing application information. The library distinguishes itself through a modular adapter-based architecture that decouples data manipulation from the underlying storage medium. This design allows for the
Maintains a complete copy of the database in volatile memory for high-speed operations and accelerated testing.
This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha
Provides a high-performance in-memory data store for caching, document storage, and vector search.
Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network
Buffers events in memory or on disk to handle traffic spikes and prevent data loss during downstream outages.
Automa is a browser-based automation platform that enables users to build, schedule, and execute repetitive web tasks through a visual, no-code interface. By operating as a browser extension, it provides a canvas-based environment where users construct workflows by connecting functional blocks to interact with web elements, manage browser state, and process data. The platform distinguishes itself through its deep integration with the browser environment, allowing for complex orchestration such as event-driven triggers, cross-origin request handling, and the ability to package workflows as sta
Clears variables and table columns to manage memory and reset workflow state.
Bokeh is a Python data visualization library and interactive plotting framework used to create high-performance graphics and data dashboards that render in web browsers. It serves as a tool for generating standalone HTML documents, embedded components for digital notebooks, and full-stack web applications powered by a Python backend. The project distinguishes itself through its ability to handle large or streaming datasets while maintaining smooth interactivity. It enables linked brushing across multiple views, allowing data selected in one plot to automatically highlight corresponding data i
Organizes datasets into named arrays using a columnar format to optimize data transfer from Python to the browser.
GitButler is a desktop application designed for managing Git version control workflows. It provides a visual interface for organizing concurrent development tasks, allowing users to work on multiple features or bug fixes simultaneously within a single repository without the need for manual context switching. The application utilizes a virtual file system overlay and in-memory staging to track granular file modifications independently of the underlying version control system. It features a conflict isolation engine that breaks down complex merge conflicts into smaller, manageable segments, and
Tracks granular file modifications in a volatile layer before committing them to the underlying version control system.
Sandboxie is an operating system-level virtualization tool designed to run Windows applications in isolated, secure environments. By intercepting system calls and redirecting file system and registry modifications to a separate, discardable storage area, it prevents untrusted software from making permanent changes to the host system. This containment ensures that browser history, temporary files, and potential malware remain trapped within the sandbox, protecting the integrity and privacy of the underlying host. The software distinguishes itself through granular control over the isolation env
Uses volatile memory to store temporary sandbox files for high-speed performance and automatic cleanup.
StatsD is a network-based metrics daemon and aggregator that collects application performance data, such as counters and timers, for periodic delivery to backend services. It functions as system monitoring middleware, receiving telemetry via UDP to minimize performance overhead on monitored services. The system acts as a distributed metrics router, employing consistent hashing to distribute data points across clusters and ensure aggregation accuracy. It includes cluster health monitoring to track node availability and automatically recalculate routing paths when services go offline. The proj
Provides in-memory buckets for accumulating counters and timers before periodic backend delivery.
Caffeine is a high-performance caching library for the Java virtual machine designed to manage object lifecycles within the application heap. It functions as a thread-safe, memory-resident data store that reduces latency by keeping frequently accessed objects available for immediate retrieval. The library distinguishes itself through a sophisticated eviction strategy that balances recency and frequency to determine which entries to retain. It utilizes a frequency-based admission policy to evaluate the historical access patterns of new data, ensuring that the cache remains populated with the m
Functions as a memory-resident storage mechanism that reduces application latency by caching objects in the heap.
Abseil is a common utility library for C++ that provides foundational building blocks for applications. It serves as a collection of optimized utility functions and data structures that augment the C++ standard library across different compiler versions. The library is distinguished by its high-performance containers, including SIMD-accelerated hash maps and sets for efficient key-value lookups. It also provides a comprehensive framework for computing absolute time points, durations, and timestamps across global time zones. The project covers a broad range of capability areas, including conc
Uses a specialized SIMD-based table implementation for high-performance lookups in unordered containers.
This project is a retrieval-augmented generation application designed to answer questions from uploaded PDF documents. It functions as a document question-answering engine and a streaming AI chat interface that provides responses backed by specific source citations. The system utilizes a state-machine workflow orchestrator to coordinate multi-step document ingestion and retrieval pipelines. This orchestration allows for step-by-step visualization and debugging of the process as documents are parsed and processed. The application manages the full lifecycle of document interaction, including P
Stores numerical vector embeddings of PDF text chunks to enable efficient semantic similarity searching.
Arrow is a cross-language development platform for in-memory data. It provides a standardized, language-independent columnar memory format designed to accelerate analytical operations and improve memory efficiency on modern computing hardware. By utilizing a schema-driven approach, the framework enables the efficient organization of both flat and nested data structures. The project functions as an analytical data processing engine that facilitates high-performance computation directly on memory-resident datasets. It distinguishes itself through a zero-copy architecture, which allows multiple
Provides a language-independent standard for organizing flat or nested data in memory.
MailHog is an email testing tool that functions as a local SMTP server to capture outgoing messages during the development process. By intercepting SMTP traffic, it allows developers to verify email content, headers, and attachments without transmitting messages to actual external recipients. Captured data is stored in an in-memory buffer, ensuring that sensitive information remains accessible for inspection through a dedicated web-based dashboard. The platform distinguishes itself by incorporating network simulation capabilities, which allow for the injection of artificial delays or connecti
Maintains a volatile buffer of captured email data to allow rapid retrieval and viewing without requiring persistent disk storage.
X-algorithm is a modular recommendation engine framework designed to orchestrate personalized content feeds. It functions as a machine learning ranking system that manages the end-to-end lifecycle of content delivery, from initial candidate retrieval to final display ordering. The system distinguishes itself through a multi-stage pipeline that integrates vector-based similarity search with transformer-based engagement prediction. By mapping user history and content features into high-dimensional embeddings, it performs rapid approximate nearest neighbor searches to identify relevant items. Th
Maintains low-latency access to recent event streams and user interaction data for sub-millisecond retrieval.
Memori is an AI agent memory middleware platform designed to provide persistent, context-aware recall for language models. It functions as a non-intrusive layer that intercepts outbound model requests to automatically capture interaction history and execution traces, ensuring that agents maintain continuity across sessions without requiring modifications to existing application logic. The platform distinguishes itself through a dual-model storage architecture that maintains information as both structured relational primitives for precise fact retrieval and rolling narrative summaries for situ
Allows persistence of structured memory records into custom relational or document databases.