Kvrocks

Kvrocks is a distributed key-value store and Redis-compatible NoSQL database. It utilizes a RocksDB storage engine to provide disk-based persistence, allowing for high-capacity data storage with reduced memory costs compared to in-memory systems.

The system functions as a vector database and full-text search engine, supporting nearest-neighbor searches on vector embeddings and complex document queries via text matching. It employs a proxyless cluster architecture with slot-based routing to distribute data and scale capacity across multiple nodes.

The platform covers a wide range of data management capabilities, including JSON document management, time-series data, and real-time stream processing. It provides advanced search and indexing through geospatial querying, secondary indexing, and query plan analysis, while offering probabilistic data sketching for memory-efficient cardinality and membership estimation.

Additional operational features include atomic transactions, pub/sub messaging, and namespace data isolation for multi-tenant environments.

Features

Unbounded Key-Value Stores - Provides a disk-based key-value store using RocksDB to handle massive NoSQL datasets exceeding physical memory.
Redis Compatible Stores - Implements the Redis wire protocol to ensure full compatibility with existing Redis clients and ecosystems.
Vector Databases - Provides a dedicated vector database for storing and querying high-dimensional embeddings.
Cluster Request Redirections - Redirects client requests to the appropriate node using slot-based routing and redirection responses.
Cluster Request Routing - Provides topology metadata to compatible SDKs to direct requests to the correct cluster node.
Embedded Storage Engines - Implements a RocksDB-based embedded storage engine for high-capacity disk-based persistence.
Distributed Key-Value Stores - Operates as a distributed key-value store that partitions data across nodes using a proxyless architecture.
Full-Text Search Indexes - Provides full-text search indexing for complex document queries using text matching.
Disk Persistence - Implements disk-based persistence to allow high-capacity data storage beyond the limits of physical memory.
Full Text Search - Supports full-text search querying for documents matching complex text patterns.
Full-Text Search Engines - Functions as a full-text search engine supporting complex document queries and geospatial coordinates.
Inter-Node Data Synchronization - Maintains data consistency across a distributed cluster via primary-replica state synchronization.
Primary-Replica Replication - Uses a primary-replica architecture to maintain data consistency by streaming incremental updates across nodes.
Cluster Topology Management - Defines node roles and slot assignments using versioned updates to ensure consistent request routing.
LSM-Tree Storage Engines - Utilizes a RocksDB-based LSM-tree storage engine to persist key-value data efficiently on disk.
Vector Search - Provides vector search capabilities using nearest-neighbor searches on float64 embeddings.
Slot-Based Cluster Scaling - Manages data across multiple nodes using a proxyless cluster architecture with slot-based rebalancing.
Hash Slot Sharding - Distributes data across cluster nodes using a fixed number of hash slots for consistent request routing.
Proxyless Cluster Architectures - Implements a proxyless cluster architecture where clients route requests directly to target nodes via topology metadata.
Append-Only Message Logs - Implements append-only message logs with consumer group support for real-time distributed stream processing.
Backup & Recovery - Creates full and incremental data copies to prevent loss and enable recovery to remote servers.
Cardinality Estimation - Employs probabilistic counting algorithms to estimate the number of unique elements in large sets.
Automatic Background Compactions - Provides automatic background compaction of LSM-tree sorted runs to reclaim disk space and optimize read performance.
Atomic Transactions - Groups multiple commands into a single uninterruptible block to ensure data consistency.
Quantile Sketches - Uses sketch-based algorithms to efficiently estimate ranks and quantiles of observation distributions.
Geospatial Indexes - Implements geospatial indexing for location-based searches and distance calculations.
Hash Field Management - Provides hash data structures supporting individual field manipulation and per-field expiration.
Hybrid Vector and Full-Text Search - Integrates nearest-neighbor vector similarity searches with lexical full-text indexing for hybrid retrieval.
JSON Document Manipulation - Enables manipulation of hierarchical JSON documents using path expressions, array operations, and numeric increments.
Key Lifecycle Management - Manages data lifecycles through time-to-live settings, expiration timers, and asynchronous deletions.
Namespace Isolation - Implements namespace-based data isolation to separate environments for multi-tenant users.
Distributed Ordered Lists - Stores sequences of strings supporting index-based access and blocking pops across a distributed cluster.
Probabilistic Data Sketches - Employs Bloom filters and HyperLogLog for memory-efficient cardinality estimation and membership testing.
Bloom Filters - Uses Bloom filters for probabilistic membership testing to minimize memory usage.
Real-Time Data Streaming - Implements append-only logs with consumer groups for real-time distributed stream processing.
Indexed Document Retrieval - Allows retrieval of documents and records from indexes using sorting and limiting.
Secondary Indexes - Supports secondary indexing on tag, numeric, and vector fields for complex queries.
Server-Side Scripting - Enables the execution of server-side scripts for performing complex atomic operations.
Sorted Sets - Stores unique members with numerical scores to enable range queries by index or lexicographical order.
Atomic String Operations - Provides atomic increments and compare-and-swap operations for string-based key-value storage.
Time-Series Data Modeling - Stores and retrieves timestamped data points for efficient temporal analysis and aggregation.
Unordered Unique Collection Management - Manages collections of unique elements supporting set-theoretic intersection, union, and difference operations.
Value-Log Reclamation - Triggers background compaction to permanently remove obsolete data and recover physical disk space.
Pub-Sub Messaging - Implements a publish-subscribe messaging pattern for broadcasting data across named channels.
Shard Rebalancing - Moves data slots between nodes to optimize storage distribution and scale system capacity.

apache/incubator-kvrocks

4,339View on GitHub

Kvrocks is a disk-based NoSQL database and distributed key-value store that leverages the RocksDB storage engine to persist large datasets to physical disk. It is designed to be a Redis-compatible database, utilizing the standard Redis communication protocol to ensure interoperability with existing client libraries and tools. The project distinguishes itself by combining a disk-persistent storage model with advanced retrieval capabilities, including vector search for k-nearest neighbor queries, full-text search indexing, and geospatial query execution. It supports distributed clustering with

tporadowski/redis

9,987View on GitHub

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

microsoft/garnet

11,885View on GitHub

Garnet is a multi-threaded in-memory database and distributed key-value store. It functions as a high-performance remote cache store that implements the RESP wire protocol to maintain compatibility with existing Redis clients and libraries. The project is distinguished by a shared-memory architecture that enables parallel request processing across multiple cores for sub-millisecond latency. It features a tiered storage system that automatically offloads colder data from system memory to SSD or cloud storage layers, and includes a specialized vector search database for high-dimensional similar

redis/go-redis

22,159View on GitHub

This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha

apachekvrocks

Features