# apache/incubator-kvrocks

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/apache-incubator-kvrocks).**

4,339 stars · 633 forks · C++ · Apache-2.0

## Links

- GitHub: https://github.com/apache/incubator-kvrocks
- Homepage: https://kvrocks.apache.org/
- awesome-repositories: https://awesome-repositories.com/repository/apache-incubator-kvrocks.md

## Description

Kvrocks is a disk-based NoSQL database and distributed key-value store that leverages the RocksDB storage engine to persist large datasets to physical disk. It is designed to be a Redis-compatible database, utilizing the standard Redis communication protocol to ensure interoperability with existing client libraries and tools.

The project distinguishes itself by combining a disk-persistent storage model with advanced retrieval capabilities, including vector search for k-nearest neighbor queries, full-text search indexing, and geospatial query execution. It supports distributed clustering with slot-based data distribution and topology management to enable horizontal scaling and high availability.

The system covers a wide array of data storage types, including JSON documents, streams, sorted sets, hash maps, and bitmaps. It provides comprehensive data management tools such as atomic transactions, log-based replication, and probabilistic data structures for cardinality estimation and membership checking. Additionally, it includes server-side scripting, pub/sub messaging, and detailed monitoring for server health and storage engine performance.

## Tags

### Data & Databases

- [Distributed Key-Value Stores](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-persistence-storage/specialized-storage-engines/distributed-key-value-stores.md) — A scalable data system that manages data across multiple nodes using cluster topology and slot migration.
- [Disk-Based Key-Value Storage](https://awesome-repositories.com/f/data-databases/disk-based-key-value-storage.md) — Persists key-value data to disk using RocksDB to reduce memory costs while maintaining high-performance access. ([source](https://cdn.jsdelivr.net/gh/apache/incubator-kvrocks@main/README.md))
- [Unbounded Key-Value Stores](https://awesome-repositories.com/f/data-databases/in-memory-data-stores/disk-backed-stores/bounded-disk-key-value-stores/unbounded-key-value-stores.md) — Provides a disk-based key-value store using RocksDB to manage datasets that exceed physical memory.
- [LSM-Tree Storage Engines](https://awesome-repositories.com/f/data-databases/storage-engines/key-value/log-structured-merge-trees/lsm-tree-key-value-stores/lsm-tree-storage-engines.md) — Leverages a Log-Structured Merge-Tree storage engine to persist large datasets to physical disk.
- [Stream Append Operations](https://awesome-repositories.com/f/data-databases/append-only-storage-engines/stream-append-operations.md) — Records events with unique millisecond-sequence identifiers in an append-only data structure. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [Cluster Request Redirections](https://awesome-repositories.com/f/data-databases/cluster-request-redirections.md) — Sends the correct node address to clients when a request hits a node not serving the target slot. ([source](https://kvrocks.apache.org/docs/cluster))
- [Cluster Topology Management](https://awesome-repositories.com/f/data-databases/cluster-topology-management.md) — Configures node roles and slot assignments across a cluster using versioned updates to ensure consistency. ([source](https://kvrocks.apache.org/docs/cluster))
- [Embedded Storage Engines](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-persistence-storage/data-storage/file-based-storage/embedded-storage-engines.md) — Implements a high-performance storage layer based on the RocksDB embedded engine for disk-based persistence.
- [Disk-Based NoSQL Databases](https://awesome-repositories.com/f/data-databases/disk-based-nosql-databases.md) — Provides a disk-persistent NoSQL storage system for large key-value and JSON datasets.
- [Distributed Database Clusters](https://awesome-repositories.com/f/data-databases/distributed-database-clusters.md) — Organizes database nodes into a coordinated cluster for high availability and horizontal scaling.
- [Dynamic Slot Migrations](https://awesome-repositories.com/f/data-databases/dynamic-slot-migrations.md) — Moves data ownership between nodes in real time to enable scaling and load balancing. ([source](https://kvrocks.apache.org/docs/cluster))
- [Hash Field Management](https://awesome-repositories.com/f/data-databases/hash-field-management.md) — Manages field-value pairs within hash structures with support for field-level expiration and numeric increments. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Hash Map Storage](https://awesome-repositories.com/f/data-databases/hash-map-storage.md) — Maps keys to field-value pairs by splitting entries to maintain performance as maps grow. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [Incremental Data Synchronization](https://awesome-repositories.com/f/data-databases/incremental-data-synchronization.md) — Synchronizes data from master nodes to replicas through incremental updates and full backup restoration. ([source](https://kvrocks.apache.org/docs/replication))
- [JSON Document Storage](https://awesome-repositories.com/f/data-databases/json-document-storage.md) — Saves semi-structured data using JSON or CBOR encoding for native document persistence. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [JSON Document Manipulation](https://awesome-repositories.com/f/data-databases/json-document-stores/json-document-manipulation.md) — Provides logic for updating property values and modifying arrays within stored JSON documents. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Key-Value Pair Managers](https://awesome-repositories.com/f/data-databases/key-value-pair-managers.md) — Provides a primary storage mechanism for mapping keys to raw values with support for automatic expiration. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [Ordered List Management](https://awesome-repositories.com/f/data-databases/ordered-list-management.md) — Maintains sequences of elements with support for blocking pops and atomic transfers. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Redis Cluster Deployments](https://awesome-repositories.com/f/data-databases/redis-cluster-deployments.md) — Organizes multiple nodes into a cluster topology using a centralized controller to manage roles and slots. ([source](https://kvrocks.apache.org/docs/cluster/))
- [Client-Compatible Topologies](https://awesome-repositories.com/f/data-databases/redis-cluster-deployments/client-compatible-topologies.md) — Creates a cluster topology that works with standard cluster clients without requiring modifications. ([source](https://kvrocks.apache.org/docs/cluster))
- [Redis Compatible Stores](https://awesome-repositories.com/f/data-databases/redis-compatible-stores.md) — Implements the Redis protocol to ensure seamless compatibility with existing ecosystem clients and proxy tools. ([source](https://kvrocks.apache.org/docs/cluster/))
- [String Value Assignments](https://awesome-repositories.com/f/data-databases/storage-engines/key-value/string-value-assignments.md) — Manages the assignment of string values to keys, including atomic updates and manipulation. ([source](https://kvrocks.apache.org/docs/supported-commands/))
- [String Data Structures](https://awesome-repositories.com/f/data-databases/string-data-structures.md) — Implements primitive string and binary storage with support for atomic increments and substring operations. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Backup and Recovery](https://awesome-repositories.com/f/data-databases/backup-and-recovery.md) — Creates full and incremental copies of the dataset for disaster recovery and persistence. ([source](https://kvrocks.apache.org/docs/category/operation))
- [Bitmap Storage](https://awesome-repositories.com/f/data-databases/bitmap-storage.md) — Implements fragmented storage for large sequences of bits to enable efficient arbitrary index writing. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [Point-in-Time Database Snapshots](https://awesome-repositories.com/f/data-databases/data-governance-modeling/data-management-governance/backup-recovery-systems/database-backup-restoration/point-in-time-database-snapshots.md) — Generates point-in-time snapshots of the database to disk via complete or incremental copies. ([source](https://kvrocks.apache.org/docs/backup))
- [Key Expiration Management](https://awesome-repositories.com/f/data-databases/data-governance-modeling/data-management-governance/data-lifecycle-retention/data-lifecycle-management/key-expiration-policies/access-link-expiration/content-expiration-policies/storage-key-expirations/key-expiration-management.md) — Handles key lifecycles using existence checks and time-to-live settings to control automatic eviction. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Atomic Transaction Execution](https://awesome-repositories.com/f/data-databases/data-integration-synchronization/data-integration/database-integrations/atomic-transaction-execution.md) — Groups multiple commands into atomic blocks to ensure they either execute completely or not at all. ([source](https://kvrocks.apache.org/docs/supported-commands/))
- [Full Text Search](https://awesome-repositories.com/f/data-databases/full-text-search.md) — Creates and queries full-text search indexes to find documents matching specific text filters and keywords. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Full-Text Search Indexes](https://awesome-repositories.com/f/data-databases/full-text-search-indexes.md) — Creates and queries searchable full-text indexes to retrieve documents using complex text filters.
- [Geospatial Indexes](https://awesome-repositories.com/f/data-databases/geospatial-query-mapping/geospatial-indexes.md) — Implements geospatial indexing to calculate distances and find members within a specific radius. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Centroid-Based Approximations](https://awesome-repositories.com/f/data-databases/grouped-aggregations/quantile-digest-aggregators/quantile-rank-estimators/centroid-based-approximations.md) — Clusters data points into centroids to provide real-time approximations of quantiles for analytics. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [Sorted Integer Sets](https://awesome-repositories.com/f/data-databases/integer-hash-sets/sorted-integer-sets.md) — Stores and retrieves ranges of integers in a sorted collection for numerical range queries. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Binary Document Encodings](https://awesome-repositories.com/f/data-databases/json-document-storage/binary-document-encodings.md) — Saves structured JSON data using binary encoding to reduce disk usage and improve parsing speed.
- [Optimistic Locking](https://awesome-repositories.com/f/data-databases/optimistic-locking.md) — Provides atomic blocks that only execute if the underlying data has not changed since being read.
- [Probabilistic Data Structures](https://awesome-repositories.com/f/data-databases/probabilistic-data-structures.md) — Uses sketches and filters to estimate set cardinality and membership with minimal memory overhead.
- [Probabilistic Membership Filters](https://awesome-repositories.com/f/data-databases/probabilistic-data-structures/probabilistic-membership-filters.md) — Provides memory-efficient probabilistic filters to verify set membership with minimal false positives. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [Real-Time Data Streaming](https://awesome-repositories.com/f/data-databases/real-time-data-streaming.md) — Manages chronological event logs and consumer groups for asynchronous data processing and messaging.
- [Complex Search Querying](https://awesome-repositories.com/f/data-databases/search-indexing/complex-search-querying.md) — Executes complex queries using search-compatible syntax or a subset of SQL SELECT statements for record retrieval. ([source](https://kvrocks.apache.org/docs/kvrocks-search))
- [Secondary Indexes](https://awesome-repositories.com/f/data-databases/secondary-indexes.md) — Provides secondary indexes on numeric and vector fields to enable complex queries over large datasets. ([source](https://kvrocks.apache.org/docs/kvrocks-search))
- [Server-Side Scripting](https://awesome-repositories.com/f/data-databases/server-side-scripting.md) — Executes scripts on the server to perform complex logic atomically and reduce network round-trips. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Sorted Sets](https://awesome-repositories.com/f/data-databases/sorted-sets.md) — Maintains a set of unique integer members sorted in ascending order. ([source](https://kvrocks.apache.org/community/data-structure-on-rocksdb/))
- [Compaction Management](https://awesome-repositories.com/f/data-databases/storage-engines/b-tree/log-structured-merge-trees/compaction-management.md) — Automatically merges and rewrites sorted runs in the background to reclaim space and maintain read performance.
- [Time Series Management](https://awesome-repositories.com/f/data-databases/time-series-management.md) — Supports the storage and querying of timestamped samples with built-in retention policies and range aggregation. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Unordered Unique Collection Management](https://awesome-repositories.com/f/data-databases/unordered-unique-collection-management.md) — Manages collections of unique elements and performs set-theoretic operations like intersections, unions, and differences. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Vector Search](https://awesome-repositories.com/f/data-databases/vector-search.md) — Provides high-dimensional vector search capabilities including k-nearest neighbor and range queries.

### DevOps & Infrastructure

- [Capacity Scaling](https://awesome-repositories.com/f/devops-infrastructure/cluster-node-management/capacity-scaling.md) — Redistributes data slots between nodes to balance load or increase total storage capacity. ([source](https://kvrocks.apache.org/docs/cluster/))
- [Primary-Replica Log Replication](https://awesome-repositories.com/f/devops-infrastructure/high-availability-clusters/primary-replica-log-replication.md) — Synchronizes state from master to replica nodes by streaming incremental change logs and snapshots.

### Networking & Communication

- [Message Stream Consumer Groups](https://awesome-repositories.com/f/networking-communication/message-stream-handlers/message-stream-consumer-groups.md) — Appends and consumes entries in a log-like structure with support for parallel processing via consumer groups. ([source](https://kvrocks.apache.org/docs/supported-commands))
- [Pub-Sub Messaging](https://awesome-repositories.com/f/networking-communication/pub-sub-messaging.md) — Implements a publish-subscribe messaging system using named channels for asynchronous client communication. ([source](https://kvrocks.apache.org/docs/supported-commands))

### Software Engineering & Architecture

- [Hash Slot Sharding](https://awesome-repositories.com/f/software-engineering-architecture/hash-based-data-distribution/hash-slot-sharding.md) — Partitions the keyspace into fixed hash slots assigned to nodes for horizontal scaling.

### Part of an Awesome List

- [Vector Databases and Search](https://awesome-repositories.com/f/awesome-lists/data/vector-databases-and-search.md) — Provides a specialized engine for storing and performing similarity searches on high-dimensional vector embeddings.
- [Database Systems](https://awesome-repositories.com/f/awesome-lists/data/database-systems.md) — Distributed NoSQL database compatible with Redis.
- [Databases](https://awesome-repositories.com/f/awesome-lists/data/databases.md) — Distributed key-value store compatible with Redis.

### Operating Systems & Systems Programming

- [Bitwise Logic Operations](https://awesome-repositories.com/f/operating-systems-systems-programming/bit-level-binary-parsers/bit-level-slicing/bitwise-logic-operations.md) — Executes low-level bitwise logic operations including AND, OR, and XOR across stored keys. ([source](https://kvrocks.apache.org/docs/supported-commands))