buraksezer/consistent
Consistent
Consistent is a Go library that implements consistent hashing with bounded loads to distribute data keys across nodes in a distributed system. It provides a mechanism for mapping keys to cluster members that minimizes data movement during membership changes while preventing performance hotspots.
The library distinguishes itself by enforcing strict capacity limits on individual nodes, ensuring that no single member becomes overwhelmed by excessive key assignments. It supports virtual node mapping to distribute physical capacity across the hash ring, allowing for granular control over load balancing and resource utilization.
The project covers a broad range of distributed system requirements, including the ability to inject custom hashing algorithms to optimize data locality. It also facilitates high availability by identifying multiple candidate nodes for each key, enabling reliable data replication and redundancy across the cluster.
Features
- Consistent Hashing - Provides a Go implementation of consistent hashing with bounded loads for distributed systems.
- Load Balancing Algorithms - Restrict the number of partitions assigned to each node to prevent individual members from becoming overloaded during high traffic periods or intensive cluster maintenance operations.
- Consistent Hashing Implementations - Distributes physical node capacity across multiple points on the hash ring to improve load balancing and granular control.
- Data Partitioning Strategies - Maps keys to cluster members to minimize data movement and prevent performance hotspots.
- Replica Candidate Locators - Locate multiple candidate nodes for every data key to facilitate reliable data replication and ensure high availability if a primary node becomes unavailable or goes offline.
- Load Balancing Strategies - Implements load balancing strategies that cap key assignments to prevent node overloading.
- Bounded Load Policies - Provides mechanisms to enforce strict capacity limits on individual nodes to prevent performance hotspots.
- Algorithmic Load Limiters - Restricts the total number of keys assigned to any single node to prevent performance bottlenecks.
- Distributed Systems - Consistent hashing implementation with bounded loads.
- Distributed Systems - Consistent hashing algorithm library.
- High Availability Configurations - Facilitates high availability by identifying multiple candidate nodes for data storage.
- Distributed Key-Value Stores - Assign data keys to specific nodes using consistent hashing to ensure balanced storage distribution and minimize data movement when the total number of cluster nodes changes.
- Custom Hashing Implementations - Allows injection of custom hashing algorithms to optimize data locality and key distribution.
- Relocation Metrics - Measure the percentage of data keys that move between nodes during cluster membership changes to maintain balance and reduce unnecessary data transfer across the network infrastructure.
- Distributed Systems & Scalability - Manages cluster membership and resource allocation to maintain performance during scaling.
