twitter/twemproxy
Twemproxy
Twemproxy is a lightweight proxy that routes and distributes requests across multiple Redis and Memcached backend servers. It functions as a protocol translation gateway and distributed cache shard manager, partitioning data across clusters to balance load and storage capacity.
The system acts as a high-availability cache orchestrator, employing health monitoring and automatic server ejection to maintain continuous access to cached data. It integrates with sentinels for dynamic master and replica discovery and utilizes consistent hashing and tag-based key grouping to manage data distribution across server pools.
The proxy handles a broad range of traffic management capabilities, including connection multiplexing, request pipelining, and the forwarding of standard cache operations such as data retrieval, storage, and atomic arithmetic. It also provides operational observability through the exposure of performance metrics and server health statistics.
Features
- Redis Cluster Proxies - Routes Redis requests across multiple backend servers to distribute load and scale storage capacity.
- Cache Proxies - Functions as a lightweight proxy that routes and distributes requests across multiple Redis and Memcached backend servers.
- Data Sharding - Partitions and distributes cache data across multiple nodes using hashing modes to improve scalability.
- Distributed Caching - Partitions data across cache clusters using hashing to balance load and storage capacity.
- Distributed Sharding Architectures - Splits data across multiple backend nodes using consistent hashing to manage large datasets and prevent hotspots.
- Hash-Tag Key Mapping - Routes multiple distinct keys to the same backend server by hashing only a specific tagged portion of the key string.
- Redis Clients - Forwards Redis data operations for strings, hashes, lists, and sets to distributed backend servers.
- Sentinel Management - Coordinates with Redis Sentinel instances to dynamically discover master and replica nodes for automated failover.
- Automatic Server Ejection - Twemproxy removes dead servers from the active rotation after consecutive failures and reintegrates them after a timeout.
- High Availability Clustering - Implements high-availability clustering by coordinating backend nodes and ensuring continuous service availability through failover mechanisms.
- Cache Protocol Proxies - Translates and forwards requests using standard cache protocols to maintain communication between clients and backend cache instances.
- High Availability Routing - Manages server pools and automatically ejects unhealthy nodes to maintain continuous access to cached data.
- Load Balancers - Distributes client requests across multiple server instances to balance the load and scale storage capacity.
- Cache - Provides a network layer that translates and forwards standard cache protocol commands between clients and distributed backends.
- Traffic Proxying - Intercepts and routes network traffic between clients and distributed cache services.
- Consistent Hashing - Distributes keys across a pool of backend servers using circular hash space to minimize remapping during scaling.
- Upstream Health Monitoring - Specifically tracks the availability and status of backend upstream cache servers to manage the server pool.
- Failure Isolation Mechanisms - Isolates unresponsive backend nodes based on failure limits to facilitate system recovery.
- Distributed Load Balancing - Distributes cache traffic across a pool of Memcached servers to ensure high availability and balanced resource usage.
- Backend Server Pools - Organizes backend servers into independent pools via configuration files to isolate different caching environments.
- Node Name Key Mappings - Associates keys with logical node names to allow server relocation without disrupting the hashing ring.
- Proxy Operational Statistics - Fetches operational status and performance metrics through a network socket for external analysis.
- Distributed Script Routing - Executes scripts across backend servers by routing requests to instances where the target keys hash to the same location.
- Connection Multiplexing - Shares a pool of persistent backend connections across multiple client sessions to reduce TCP overhead.
- Protocol-Transparent Forwarding - Forwards cache commands by parsing minimal headers and passing the payload directly to backends.
- Asynchronous Event Loops - Uses a single-threaded asynchronous I/O model to handle thousands of concurrent network connections.
- Request Pipelines - Batches multiple client requests into a single network message to reduce round-trip time and increase throughput.
- Zero-Copy Buffers - Moves data between client and server sockets using pre-allocated memory buffers to minimize CPU copying cycles.
- Database Performance Metrics - Exposes operational statistics and performance counters in JSON format for real-time observability of the cache cluster.
- Caching and Proxies - Fast, lightweight proxy for memcached and redis.
- Database Systems - Fast proxy for memcached and redis clusters.
- Proxy and Cache Tools - Fast proxy for memcached and redis.
