cloudflare/pingora
Pingora
Pingora is a Rust-based framework for building high-performance network services, including HTTP reverse proxies, layer seven load balancers, and TLS termination proxies. It serves as an asynchronous network library designed to intercept and route HTTP, gRPC, and WebSocket traffic between clients and upstream backend servers.
The project enables zero-downtime service updates by handing over listening sockets between processes during binary or configuration upgrades. It utilizes a programmable multi-phase pipeline to modify request and response bodies and headers, and it provides a pluggable TLS backend for managing secure encryption and client certificate validation.
The framework covers a broad range of traffic management capabilities, including consistent hashing for load balancing, backend health monitoring, and failover strategies. It also implements high-performance content caching with in-memory storage and maintains upstream connection pools to reduce handshake latency.
Features
- Application Layer Load Balancing - Distributes network traffic across backend servers using application-layer logic, consistent hashing, and health checks.
- Load Balancing Strategies - Implements a high-performance L7 load balancer using consistent hashing and failover strategies.
- Traffic Routing - Provides a framework for building high-performance network proxies that route HTTP, gRPC, and WebSocket traffic.
- Zero-Downtime Binaries Upgrades - Enables zero-downtime service updates by handing over listening sockets between processes during binary upgrades.
- SSL Termination Proxies - Acts as a network gateway that handles secure encryption and decryption of traffic to offload cryptographic overhead from backends.
- Zero-Downtime Upgrades - Performs graceful upgrades by handing over listening sockets from an old process to a new version.
- Asynchronous Network Frameworks - Serves as an asynchronous network foundation for building high-throughput, event-driven protocol servers and clients.
- HTTP Proxies - Acts as an intermediary for HTTP/1 and HTTP/2 traffic between clients and destination servers.
- HTTP Request Customization - Allows the modification of HTTP request headers and metadata through a multi-phase interceptor pipeline.
- HTTP Traffic Manipulation Tools - Features a programmable multi-phase pipeline to intercept and modify HTTP request and response bodies, headers, and metadata.
- Load Balancers - Implements a Layer 7 load balancer that distributes requests across multiple peers using consistent hashing and health monitoring.
- Response Body Modifiers - Intercepts and alters the content of response bodies sent from upstream servers before delivery to the client.
- TLS Termination - Handles TLS termination at the network edge, managing encryption and decryption for incoming traffic.
- Custom Proxy Frameworks - Provides a programmable framework for building high-performance network proxies with custom logic for HTTP, gRPC, and WebSocket traffic.
- Traffic Distribution - Distributes network traffic across upstream peers using algorithms like round robin and consistent hashing.
- Zero-Downtime Socket Handovers - Enables zero-downtime updates by handing over listening network sockets from an old process to a new binary.
- Rust Network Frameworks - Provides a high-performance framework written in Rust for building network services and load balancers.
- Network Encryption - Secures communications between clients and servers using industry standard TLS encryption libraries.
- Consistent Hashing - Uses consistent hashing algorithms to distribute network traffic across upstream peers and ensure stable mapping.
- Health Checks - Performs periodic health checks on upstream nodes to remove unhealthy peers from the active routing pool.
- Microservices Health Monitoring - Monitors the operational status of backend microservices to ensure traffic only reaches healthy instances.
- Asynchronous Request Processing - Implements non-blocking request processing using an asynchronous Rust runtime for high-concurrency network services.
- HTTP Request Handling - Routes incoming HTTP requests to upstream peers while allowing header modification before backend transmission.
- Reverse Proxies - Functions as a high-performance reverse proxy for routing HTTP and gRPC traffic to upstream backend services.
- Request Body Handling - Intercepts and transforms the content of incoming request bodies through a programmable pipeline.
- Request Pipelines - Features a programmable multi-phase pipeline of interceptors to modify requests and responses during their lifecycle.
- API Response Caching - Caches HTTP response content in memory to serve subsequent requests faster with background updates.
- In-Memory Caching - Implements high-performance in-memory caching of server responses to reduce backend load and latency.
- In-Memory Caches - Provides an in-memory cache using advanced replacement algorithms to maximize retrieval speed and throughput.
- Cache Replacement Algorithms - Provides specialized replacement algorithms for in-memory caching to maximize retrieval speed and memory efficiency.
- Configuration Hot-Reloading - Implements mechanisms to update system state and settings dynamically without interrupting active network connections.
- Graceful Shutdowns - Provides grace periods and timeouts to ensure active requests are processed before service shutdown.
- Regional Failover - Detects upstream failures and automatically reroutes traffic to healthy alternative endpoints.
- Zero-Downtime Deployments - Enables zero-downtime service updates by handing over listening sockets between processes during binary or configuration upgrades.
- Connection Pooling - Maintains a pool of persistent TCP connections to upstream servers to eliminate repeated handshake latency.
- HTTP Request Piping - Streams request bodies directly into chains of subrequests and propagates responses through a state machine.
- Protocol Feature Forwarders - Forwards specialized protocol traffic, including gRPC and WebSockets, ensuring reliable communication for non-standard protocols.
- Retry Policies - Configures maximum retry attempts for failed upstream requests to improve overall system resilience.
- State-Machine Relayers - Pipes request and response payloads through a state-machine architecture to manage data flow integrity.
- Pluggable TLS Backends - Provides a pluggable TLS backend that abstracts encryption logic to allow switching between security libraries.
- Rate Limiting - Restricts the volume of incoming requests based on defined thresholds to prevent system overload.
- API Gateways - Library for building fast, reliable network services.
- Networking & Connectivity - Library for building fast, reliable network services.
- Web Servers & Proxies - Library for building fast network services.
- Network Frameworks - Asynchronous framework for building programmable network services.
- Web and File Transfer Tools - Asynchronous framework for building programmable network services.
- Security & Privacy - Library for building network services.
