probelabs/goreplay
Goreplay
Goreplay is an HTTP traffic mirroring tool designed to capture live network traffic from production environments and replay it into test systems for validation. It includes a specialized Kubernetes traffic capturer that operates as a daemonset to mirror traffic from specific pods using label selectors and namespace filters, alongside a TCP traffic recorder for intercepting raw network packets.
The project features a Kafka traffic pipeline for streaming captured payloads to topics or ingesting messages for playback, and an HTTP request transformer to mask sensitive data or rewrite headers and URLs during the replay process. It supports a pluggable architecture for traffic routing and steering, allowing data to be directed between various input sources and output destinations.
Capabilities cover a broad range of traffic processing and storage, including payload modification via regular expressions, rate limiting, and hash-based sampling. Captured data can be archived to local file storage, Amazon S3 buckets, or indexed in Elasticsearch for behavioral analysis. The replay engine supports HTTP, TCP, and binary protocols, with the ability to respect original timing and delays.
Features
- Network Traffic Replay Tools - Captures live HTTP traffic from production and re-injects it into target environments for system validation.
- HTTP Traffic Shadowing - Mirrors live HTTP traffic from a production environment to a test system to validate changes with real world data.
- Traffic Capture Frameworks - Listens for incoming HTTP requests on a specified address and captures payloads for redirection and mirroring.
- Pod Traffic Mirroring - Intercepts network messages from specific pods using label selectors and namespace filters within a Kubernetes cluster.
- Payload Rewriters - Rewrites URLs, headers, and parameters using regular expressions and hashing to transform production traffic.
- Traffic Steering Plugins - Directs captured network data between input sources and output destinations using a pluggable architecture.
- Network Testing - Captures and mirrors traffic from specific pods using label selectors to test microservices within a Kubernetes cluster.
- Traffic Playback Engines - Replays recorded network requests while respecting original timing and delays to simulate real-world traffic patterns.
- Request and Response Transformers - Provides a transformation engine to modify URL paths, headers, and body parameters during traffic replay.
- Traffic-to-HTTP Ingestors - Consumes messages from Kafka topics and transforms them into HTTP payloads for playback into a target environment.
- Elasticsearch Exporters - Indexes captured network traffic into an Elasticsearch instance using custom URIs and authentication.
- Kafka Stream Exporters - Streams captured network payloads to Kafka topics as raw strings or structured JSON objects.
- Traffic Bridge Pipelines - Provides a data bridge that streams captured network payloads to Kafka topics or ingests messages for HTTP playback.
- Payload Pattern Rewriting - Modifies URLs, headers, and body parameters on the fly using pattern matching to sanitize or redirect traffic.
- Traffic Throttling - Controls the data processing rate for plugins to maintain system stability and prevent destination overload.
- External Command Piping - Routes captured traffic through external system commands or binaries for custom authentication and filtering.
- File-Based Replayers - Loads previously recorded HTTP requests from files and replays them while maintaining original request ordering.
- Non-HTTP Protocol Replay - Processes and mirrors non-HTTP traffic, such as Thrift or ProtocolBuffers, to test specialized backend services.
- TCP Traffic Replayers - Sends captured network messages to a remote server using a TCP client with worker pools.
- Timing-Preserving Replayers - Emits recorded network traffic while respecting the original timing and delays between requests.
- TCP Proxies - Provides a utility for intercepting, inspecting, and recording raw TCP traffic streams.
- TCP Session Management - Duplicates raw TCP sessions to maintain the exact state and sequence of network connections.
- Traffic Filters - Excludes specific requests or responses from replay based on criteria like HTTP methods or headers.
- Traffic Routing Tools - Sends and receives captured network traffic from Kafka topics using JSON format with SASL and TLS encryption.
- TCP and UDP Forwarders - Sends raw network payloads to remote addresses using secure or insecure TCP connections.
- Traffic Distribution - Distributes incoming network traffic equally among multiple output destinations to balance the load.
- Data Masking - Removes or replaces personally identifiable information in captured HTTP requests to protect privacy during testing.
- Rate Limiting - Caps the rate of processed requests using fixed counts or percentages to protect target systems.
- Traffic Modification Plugins - Directs network data between input sources and output destinations via a pluggable architecture for flexible steering.
- Multi-Destination Forwarding - Replicates and sends captured network traffic to multiple independent HTTP endpoints simultaneously.
- Traffic Analysis - Captures and stores HTTP payloads in Elasticsearch or S3 for long term archival and behavioral analysis.
- Response Validation - Validates that replayed traffic output matches original production responses by mapping related requests and responses.
- Network and Security - Tool for capturing and replaying live HTTP traffic.
