klauspost/compress

Features

• High-Performance Compression Libraries - Provides high-performance Go implementations of Zstandard, Snappy, and Huffman coding.
• Data Compression and Decompression - Reduces data size for storage or transmission and restores it to its original form using optimized algorithms.
• Compression Libraries - A comprehensive collection of high-performance Go packages for data compression and decompression.
• Dictionary-Based Compression - Uses pre-trained data samples as lookup tables to improve compression ratios for small or similar data blocks.
• Compression Dictionary Generation - Analyzes sample data sets to generate optimized dictionaries that improve compression ratios for common data formats.
• Huffman Decoding - The library decodes compressed blocks back into their original form using specialized decoding tables and stateless decoders.
• Snappy Compression - Implements S2 decompression that maintains compatibility with standard Snappy-encoded content.
• S2 Extensions - Provides S2 compression, a high-throughput extension of Snappy with concurrent processing support.
• Zstandard Compression - Implements the Zstandard algorithm with configurable settings to balance speed and compression ratios.
• Huffman Coding Implementations - Implements a specialized Huffman codec optimized for modern CPU architectures to minimize byte counts.
• Huffman Codecs - Encodes independent data blocks using a Huffman codec optimized for modern CPUs to minimize byte counts for similar values.
• Parallel Compression Tools - Implements a parallel compression framework that utilizes multiple threads to increase stream throughput.
• Compressed Index Seeking - Creates a coordinate map of compressed offsets to allow random access without full stream decompression.
• Parallel Implementations - Provides a parallelized implementation of Gzip to distribute workloads across multiple CPU cores.
• Small Data Compression Optimization - Encodes and decodes payloads under 100KB using a non-streaming interface to avoid stream overhead.
• Finite State Entropy Decompression - Restores compressed byte blocks to their original form using the inverse of the entropy encoding process.
• Finite State Entropy Compression - Compresses byte blocks using near-optimal symbol encoding to reduce size for inputs with many similar values.
• Payload Optimization - Improves bandwidth efficiency by optimizing the size of compressed request and response payloads.
• Allocation-Free Decoding - Implements decompression using pre-allocated buffers to eliminate garbage collection overhead during data restoration.
• Allocation-Free Decoders - Offers an optimized Zstandard decoder that eliminates memory allocations during data restoration.
• Finite State Entropy Coding - Uses near-optimal symbol encoding to compress byte blocks based on the probability of value occurrences.
• Side-Channel Attack Mitigations - Prevents BREACH side-channel attacks by injecting random jitter and padding into compressed server responses.
• Stateless Buffer Management - Executes compression without maintaining internal state between write calls to reduce memory overhead for concurrent tasks.
• Compression Middleware - Provides middleware for automatically compressing and decompressing HTTP response bodies.
• Compression Utilities - Optimized and high-performance compression packages.

Star history