1 dépôt
Mechanisms for splitting data into independent blocks to be compressed or decompressed simultaneously across CPU cores.
Distinct from Thread Scaling: Existing candidates focus on network packets or graphics commands; this is for general data compression throughput.
Explore 1 awesome GitHub repository matching operating systems & systems programming · Parallel Block Processing. Refine with filters or upvote what's useful.
Zstandard is a lossless data compression library and archive format designed for high compression ratios and fast real-time processing. It functions as a real-time data compressor and multi-threaded compression engine capable of distributing workloads across multiple CPU cores to increase throughput. The system features a dictionary-based compressor that trains on sample data to improve the compression ratio and speed of small files. It also provides long distance pattern matching to identify repeated sequences across large files. The library covers a broad range of capabilities including st
Distributes independent chunks of data to multiple CPU cores for simultaneous compression or decompression.