19 Repos
Libraries for archiving and reducing data size using various algorithms.
Explore 19 awesome GitHub repositories matching part of an awesome list · Data Compression. Refine with filters or upvote what's useful.
Zstandard is a lossless data compression library and archive format designed for high compression ratios and fast real-time processing. It functions as a real-time data compressor and multi-threaded compression engine capable of distributing workloads across multiple CPU cores to increase throughput. The system features a dictionary-based compressor that trains on sample data to improve the compression ratio and speed of small files. It also provides long distance pattern matching to identify repeated sequences across large files. The library covers a broad range of capabilities including st
Reduces the size of data streams in real-time to balance high throughput and efficient compression ratios.
This project provides a lossless compression algorithm and a byte-oriented compression library designed for high-speed data reduction and maximum decompression speed. It functions as a stream-oriented compression engine, a software library for encoding and decoding data blocks, and a command-line tool for managing interoperable compressed frames. The system distinguishes itself through the use of predefined pattern dictionaries to improve compression ratios for small data sets and small packets. It supports multiple processing modes, including high-speed block compression for minimal latency
Provides a lossless compression algorithm ensuring perfect recovery of original content with tunable speed and ratio.
This project is a comprehensive educational resource and curriculum focused on site reliability engineering, distributed systems, and infrastructure operations. It provides technical guides, a systems engineering course, and instructional manuals designed to teach the principles of managing large-scale computing environments. The curriculum covers high-level architectural design for scalability and resilience, including fault-tolerant infrastructure, high-availability patterns, and microservices decomposition. It emphasizes the practical application of site reliability engineering through the
Provides guidance on using libraries and algorithms for archiving and reducing data size.
c3c is the compiler for the C3 programming language, transforming source code into executable binaries, static libraries, or dynamic libraries using an LLVM backend. It implements a system based on result-based error handling, scoped memory pooling, and a semantic macro system. The compiler provides first-class support for hardware-backed SIMD vectors that map directly to processor instructions and enables runtime polymorphism through interface-based dynamic dispatch. The project covers a broad set of low-level capabilities, including manual and pooled memory management, inline assembly inte
Implements data size reduction using Gzip, Deflate, and Zip algorithms.
NuttX ist ein POSIX-konformes Echtzeitbetriebssystem für Mikrocontroller, das von 8-Bit- bis 64-Bit-Architekturen reicht. Es bietet eine deterministische Ausführungsumgebung mit einem Echtzeit-Task-Scheduler und einem POSIX-Embedded-Kernel, um die Portabilität von Code über diverse Hardware-Ziele hinweg sicherzustellen. Das Projekt zeichnet sich durch eine umfassende Hardware-Abstraktionsschicht aus, die standardisierte Treiber für I2C, SPI, CAN und USB für verschiedene Halbleiter-Chipsätze bereitstellt. Es verfügt zudem über einen Embedded-Netzwerk-Stack mit Unterstützung für TCP, UDP, IPv4 und IPv6 sowie industrielle Protokolle wie Modbus und DroneCAN. Das System deckt ein breites Spektrum an Funktionen ab, einschließlich fortschrittlichem Speichermanagement mit On-Demand-Paging, sicheren kryptografischen Operationen und einem virtuellen Dateisystem zur Verwaltung unterschiedlicher Speichertypen. Es integriert zudem High-Level-Schnittstellen für grafische Benutzeroberflächen, Audiosynthese sowie die Ausführung mehrerer Skriptsprachen und WebAssembly-Workloads. Die Umgebung wird mittels Make oder CMake für spezifische Board-Konfigurationen und Halbleiter-Chipsätze erstellt.
Reduces data size using LZF or zlib compression libraries to optimize storage and bandwidth.
Synopse mORMot 1 ORM/SOA/MVC framework - Please upgrade to mORMot 2 !
High-speed compression units featuring multiple industry-standard algorithms.
Documentation
Utilities for reading and writing tar archives.
Fast In-Memory Data Compression Algorithm (header-only C/C++) 480+MB/s compress, 2800+MB/s decompress, ratio% better than LZ4, Snappy, and Zstd@-1
High-speed LZAV compression algorithm implementation.
Abbrevia is a compression toolkit for Delphi and C++Builder.
Advanced toolkit for data compression and archiving.
Набор классов для архивирования.
Work with Zip archives using standard compression methods.
High-performance native compression bindings for .NET.
Lightweight library for creating and reading ZIP files.
Performance benchmarking suite for .NET compression libraries.
High-level abstraction for various compression algorithms.
A simple rust library to read and write Zip archives, which is also my pet project for learning Rust. At the moment you can list the files in a Zip archive, as well as extracting them if they are either stored (uncompressed) or deflated, but I plan to add write support soon.
Support for reading and writing ZIP archives.