10 dépôts
Handling of binary data sequences directly in memory to avoid temporary disk storage.
Distinguishing note: Closest candidates are for network protocol parsing or DB storage, not generic memory-based file I/O.
Explore 10 awesome GitHub repositories matching operating systems & systems programming · In-Memory Byte Stream Processing. Refine with filters or upvote what's useful.
MediaMTX is a multi-protocol media server designed for routing, proxying, and recording real-time audio and video streams. It functions as a programmable media router and a gateway between streaming standards such as RTMP, RTSP, and WebRTC, enabling the conversion of live media between different protocols. The server distinguishes itself through on-the-fly format transmuxing and protocol-agnostic routing, which decouples input and output protocols via an internal media bus. It features a programmable automation system that executes external shell commands via event hooks triggered by client c
Distributes a single incoming media source to multiple concurrent readers via in-memory packet duplication.
pypdf is a Python library for parsing, manipulating, and generating PDF documents. It provides high-level operations for document processing, such as merging multiple files into one or splitting a single document into smaller files. The project includes specialized tools for managing interactive elements, including the creation and modification of annotations, hyperlinks, and form fields. It also supports advanced metadata management, allowing for the extraction and modification of standard document properties and XML-based XMP metadata. Beyond basic structural changes, the library covers pa
Performs read and write operations directly on byte sequences to bypass the need for temporary disk files.
Okio is a Java I/O library providing a set of tools for efficient byte-stream processing and file system operations. It functions as a buffered byte stream handler and streaming data transformer, utilizing a cross-platform file system API to manage data movement. The project is distinguished by its use of pooled mutable byte buffers that treat sequences as queues to reduce memory copying and garbage collection churn. It further decouples file operations from the host operating system through an abstraction-based file system, allowing for consistent path manipulation and atomic operations acro
Manages bytes as growable queues to significantly reduce memory copying and GC churn during I/O.
QOI is a lossless image codec and encoding standard designed for high-speed compression and decompression of raw pixel data. It provides a toolkit for translating raw image buffers into a compact format and back into pixel representations without any loss of quality. The implementation focuses on fast image encoding and decoding, enabling the rapid conversion of compressed image data back into raw pixels. It also supports image format conversion to ensure compatibility across different software systems and hardware.
Implements linear byte-stream processing to avoid complex memory overhead during image encoding and decoding.
rustls is a modern implementation of the Transport Layer Security protocol written in the Rust programming language. It serves as a cryptographic transport layer for establishing encrypted connections between clients and servers to ensure data privacy and integrity. The library features a pluggable crypto provider framework, allowing the substitution of cryptographic primitive implementations to meet specific platform architecture or regulatory compliance requirements. It provides capabilities for secure server configuration to handle encrypted incoming connections and secure client connecti
Implements mutable byte buffers to handle network streams and minimize memory copying.
The Rust RFCs repository is the formal home for the Rust language evolution process, housing the structured design documents and community review mechanisms that govern changes to the Rust programming language, its compiler, and its standard library. It defines the complete lifecycle for proposing, discussing, and implementing substantial changes through RFC documents, from initial submission and community feedback through final comment periods and sub-team sign-offs. The repository codifies the governance and collaboration processes that shape Rust's development, including mechanisms for com
Defines the read_exact method on the Read trait for guaranteed buffer filling.
protobuf-net est un framework de sérialisation binaire et une bibliothèque .NET qui implémente la spécification Protocol Buffers. Il fonctionne comme un outil de sérialisation basé sur un schéma et un mappeur de données "contract-first", convertissant des graphes d'objets complexes en un format binaire compact pour le stockage et la transmission réseau. La bibliothèque permet l'échange de données multiplateforme et la mise en réseau haute performance en réduisant la taille des charges utiles et le temps de traitement. Elle prend spécifiquement en charge la persistance de la hiérarchie des objets, permettant la sérialisation et la reconstruction de structures de classes complexes incluant l'héritage et les types dérivés. Le projet couvre des capacités fondamentales, notamment la sérialisation et la désérialisation de données binaires, la génération de classes pilotée par schéma et la gestion de l'héritage de types via des identifiants uniques. Il fournit des mécanismes pour mapper les structures de données via des attributs statiques ou des configurations dynamiques au runtime.
Processes binary data directly through sequential byte streams to minimize memory overhead during large transfers.
Janet est un langage de programmation dynamique basé sur Lisp présentant une machine virtuelle de bytecode basée sur des registres et un moteur de scripting intégrable. Il fonctionne comme un runtime de concurrence basé sur des fibres et inclut un moteur d'analyse basé sur des Parsing Expression Grammars (PEG). Le projet se distingue par sa capacité à être intégré dans des applications C ou C++ via une interface d'en-tête minimale. Il utilise un système de macros de style Lisp pour la transformation de code à la compilation et emploie l'héritage de table basé sur des prototypes pour un comportement orienté objet. Le runtime couvre un large ensemble de capacités, incluant la gestion d'IO asynchrone via une boucle d'événements non bloquante, l'interopérabilité de bibliothèque native via une interface de fonction étrangère (FFI) et un traitement de texte complet utilisant des grammaires PEG. Il fournit également des outils pour l'automatisation système, tels qu'une boucle read-eval-print (REPL), un système de module pour la résolution de symboles et des utilitaires pour la communication par socket réseau et la gestion du système de fichiers. L'environnement inclut des outils de diagnostic pour le débogage de l'exécution de bytecode et peut bundler le code source en exécutables binaires autonomes.
Provides growable byte buffers to store and modify sequences of bytes representing binary data.
python-magic is a C-binding wrapper that provides a Python interface for the libmagic system library. It functions as a file signature analyzer and MIME type detector, identifying file formats by comparing header bytes against a database of known binary signatures. The library enables the identification of file types from both file paths and raw data buffers. It supports custom file signature matching through the injection of user-provided magic databases, allowing for the detection of specialized or proprietary formats. The project covers binary data analysis and MIME type mapping to transl
Provides the ability to analyze binary data sequences directly in memory to identify file types without disk persistence.
Kotlinx-io is a multiplatform library designed for input and output operations, providing a unified interface for streaming data, managing byte buffers, and interacting with local filesystems. It serves as a cross-platform abstraction layer that standardizes how applications handle data movement across different operating systems and hardware architectures. The library distinguishes itself by providing high-performance tools for both mutable and immutable byte sequences. It utilizes segmented memory pools and direct memory access to minimize allocation overhead and prevent unnecessary data co
Modifies dynamic byte sequences using efficient memory segments to reduce allocation overhead.