10 repositorios
Handling of binary data sequences directly in memory to avoid temporary disk storage.
Distinguishing note: Closest candidates are for network protocol parsing or DB storage, not generic memory-based file I/O.
Explore 10 awesome GitHub repositories matching operating systems & systems programming · In-Memory Byte Stream Processing. Refine with filters or upvote what's useful.
MediaMTX is a multi-protocol media server designed for routing, proxying, and recording real-time audio and video streams. It functions as a programmable media router and a gateway between streaming standards such as RTMP, RTSP, and WebRTC, enabling the conversion of live media between different protocols. The server distinguishes itself through on-the-fly format transmuxing and protocol-agnostic routing, which decouples input and output protocols via an internal media bus. It features a programmable automation system that executes external shell commands via event hooks triggered by client c
Distributes a single incoming media source to multiple concurrent readers via in-memory packet duplication.
pypdf is a Python library for parsing, manipulating, and generating PDF documents. It provides high-level operations for document processing, such as merging multiple files into one or splitting a single document into smaller files. The project includes specialized tools for managing interactive elements, including the creation and modification of annotations, hyperlinks, and form fields. It also supports advanced metadata management, allowing for the extraction and modification of standard document properties and XML-based XMP metadata. Beyond basic structural changes, the library covers pa
Performs read and write operations directly on byte sequences to bypass the need for temporary disk files.
Okio is a Java I/O library providing a set of tools for efficient byte-stream processing and file system operations. It functions as a buffered byte stream handler and streaming data transformer, utilizing a cross-platform file system API to manage data movement. The project is distinguished by its use of pooled mutable byte buffers that treat sequences as queues to reduce memory copying and garbage collection churn. It further decouples file operations from the host operating system through an abstraction-based file system, allowing for consistent path manipulation and atomic operations acro
Manages bytes as growable queues to significantly reduce memory copying and GC churn during I/O.
QOI is a lossless image codec and encoding standard designed for high-speed compression and decompression of raw pixel data. It provides a toolkit for translating raw image buffers into a compact format and back into pixel representations without any loss of quality. The implementation focuses on fast image encoding and decoding, enabling the rapid conversion of compressed image data back into raw pixels. It also supports image format conversion to ensure compatibility across different software systems and hardware.
Implements linear byte-stream processing to avoid complex memory overhead during image encoding and decoding.
rustls is a modern implementation of the Transport Layer Security protocol written in the Rust programming language. It serves as a cryptographic transport layer for establishing encrypted connections between clients and servers to ensure data privacy and integrity. The library features a pluggable crypto provider framework, allowing the substitution of cryptographic primitive implementations to meet specific platform architecture or regulatory compliance requirements. It provides capabilities for secure server configuration to handle encrypted incoming connections and secure client connecti
Implements mutable byte buffers to handle network streams and minimize memory copying.
The Rust RFCs repository is the formal home for the Rust language evolution process, housing the structured design documents and community review mechanisms that govern changes to the Rust programming language, its compiler, and its standard library. It defines the complete lifecycle for proposing, discussing, and implementing substantial changes through RFC documents, from initial submission and community feedback through final comment periods and sub-team sign-offs. The repository codifies the governance and collaboration processes that shape Rust's development, including mechanisms for com
Defines the read_exact method on the Read trait for guaranteed buffer filling.
protobuf-net es un framework de serialización binaria y biblioteca .NET que implementa la especificación Protocol Buffers. Funciona como una herramienta de serialización basada en esquemas y un mapeador de datos orientado a contratos, convirtiendo gráficos de objetos complejos en un formato binario compacto para almacenamiento y transmisión en red. La biblioteca permite el intercambio de datos multiplataforma y redes de alto rendimiento al reducir los tamaños de carga útil y el tiempo de procesamiento. Admite específicamente la persistencia de jerarquías de objetos, permitiendo la serialización y reconstrucción de estructuras de clases complejas que incluyen herencia y tipos derivados. El proyecto cubre capacidades principales que incluyen la serialización y deserialización de datos binarios, generación de clases basada en esquemas y la gestión de la herencia de tipos a través de identificadores únicos. Proporciona mecanismos para mapear estructuras de datos a través de atributos estáticos o configuraciones dinámicas en tiempo de ejecución.
Processes binary data directly through sequential byte streams to minimize memory overhead during large transfers.
Janet es un lenguaje de programación dinámico basado en Lisp que cuenta con una máquina virtual de bytecode basada en registros y un motor de scripting integrable. Funciona como un runtime de concurrencia basado en fibras e incluye un motor de análisis basado en Gramáticas de Expresión de Análisis (PEG). El proyecto se distingue por su capacidad de integrarse en aplicaciones de C o C++ a través de una interfaz de cabecera mínima. Utiliza un sistema de macros al estilo Lisp para la transformación de código en tiempo de compilación y emplea herencia de tablas basada en prototipos para el comportamiento orientado a objetos. El runtime cubre un amplio conjunto de capacidades, incluyendo la gestión de IO asíncrona a través de un bucle de eventos no bloqueante, interoperabilidad de bibliotecas nativas a través de una interfaz de funciones externas y procesamiento de texto integral utilizando gramáticas PEG. También proporciona herramientas para la automatización del sistema, como un bucle de lectura-evaluación-impresión (REPL), un sistema de módulos para la resolución de símbolos y utilidades para la comunicación de sockets de red y la gestión del sistema de archivos. El entorno incluye herramientas de diagnóstico para la depuración de la ejecución de bytecode y puede empaquetar el código fuente en ejecutables binarios independientes.
Provides growable byte buffers to store and modify sequences of bytes representing binary data.
python-magic is a C-binding wrapper that provides a Python interface for the libmagic system library. It functions as a file signature analyzer and MIME type detector, identifying file formats by comparing header bytes against a database of known binary signatures. The library enables the identification of file types from both file paths and raw data buffers. It supports custom file signature matching through the injection of user-provided magic databases, allowing for the detection of specialized or proprietary formats. The project covers binary data analysis and MIME type mapping to transl
Provides the ability to analyze binary data sequences directly in memory to identify file types without disk persistence.
Kotlinx-io is a multiplatform library designed for input and output operations, providing a unified interface for streaming data, managing byte buffers, and interacting with local filesystems. It serves as a cross-platform abstraction layer that standardizes how applications handle data movement across different operating systems and hardware architectures. The library distinguishes itself by providing high-performance tools for both mutable and immutable byte sequences. It utilizes segmented memory pools and direct memory access to minimize allocation overhead and prevent unnecessary data co
Modifies dynamic byte sequences using efficient memory segments to reduce allocation overhead.