awesome-repositories.com
Blog
awesome-repositories.com

Découvrez les meilleurs dépôts open-source grâce à notre recherche par IA.

ExplorerRecherches sélectionnéesAlternatives open sourceLogiciels auto-hébergésBlogPlan du site
ProjetÀ proposNotre méthodologiePresseServeur MCP
Mentions légalesConfidentialitéConditions d'utilisation
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

65 dépôts

Awesome GitHub RepositoriesMemory Buffering

Techniques for staging data in volatile memory to optimize disk I/O performance.

Distinct from Sorted Memory Buffers: General memory buffering for file I/O, distinct from LSM-tree sorted buffers or terminal grids.

Explore 65 awesome GitHub repositories matching software engineering & architecture · Memory Buffering. Refine with filters or upvote what's useful.

Awesome Memory Buffering GitHub Repositories

Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
  • xuri/excelizeAvatar de xuri

    xuri/excelize

    20,668Voir sur GitHub↗

    Excelize is a Go library designed for reading, writing, and modifying Microsoft Excel files in XML-based formats. It functions as a spreadsheet file parser and generator that enables the programmatic extraction and modification of data. The library includes a streaming spreadsheet processor to handle massive datasets incrementally, preventing system memory exhaustion during large-scale read and write operations. It also provides a chart generator to convert worksheet values or external data sources into visual representations within the spreadsheet. Beyond core file processing, the project c

    Implements memory buffers to stage spreadsheet data and optimize disk access patterns.

    Go
    Voir sur GitHub↗20,668
  • tencent/mmkvAvatar de Tencent

    Tencent/MMKV

    18,623Voir sur GitHub↗

    MMKV is a high-performance, cross-platform key-value storage framework designed for mobile platforms and POSIX environments, including Android, iOS, macOS, and Windows. It provides a persistence layer that utilizes memory-mapped files and binary serialization to achieve low-latency data access. The project distinguishes itself through native support for multi-process synchronization, allowing concurrent read and write operations across different application processes. It also implements security via AES encryption for data at rest, featuring symmetric encryption and key rotation to protect st

    Optimizes performance by passing large values using native memory buffers to avoid expensive data copying.

    C++
    Voir sur GitHub↗18,623
  • skywind3000/kcpAvatar de skywind3000

    skywind3000/kcp

    16,786Voir sur GitHub↗

    KCP is a low-latency transport protocol and reliability layer designed to provide TCP-like guaranteed delivery and ordering over UDP. It implements an automatic repeat request protocol to minimize network jitter and delays, prioritizing timeliness over bandwidth efficiency. The project distinguishes itself through a pluggable congestion controller and flow control logic, allowing the transport layer to be tuned for specific network environments. It includes a NAT keep-alive system that uses periodic heartbeat packets to maintain active communication paths and prevent address translation timeo

    Manages transmission throughput and memory usage via configurable window-based buffering for send and receive queues.

    C
    Voir sur GitHub↗16,786
  • nikopueringer/corridorkeyAvatar de nikopueringer

    nikopueringer/CorridorKey

    13,940Voir sur GitHub↗

    Perfect Green Screen Keys

    Decodes video frames into a temporary buffer for individual processing before re-encoding the output stream.

    Python
    Voir sur GitHub↗13,940
  • assimp/assimpAvatar de assimp

    assimp/assimp

    12,993Voir sur GitHub↗

    Assimp is a cross-platform 3D asset pipeline and import library that loads numerous industry-standard 3D file formats into a single unified internal data structure. It functions as a framework for converting 3D models between different file formats across multiple operating systems and architectures. The project provides a 3D mesh processing tool for normalizing and optimizing geometry through triangulation, vertex removal, and normal generation. It also includes a 3D asset export utility to write internal scene data back into various external file formats. The system covers broad capability

    Allows 3D model data to be imported directly from memory buffers.

    C++3mfandroidasset-pipeline
    Voir sur GitHub↗12,993
  • twitter/twemproxyAvatar de twitter

    twitter/twemproxy

    12,340Voir sur GitHub↗

    Twemproxy is a lightweight proxy that routes and distributes requests across multiple Redis and Memcached backend servers. It functions as a protocol translation gateway and distributed cache shard manager, partitioning data across clusters to balance load and storage capacity. The system acts as a high-availability cache orchestrator, employing health monitoring and automatic server ejection to maintain continuous access to cached data. It integrates with sentinels for dynamic master and replica discovery and utilizes consistent hashing and tag-based key grouping to manage data distribution

    Adjusts memory buffer sizes for zero-copy data transfers to balance concurrent connections against data throughput.

    C
    Voir sur GitHub↗12,340
  • expressjs/multerAvatar de expressjs

    expressjs/multer

    12,055Voir sur GitHub↗

    Multer is a multipart form parser and file upload manager designed as middleware for Node.js and Express.js applications. It extracts text fields and uploaded files from incoming requests to populate the request body and file objects for server-side use. The project utilizes pluggable storage engines to abstract file persistence, allowing developers to switch between saving files to a physical disk or holding them as buffers in system memory. It supports customizable file naming strategies and a custom storage engine interface for defining how files are saved and deleted. The middleware incl

    Holds uploaded files as buffer objects in volatile memory for immediate programmatic processing.

    JavaScript
    Voir sur GitHub↗12,055
  • panjf2000/gnetAvatar de panjf2000

    panjf2000/gnet

    11,186Voir sur GitHub↗

    gnet is a high-performance event-driven networking framework for Go, designed for building scalable TCP, UDP, and Unix socket servers. It functions as a non-blocking socket manager and multi-reactor network engine that handles thousands of simultaneous connections with low memory overhead. The framework distinguishes itself by using a multi-reactor architecture that distributes I/O across multiple event loops pinned to operating system threads to minimize context switching. It employs edge-triggered polling to reduce system call frequency and utilizes elastic ring-buffers to minimize allocati

    Minimizes allocation costs by employing reusable buffer structures that adapt to incoming network data.

    Go
    Voir sur GitHub↗11,186
  • kkroening/ffmpeg-pythonAvatar de kkroening

    kkroening/ffmpeg-python

    10,999Voir sur GitHub↗

    ffmpeg-python is a Python wrapper that translates programmatic method calls into command-line arguments for executing FFmpeg media processing tasks. It functions as a multimedia transcoding interface and a media stream capture tool, allowing for the recording of live audio and video from hardware devices and network sources. The library features a fluent interface for constructing complex directed graphs of audio and video filters through method chaining. It also includes an FFprobe metadata extractor that retrieves structured technical properties from media files and returns them as Python d

    Streams raw audio or video data directly into memory buffers for real-time processing by external software.

    Python
    Voir sur GitHub↗10,999
  • koral--/android-gif-drawableAvatar de koral--

    koral--/android-gif-drawable

    9,648Voir sur GitHub↗

    android-gif-drawable is a rendering library for displaying and controlling animated GIF images within Android image views and drawables. It provides a custom drawable implementation for frame-based animations, a playback system for seeking and looping, and a metadata extractor for retrieving technical properties such as frame counts and loop settings. The library enables the synchronization of a single animation instance across multiple views to ensure consistent playback. It supports loading GIF data from various sources, including assets, resources, URIs, byte arrays, files, and input strea

    Uses specialized memory buffers for frame data to reduce garbage collection overhead during playback.

    Java
    Voir sur GitHub↗9,648
  • lwansbrough/react-native-cameraAvatar de lwansbrough

    lwansbrough/react-native-camera

    9,638Voir sur GitHub↗

    This project is a cross-platform mobile camera library used to embed live camera feeds and capture photos or videos within mobile applications. It provides a unified interface for integrating mobile image capture and camera functionality across different platforms. The library includes specialized tools for reading and decoding barcode data from a live camera stream and a system for locating and identifying human faces in real-time.

    Optimizes data transfer by handling high-resolution image and video buffers in native memory.

    Java
    Voir sur GitHub↗9,638
  • mit-pdos/xv6-publicAvatar de mit-pdos

    mit-pdos/xv6-public

    9,466Voir sur GitHub↗

    xv6 is a Unix-like educational operating system designed for teaching kernel concepts. It is implemented as an instructional kernel for x86 and RISC-V architectures, featuring a virtual memory manager and support for symmetric multiprocessing. The system is designed to demonstrate core OS principles through a simplified re-implementation of Unix Version 6. It provides a controlled environment for studying system calls, trap handling, and process lifecycles. Its capability surface covers process management, including scheduling and context switching, and memory management via page tables for

    Reads a specified number of bytes from a file descriptor into a memory buffer.

    C
    Voir sur GitHub↗9,466
  • risingwavelabs/risingwaveAvatar de risingwavelabs

    risingwavelabs/risingwave

    9,093Voir sur GitHub↗

    RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

    Provides controls for memory allocation during data pre-reading to balance performance and memory consumption.

    Rustapache-icebergdata-engineeringdatabase
    Voir sur GitHub↗9,093
  • voidcosmos/npkillAvatar de voidcosmos

    voidcosmos/npkill

    9,060Voir sur GitHub↗

    npkill is a command-line filesystem scanner and disk space recovery utility. It is designed to identify and delete large dependency folders, specifically targeting node_modules directories created by the npm package manager, to reclaim storage space across multiple projects. The tool provides an interactive terminal interface that allows for the discovery and identification of dependency folders along with their total disk size and modification dates. Users can select directories for individual or bulk deletion using keyboard inputs. The utility includes capabilities for recursive filesystem

    Uses memory buffering to read filesystem metadata in chunks, preventing system memory overload during scans.

    TypeScriptclicommand-linedeveloper-tools
    Voir sur GitHub↗9,060
  • square/okioAvatar de square

    square/okio

    8,986Voir sur GitHub↗

    Okio is a Java I/O library providing a set of tools for efficient byte-stream processing and file system operations. It functions as a buffered byte stream handler and streaming data transformer, utilizing a cross-platform file system API to manage data movement. The project is distinguished by its use of pooled mutable byte buffers that treat sequences as queues to reduce memory copying and garbage collection churn. It further decouples file operations from the host operating system through an abstraction-based file system, allowing for consistent path manipulation and atomic operations acro

    Manages byte buffers and streams with pooled memory to reduce garbage collection and improve I/O performance.

    Kotlinkotlinkotlin-multiplatform
    Voir sur GitHub↗8,986
  • lipku/livetalkingAvatar de lipku

    lipku/LiveTalking

    8,042Voir sur GitHub↗

    LiveTalking is an interactive talking head engine and AI avatar management platform designed to synchronize synthetic speech with facial movements. It functions as a real-time orchestrator that connects large language models and text-to-speech services to neural-rendered digital humans. The project distinguishes itself through low-latency streaming capabilities and the ability to handle real-time conversational interruptions. It supports advanced audio-visual customization, including human voice cloning and the ability to drive avatar expressions using real-time webcam data. The platform cov

    Uses a sliding-window memory buffer for video frames to prevent disk read bottlenecks during long-duration streaming.

    Pythonaigcdigihumandigital-human
    Voir sur GitHub↗8,042
  • fluent/fluent-bitAvatar de fluent

    fluent/fluent-bit

    7,946Voir sur GitHub↗

    Fluent Bit est un collecteur de logs et de télémétrie unifié cloud-native conçu comme un pipeline de données efficace en ressources. Il ingère des logs, des métriques et des traces provenant de multiples sources, les traitant en temps réel avant d'acheminer les données vers des backends de stockage externes. Le projet fonctionne comme un processeur de flux en temps réel et un processeur de logs OpenTelemetry, capable de transformer et de filtrer les données en utilisant SQL et une logique conditionnelle. Il agit également comme un agent de traçage distribué capable d'échantillonner les traces pour réduire le volume de données tout en préservant les chemins de requête complets. Le système fournit une livraison de données fiable grâce à une mise en mémoire tampon basée sur le système de fichiers et une logique de réessai avec état pour éviter la perte de données lors des pannes. Son architecture modulaire prend en charge des plugins d'entrée et de sortie enfichables, un routage basé sur les métadonnées et la capacité d'étendre les fonctionnalités via des bibliothèques partagées. Le logiciel peut être déployé en tant que conteneur sur différentes architectures CPU et systèmes d'exploitation.

    Provides filesystem-backed buffering with checksums to ensure reliable data delivery and prevent loss during outages.

    C
    Voir sur GitHub↗7,946
  • robbyant/lingbot-mapAvatar de Robbyant

    Robbyant/lingbot-map

    7,315Voir sur GitHub↗

    Lingbot-map is a feed-forward neural network designed for real-time 3D scene reconstruction from streaming video. It processes video frames one at a time without iterative optimization, producing dense geometry and camera poses at interactive frame rates directly from a live feed. The project distinguishes itself through its ability to maintain stable geometry and pose alignment across very long video sequences, handling thousands of frames without drift. It achieves this through a combination of coordinate grounding memory, sliding-window inference with overlapping keyframes, and a paged KV

    Maintains a persistent coordinate frame across thousands of frames by aligning geometric cues with trajectory memory.

    Python
    Voir sur GitHub↗7,315
  • cloudwego/hertzAvatar de cloudwego

    cloudwego/hertz

    7,279Voir sur GitHub↗

    Hertz is a high-performance Go HTTP framework designed for building scalable microservices, RESTful APIs, and AI applications. It functions as a high-performance web server and a communication framework for microservices, utilizing non-blocking I/O and zero-copy memory management to handle high-concurrency traffic. The project distinguishes itself through a microservices communication toolkit that supports high-efficiency remote procedure calls via gRPC and Thrift protocols. It implements an asynchronous middleware engine based on an onion model, allowing for a pluggable request-response pipe

    Uses memory blocks and reference counting to move data between the network and application without duplicating buffers.

    Gogohttpmicroservices
    Voir sur GitHub↗7,279
  • muaz-khan/recordrtcAvatar de muaz-khan

    muaz-khan/RecordRTC

    6,913Voir sur GitHub↗

    RecordRTC est une bibliothèque JavaScript pour capturer des flux audio, vidéo et d'écran au sein du navigateur. Elle fonctionne comme un outil d'enregistrement multimédia qui permet la capture de flux multimédia et leur exportation sous forme de fichiers lisibles. La bibliothèque fournit des utilitaires spécialisés pour enregistrer des fenêtres de bureau ou des écrans entiers et capturer des animations 2D ou 3D à partir d'éléments canvas. Elle gère ces tâches en s'interfaçant avec les objets de flux multimédia du navigateur pour enregistrer l'activité sous forme de fichiers vidéo.

    Implements memory buffering to stage recorded media chunks before consolidating them into a final downloadable file.

    JavaScript
    Voir sur GitHub↗6,913
Préc.123…4Suivant
  1. Home
  2. Software Engineering & Architecture
  3. Memory Buffering

Explorer les sous-tags

  • Animation Frame BuffersSpecialized memory buffering for sequential animation frames to reduce garbage collection overhead. **Distinct from Memory Buffering:** Focuses on reducing GC churn for high-frequency frame updates rather than disk I/O optimization.
  • Buffer Interface StandardizationStandardizing the interfaces for sequences of memory buffers to enable interoperable and efficient data transfer. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on the interface standardization for buffer sequences rather than the act of staging data.
  • Bytecode Buffer ReadersReads compiled Python bytecode from in-memory buffers for decompilation without requiring a file on disk. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on reading bytecode from buffers specifically, not general I/O buffering strategies.
  • Disk-Backed BufferingPersistent storage of memory buffers on the filesystem to prevent data loss during crashes. **Distinct from Memory Buffering:** Distinct from general Memory Buffering by focusing on filesystem persistence for reliability rather than just I/O optimization.
  • Disk-Memory ThresholdsLogic that toggles between memory buffering and disk persistence based on a size threshold. **Distinct from Memory Buffering:** A specific strategy for deciding storage medium based on file size, not general I/O optimization.
  • File Descriptor BufferingReading raw byte streams from system file descriptors into memory buffers. **Distinct from File Descriptor Ingestion:** Focuses on the kernel-level read from a file descriptor to a buffer, rather than web-API file reading.
  • Fixed Memory Buffers2 sous-tagsRuntimes that use a pre-allocated, fixed-size memory area to ensure a predictable footprint. **Distinct from Memory Buffering:** Focuses on using a fixed buffer for the entire runtime footprint rather than staging data for I/O optimization.
  • Frame Memory Buffers2 sous-tagsBuffers that maintain a limited set of video frames to balance accuracy and inference overhead. **Distinct from Memory Buffering:** Specifically handles video frame sequences for inference rather than general disk I/O buffering.
  • In-Memory Asset ImportsCapabilities for loading asset data directly from volatile memory buffers instead of disk files. **Distinct from Memory Buffering:** Focuses on importing 3D assets from memory, not just general I/O buffering.
  • Memory-Buffer File I/O ReplacementsAll file-accessing functions have memory-buffer equivalents, enabling fully static operation without any filesystem dependency. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on replacing file I/O entirely with memory buffers, not just staging data for disk I/O optimization.
  • Native Buffer Transfers2 sous-tagsOptimizations that pass data via native memory pointers to avoid expensive copying between managed and unmanaged memory. **Distinct from Memory Buffering:** Distinct from general memory buffering as it specifically focuses on zero-copy transfers between native and managed layers.
  • Optimized Buffer ReadsUsing size comparison and direct memory copy instead of looping when reading from in-memory buffers. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on the optimization technique for buffer reads, not general buffering strategies.
  • Prefetch BufferingConfigurations for pre-reading data chunks into memory to optimize query performance. **Distinct from Memory Buffering:** Focuses specifically on the pre-reading buffer configuration rather than general memory buffering techniques.
  • Reusable Buffer PoolsMemory structures that are recycled and adapted to incoming data to minimize allocation overhead. **Distinct from Memory Buffering:** Focuses on the reuse and adaptation of buffers for high-frequency networking rather than staging data for disk I/O.
  • Scratch-Space Work BuffersDedicated temporary memory areas used to handle input-dependent processing requirements without dynamic allocation. **Distinct from Memory Buffering:** Distinct from Memory Buffering: specifically used as a sandboxed scratch space to avoid runtime heap allocation.
  • Stream Buffering3 sous-tagsBuffers for streaming raw media data between processes via pipes for real-time processing. **Distinct from Memory Buffering:** Specifically focuses on piping raw media streams into memory buffers, distinct from general disk I/O buffering.
  • Transmission Window BufferingBuffers that limit the number of outstanding packets in flight to manage throughput and memory. **Distinct from Memory Buffering:** Focuses on network transmission windows rather than general disk I/O buffering.
  • Waveform Buffer PrecomputersTechniques that prepare entire waveform buffers in memory before transmission to eliminate real-time computation overhead during signal generation. **Distinct from Memory Buffering:** Distinct from Memory Buffering: precomputes entire signal waveforms before transmission, not general I/O staging or network buffering.
  • Zero-Copy Buffer Tuning3 sous-tagsOptimization of memory buffers specifically for zero-copy data transfers between network sockets. **Distinct from Memory Buffering:** Focuses on network socket throughput and zero-copy transfers, distinct from general disk I/O buffering.