awesome-repositories.com
Blog
awesome-repositories.com

Descoperă cele mai bune repository-uri open source cu căutare AI.

ExploreazăCăutări recomandateAlternative open-sourceSoftware self-hostedBlogHartă site
ProiectDespreCum realizăm clasamentulPresăServer MCP
LegalConfidențialitateTermeni
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

65 repository-uri

Awesome GitHub RepositoriesMemory Buffering

Techniques for staging data in volatile memory to optimize disk I/O performance.

Distinct from Sorted Memory Buffers: General memory buffering for file I/O, distinct from LSM-tree sorted buffers or terminal grids.

Explore 65 awesome GitHub repositories matching software engineering & architecture · Memory Buffering. Refine with filters or upvote what's useful.

Awesome Memory Buffering GitHub Repositories

Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
  • xuri/excelizeAvatar xuri

    xuri/excelize

    20,668Vezi pe GitHub↗

    Excelize is a Go library designed for reading, writing, and modifying Microsoft Excel files in XML-based formats. It functions as a spreadsheet file parser and generator that enables the programmatic extraction and modification of data. The library includes a streaming spreadsheet processor to handle massive datasets incrementally, preventing system memory exhaustion during large-scale read and write operations. It also provides a chart generator to convert worksheet values or external data sources into visual representations within the spreadsheet. Beyond core file processing, the project c

    Implements memory buffers to stage spreadsheet data and optimize disk access patterns.

    Go
    Vezi pe GitHub↗20,668
  • tencent/mmkvAvatar Tencent

    Tencent/MMKV

    18,623Vezi pe GitHub↗

    MMKV is a high-performance, cross-platform key-value storage framework designed for mobile platforms and POSIX environments, including Android, iOS, macOS, and Windows. It provides a persistence layer that utilizes memory-mapped files and binary serialization to achieve low-latency data access. The project distinguishes itself through native support for multi-process synchronization, allowing concurrent read and write operations across different application processes. It also implements security via AES encryption for data at rest, featuring symmetric encryption and key rotation to protect st

    Optimizes performance by passing large values using native memory buffers to avoid expensive data copying.

    C++
    Vezi pe GitHub↗18,623
  • skywind3000/kcpAvatar skywind3000

    skywind3000/kcp

    16,786Vezi pe GitHub↗

    KCP is a low-latency transport protocol and reliability layer designed to provide TCP-like guaranteed delivery and ordering over UDP. It implements an automatic repeat request protocol to minimize network jitter and delays, prioritizing timeliness over bandwidth efficiency. The project distinguishes itself through a pluggable congestion controller and flow control logic, allowing the transport layer to be tuned for specific network environments. It includes a NAT keep-alive system that uses periodic heartbeat packets to maintain active communication paths and prevent address translation timeo

    Manages transmission throughput and memory usage via configurable window-based buffering for send and receive queues.

    C
    Vezi pe GitHub↗16,786
  • nikopueringer/corridorkeyAvatar nikopueringer

    nikopueringer/CorridorKey

    13,940Vezi pe GitHub↗

    Perfect Green Screen Keys

    Decodes video frames into a temporary buffer for individual processing before re-encoding the output stream.

    Python
    Vezi pe GitHub↗13,940
  • assimp/assimpAvatar assimp

    assimp/assimp

    12,993Vezi pe GitHub↗

    Assimp is a cross-platform 3D asset pipeline and import library that loads numerous industry-standard 3D file formats into a single unified internal data structure. It functions as a framework for converting 3D models between different file formats across multiple operating systems and architectures. The project provides a 3D mesh processing tool for normalizing and optimizing geometry through triangulation, vertex removal, and normal generation. It also includes a 3D asset export utility to write internal scene data back into various external file formats. The system covers broad capability

    Allows 3D model data to be imported directly from memory buffers.

    C++3mfandroidasset-pipeline
    Vezi pe GitHub↗12,993
  • twitter/twemproxyAvatar twitter

    twitter/twemproxy

    12,340Vezi pe GitHub↗

    Twemproxy is a lightweight proxy that routes and distributes requests across multiple Redis and Memcached backend servers. It functions as a protocol translation gateway and distributed cache shard manager, partitioning data across clusters to balance load and storage capacity. The system acts as a high-availability cache orchestrator, employing health monitoring and automatic server ejection to maintain continuous access to cached data. It integrates with sentinels for dynamic master and replica discovery and utilizes consistent hashing and tag-based key grouping to manage data distribution

    Adjusts memory buffer sizes for zero-copy data transfers to balance concurrent connections against data throughput.

    C
    Vezi pe GitHub↗12,340
  • expressjs/multerAvatar expressjs

    expressjs/multer

    12,055Vezi pe GitHub↗

    Multer is a multipart form parser and file upload manager designed as middleware for Node.js and Express.js applications. It extracts text fields and uploaded files from incoming requests to populate the request body and file objects for server-side use. The project utilizes pluggable storage engines to abstract file persistence, allowing developers to switch between saving files to a physical disk or holding them as buffers in system memory. It supports customizable file naming strategies and a custom storage engine interface for defining how files are saved and deleted. The middleware incl

    Holds uploaded files as buffer objects in volatile memory for immediate programmatic processing.

    JavaScript
    Vezi pe GitHub↗12,055
  • panjf2000/gnetAvatar panjf2000

    panjf2000/gnet

    11,186Vezi pe GitHub↗

    gnet este un framework de rețelistică de înaltă performanță, bazat pe evenimente, pentru Go, conceput pentru a construi servere scalabile TCP, UDP și Unix socket. Acesta funcționează ca un manager de socket-uri non-blocante și un motor de rețea multi-reactor care gestionează mii de conexiuni simultane cu un consum redus de memorie. Framework-ul se distinge prin utilizarea unei arhitecturi multi-reactor care distribuie I/O-ul pe mai multe bucle de evenimente (event loops) fixate pe thread-uri ale sistemului de operare, pentru a minimiza context switching-ul. Utilizează polling de tip edge-triggered pentru a reduce frecvența apelurilor de sistem și ring-buffere elastice pentru a minimiza costurile de alocare și presiunea asupra garbage collector-ului. Biblioteca acoperă o gamă largă de capabilități de rețelistică, inclusiv distribuția echilibrată a conexiunilor pe nucleele CPU, execuția asincronă a sarcinilor prin worker pools și un sistem structurat pentru gestionarea ciclului de viață al conexiunilor. De asemenea, oferă primitive pentru implementarea protocoalelor binare personalizate și gestionarea opțiunilor de socket de nivel jos.

    Minimizes allocation costs by employing reusable buffer structures that adapt to incoming network data.

    Go
    Vezi pe GitHub↗11,186
  • kkroening/ffmpeg-pythonAvatar kkroening

    kkroening/ffmpeg-python

    10,999Vezi pe GitHub↗

    ffmpeg-python is a Python wrapper that translates programmatic method calls into command-line arguments for executing FFmpeg media processing tasks. It functions as a multimedia transcoding interface and a media stream capture tool, allowing for the recording of live audio and video from hardware devices and network sources. The library features a fluent interface for constructing complex directed graphs of audio and video filters through method chaining. It also includes an FFprobe metadata extractor that retrieves structured technical properties from media files and returns them as Python d

    Streams raw audio or video data directly into memory buffers for real-time processing by external software.

    Python
    Vezi pe GitHub↗10,999
  • koral--/android-gif-drawableAvatar koral--

    koral--/android-gif-drawable

    9,648Vezi pe GitHub↗

    android-gif-drawable is a rendering library for displaying and controlling animated GIF images within Android image views and drawables. It provides a custom drawable implementation for frame-based animations, a playback system for seeking and looping, and a metadata extractor for retrieving technical properties such as frame counts and loop settings. The library enables the synchronization of a single animation instance across multiple views to ensure consistent playback. It supports loading GIF data from various sources, including assets, resources, URIs, byte arrays, files, and input strea

    Uses specialized memory buffers for frame data to reduce garbage collection overhead during playback.

    Java
    Vezi pe GitHub↗9,648
  • lwansbrough/react-native-cameraAvatar lwansbrough

    lwansbrough/react-native-camera

    9,638Vezi pe GitHub↗

    This project is a cross-platform mobile camera library used to embed live camera feeds and capture photos or videos within mobile applications. It provides a unified interface for integrating mobile image capture and camera functionality across different platforms. The library includes specialized tools for reading and decoding barcode data from a live camera stream and a system for locating and identifying human faces in real-time.

    Optimizes data transfer by handling high-resolution image and video buffers in native memory.

    Java
    Vezi pe GitHub↗9,638
  • mit-pdos/xv6-publicAvatar mit-pdos

    mit-pdos/xv6-public

    9,466Vezi pe GitHub↗

    xv6 is a Unix-like educational operating system designed for teaching kernel concepts. It is implemented as an instructional kernel for x86 and RISC-V architectures, featuring a virtual memory manager and support for symmetric multiprocessing. The system is designed to demonstrate core OS principles through a simplified re-implementation of Unix Version 6. It provides a controlled environment for studying system calls, trap handling, and process lifecycles. Its capability surface covers process management, including scheduling and context switching, and memory management via page tables for

    Reads a specified number of bytes from a file descriptor into a memory buffer.

    C
    Vezi pe GitHub↗9,466
  • risingwavelabs/risingwaveAvatar risingwavelabs

    risingwavelabs/risingwave

    9,093Vezi pe GitHub↗

    RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

    Provides controls for memory allocation during data pre-reading to balance performance and memory consumption.

    Rustapache-icebergdata-engineeringdatabase
    Vezi pe GitHub↗9,093
  • voidcosmos/npkillAvatar voidcosmos

    voidcosmos/npkill

    9,060Vezi pe GitHub↗

    npkill is a command-line filesystem scanner and disk space recovery utility. It is designed to identify and delete large dependency folders, specifically targeting node_modules directories created by the npm package manager, to reclaim storage space across multiple projects. The tool provides an interactive terminal interface that allows for the discovery and identification of dependency folders along with their total disk size and modification dates. Users can select directories for individual or bulk deletion using keyboard inputs. The utility includes capabilities for recursive filesystem

    Uses memory buffering to read filesystem metadata in chunks, preventing system memory overload during scans.

    TypeScriptclicommand-linedeveloper-tools
    Vezi pe GitHub↗9,060
  • square/okioAvatar square

    square/okio

    8,986Vezi pe GitHub↗

    Okio is a Java I/O library providing a set of tools for efficient byte-stream processing and file system operations. It functions as a buffered byte stream handler and streaming data transformer, utilizing a cross-platform file system API to manage data movement. The project is distinguished by its use of pooled mutable byte buffers that treat sequences as queues to reduce memory copying and garbage collection churn. It further decouples file operations from the host operating system through an abstraction-based file system, allowing for consistent path manipulation and atomic operations acro

    Manages byte buffers and streams with pooled memory to reduce garbage collection and improve I/O performance.

    Kotlinkotlinkotlin-multiplatform
    Vezi pe GitHub↗8,986
  • lipku/livetalkingAvatar lipku

    lipku/LiveTalking

    8,042Vezi pe GitHub↗

    LiveTalking is an interactive talking head engine and AI avatar management platform designed to synchronize synthetic speech with facial movements. It functions as a real-time orchestrator that connects large language models and text-to-speech services to neural-rendered digital humans. The project distinguishes itself through low-latency streaming capabilities and the ability to handle real-time conversational interruptions. It supports advanced audio-visual customization, including human voice cloning and the ability to drive avatar expressions using real-time webcam data. The platform cov

    Uses a sliding-window memory buffer for video frames to prevent disk read bottlenecks during long-duration streaming.

    Pythonaigcdigihumandigital-human
    Vezi pe GitHub↗8,042
  • fluent/fluent-bitAvatar fluent

    fluent/fluent-bit

    7,946Vezi pe GitHub↗

    Fluent Bit este un log shipper cloud-native și un colector unificat de telemetrie conceput ca un pipeline de date eficient din punct de vedere al resurselor. Acesta ingerează log-uri, metrici și urme (traces) din surse multiple, procesându-le în timp real înainte de a direcționa datele către backend-uri de stocare externe. Proiectul funcționează ca un procesor de flux în timp real și procesor de log-uri OpenTelemetry, capabil să transforme și să filtreze datele folosind SQL și logică condițională. Acționează, de asemenea, ca un agent de tracing distribuit care poate eșantiona urmele pentru a reduce volumul de date, păstrând în același timp căile complete ale cererilor. Sistemul oferă livrarea fiabilă a datelor prin buffering bazat pe sistemul de fișiere și logică de reîncercare cu stare pentru a preveni pierderea datelor în timpul întreruperilor. Arhitectura sa modulară suportă plugin-uri de input și output interschimbabile, rutare bazată pe metadate și capacitatea de a extinde funcționalitatea prin biblioteci partajate. Software-ul poate fi implementat ca un container pe diferite arhitecturi CPU și sisteme de operare.

    Provides filesystem-backed buffering with checksums to ensure reliable data delivery and prevent loss during outages.

    C
    Vezi pe GitHub↗7,946
  • robbyant/lingbot-mapAvatar Robbyant

    Robbyant/lingbot-map

    7,315Vezi pe GitHub↗

    Lingbot-map is a feed-forward neural network designed for real-time 3D scene reconstruction from streaming video. It processes video frames one at a time without iterative optimization, producing dense geometry and camera poses at interactive frame rates directly from a live feed. The project distinguishes itself through its ability to maintain stable geometry and pose alignment across very long video sequences, handling thousands of frames without drift. It achieves this through a combination of coordinate grounding memory, sliding-window inference with overlapping keyframes, and a paged KV

    Maintains a persistent coordinate frame across thousands of frames by aligning geometric cues with trajectory memory.

    Python
    Vezi pe GitHub↗7,315
  • cloudwego/hertzAvatar cloudwego

    cloudwego/hertz

    7,279Vezi pe GitHub↗

    Hertz is a high-performance Go HTTP framework designed for building scalable microservices, RESTful APIs, and AI applications. It functions as a high-performance web server and a communication framework for microservices, utilizing non-blocking I/O and zero-copy memory management to handle high-concurrency traffic. The project distinguishes itself through a microservices communication toolkit that supports high-efficiency remote procedure calls via gRPC and Thrift protocols. It implements an asynchronous middleware engine based on an onion model, allowing for a pluggable request-response pipe

    Uses memory blocks and reference counting to move data between the network and application without duplicating buffers.

    Gogohttpmicroservices
    Vezi pe GitHub↗7,279
  • muaz-khan/recordrtcAvatar muaz-khan

    muaz-khan/RecordRTC

    6,913Vezi pe GitHub↗

    RecordRTC is a JavaScript library for capturing audio, video, and screen streams within the browser. It functions as a media recording tool that enables the capture of media streams and their export as playable files. The library provides specialized utilities for recording desktop windows or entire screens and capturing 2D or 3D animations from canvas elements. It handles these tasks by interfacing with browser media stream objects to save activity as video files.

    Implements memory buffering to stage recorded media chunks before consolidating them into a final downloadable file.

    JavaScript
    Vezi pe GitHub↗6,913
Înapoi123…4Înainte
  1. Home
  2. Software Engineering & Architecture
  3. Memory Buffering

Explorează sub-etichetele

  • Animation Frame BuffersSpecialized memory buffering for sequential animation frames to reduce garbage collection overhead. **Distinct from Memory Buffering:** Focuses on reducing GC churn for high-frequency frame updates rather than disk I/O optimization.
  • Buffer Interface StandardizationStandardizing the interfaces for sequences of memory buffers to enable interoperable and efficient data transfer. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on the interface standardization for buffer sequences rather than the act of staging data.
  • Bytecode Buffer ReadersReads compiled Python bytecode from in-memory buffers for decompilation without requiring a file on disk. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on reading bytecode from buffers specifically, not general I/O buffering strategies.
  • Disk-Backed BufferingPersistent storage of memory buffers on the filesystem to prevent data loss during crashes. **Distinct from Memory Buffering:** Distinct from general Memory Buffering by focusing on filesystem persistence for reliability rather than just I/O optimization.
  • Disk-Memory ThresholdsLogic that toggles between memory buffering and disk persistence based on a size threshold. **Distinct from Memory Buffering:** A specific strategy for deciding storage medium based on file size, not general I/O optimization.
  • File Descriptor BufferingReading raw byte streams from system file descriptors into memory buffers. **Distinct from File Descriptor Ingestion:** Focuses on the kernel-level read from a file descriptor to a buffer, rather than web-API file reading.
  • Fixed Memory Buffers2 sub-tag-uriRuntimes that use a pre-allocated, fixed-size memory area to ensure a predictable footprint. **Distinct from Memory Buffering:** Focuses on using a fixed buffer for the entire runtime footprint rather than staging data for I/O optimization.
  • Frame Memory Buffers2 sub-tag-uriBuffers that maintain a limited set of video frames to balance accuracy and inference overhead. **Distinct from Memory Buffering:** Specifically handles video frame sequences for inference rather than general disk I/O buffering.
  • In-Memory Asset ImportsCapabilities for loading asset data directly from volatile memory buffers instead of disk files. **Distinct from Memory Buffering:** Focuses on importing 3D assets from memory, not just general I/O buffering.
  • Memory-Buffer File I/O ReplacementsAll file-accessing functions have memory-buffer equivalents, enabling fully static operation without any filesystem dependency. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on replacing file I/O entirely with memory buffers, not just staging data for disk I/O optimization.
  • Native Buffer Transfers2 sub-tag-uriOptimizations that pass data via native memory pointers to avoid expensive copying between managed and unmanaged memory. **Distinct from Memory Buffering:** Distinct from general memory buffering as it specifically focuses on zero-copy transfers between native and managed layers.
  • Optimized Buffer ReadsUsing size comparison and direct memory copy instead of looping when reading from in-memory buffers. **Distinct from Memory Buffering:** Distinct from Memory Buffering: focuses on the optimization technique for buffer reads, not general buffering strategies.
  • Prefetch BufferingConfigurations for pre-reading data chunks into memory to optimize query performance. **Distinct from Memory Buffering:** Focuses specifically on the pre-reading buffer configuration rather than general memory buffering techniques.
  • Reusable Buffer PoolsMemory structures that are recycled and adapted to incoming data to minimize allocation overhead. **Distinct from Memory Buffering:** Focuses on the reuse and adaptation of buffers for high-frequency networking rather than staging data for disk I/O.
  • Scratch-Space Work BuffersDedicated temporary memory areas used to handle input-dependent processing requirements without dynamic allocation. **Distinct from Memory Buffering:** Distinct from Memory Buffering: specifically used as a sandboxed scratch space to avoid runtime heap allocation.
  • Stream Buffering3 sub-tag-uriBuffers for streaming raw media data between processes via pipes for real-time processing. **Distinct from Memory Buffering:** Specifically focuses on piping raw media streams into memory buffers, distinct from general disk I/O buffering.
  • Transmission Window BufferingBuffers that limit the number of outstanding packets in flight to manage throughput and memory. **Distinct from Memory Buffering:** Focuses on network transmission windows rather than general disk I/O buffering.
  • Waveform Buffer PrecomputersTechniques that prepare entire waveform buffers in memory before transmission to eliminate real-time computation overhead during signal generation. **Distinct from Memory Buffering:** Distinct from Memory Buffering: precomputes entire signal waveforms before transmission, not general I/O staging or network buffering.
  • Zero-Copy Buffer Tuning3 sub-tag-uriOptimization of memory buffers specifically for zero-copy data transfers between network sockets. **Distinct from Memory Buffering:** Focuses on network socket throughput and zero-copy transfers, distinct from general disk I/O buffering.