Awesome GitHub RepositoriesZero-Copy

CapnProto is a zero-copy serialization framework and remote procedure call system. It serves as a C++ communication library providing a schema-based data interchange format that eliminates the need to encode or decode data before reading it from memory. The system enables high-performance data serialization and low-latency network communication. It supports cross-language data exchange by using a defined schema to ensure consistent binary representation across different platforms. The framework provides tools for implementing remote procedure calls, allowing functions to be invoked on a remo

Phalcon is a full-stack PHP web framework implemented as a compiled C extension that loads directly into the PHP interpreter. Rather than executing PHP scripts at runtime, the framework runs as a native C module, bundling routing, ORM, templating, and caching into immutable structures compiled at build time. This architecture hooks directly into PHP's internal Zend Engine API to bypass userland function call overhead, processes HTTP requests through a C-level event pipeline, and passes data between layers using pointer references instead of duplicating memory buffers. The framework manages ob

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

The Intel RealSense SDK is a software development kit providing drivers and libraries for interfacing with depth cameras to capture color, depth, and infrared data streams. It includes a depth camera driver for device discovery and sensor configuration, a stereo vision library for computing depth maps and aligning frames, and a 3D point cloud generator to transform depth and infrared frames into spatial representations. The SDK distinguishes itself through on-chip depth calculation and stereo calibration, using internal vision processors to reduce host CPU load. It supports hardware-level str

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

Bigcache is a thread-safe, in-memory key-value store for Go designed to minimize garbage collection pauses. It functions as a memory-optimized cache capable of storing gigabytes of data while maintaining low latency during high-volume operations. The system avoids garbage collection overhead by managing memory through large byte arrays and manual allocation. It utilizes a circular-buffer layout to reuse space and lock-striped sharding to reduce contention during concurrent read and write access. The project provides capabilities for high-volume in-memory caching and low-latency data retrieva

Kitex is a high-performance remote procedure call framework for building distributed microservices in Go. It provides a communication system compatible with gRPC and Protobuf standards and includes a suite of tools for microservices governance. The framework features proxyless service mesh integration, which allows it to manage network traffic and governance directly without adding sidecar proxy latency. It optimizes data throughput through a high-performance communication library that minimizes memory copies. The system covers a broad surface of distributed capabilities, including service d

better-sqlite3 is a high-performance SQLite3 client for Node.js that executes queries synchronously, returning results directly without callbacks or promises. It compiles as a native addon using N-API, binding directly to the SQLite3 C library for immediate query execution and zero-copy result serialization into native JavaScript objects. The library is optimized for Write-Ahead Logging (WAL) mode, enabling faster concurrent reads and writes in web applications. It provides durability level tuning through the synchronous pragma, allowing adjustments between FULL, NORMAL, and OFF modes to bala

CXX is a code generator that produces a safe, zero-overhead FFI bridge between Rust and C++. It lets functions and data types from either language be used directly in the other, with static type safety enforced at compile time and no copying, serialization, or runtime checks across the boundary. The bridge supports the full range of cross-language interactions: Rust can call C++ functions and use C++ types, and C++ can call Rust functions and use Rust types. Standard library types like strings, vectors, and smart pointers are mapped automatically between the two languages, while opaque types

Warp is a Python framework that JIT-compiles Python functions into CUDA kernels for GPU-accelerated parallel computation, with built-in automatic differentiation and multi-framework array interoperability. At its core, it provides a GPU kernel compilation system that enables writing and executing custom GPU kernels directly from Python, while supporting automatic gradient computation through those kernels for integration with machine learning pipelines. The framework also includes tile-based cooperative computing, where thread blocks partition into tiles for shared-memory and tensor-core opera

Fluvio is a distributed event streaming platform and cloud-native streaming engine designed for collecting, persisting, and replicating real-time data streams across a distributed cluster. It functions as a real-time data pipeline for building stateful workflows that ingest, enrich, and export data between external sources and sinks. The platform is distinguished by its use of WebAssembly to execute compiled modules for in-line data transformations and filtering. This allows for the execution of custom business logic to reshape information in motion without requiring a restart of the cluster.

This is a cross-platform media processing library that reads, writes, encodes, and decodes media in both browser and server environments. It supports common container formats including ISOBMFF, Matroska, Ogg, MPEG-TS, and HLS, and handles codec operations through a combination of WebCodecs API and WebAssembly-based encoders. Media is processed in streaming pipelines that maintain constant memory usage and automatically apply backpressure from output speed to all upstream components. The library distinguishes itself through a plugin-based codec registration system that allows extending support

Daft is a distributed dataframe library and multimodal data processor designed to handle large-scale structured and unstructured data. It functions as a vectorized execution engine that processes tables alongside images, audio, and video, utilizing a unified schema to manage diverse data types. The project distinguishes itself by combining distributed data engineering with large-scale AI inference. It provides an AI data pipeline for batch-optimizing model prompts and generating high-dimensional text embeddings, while utilizing zero-copy memory sharing to execute custom Python functions witho

TileLang is a Python-embedded domain-specific language compiler that JIT-compiles and autotunes GPU kernels. It uses a tile-based DSL, automatic software pipelining, and parallel autotuning to generate optimized GPU kernels at runtime. It supports tensor core operations with Pythonic syntax, automatic memory management, and thread mapping. The compiler searches over tile sizes, thread counts, and scheduling policies, compiling and benchmarking candidates in parallel to find the fastest kernel. It also caches compiled binaries and tuning results to disk for reuse across sessions. TileLang inc

flutterrustbridge is a code generation tool that automatically creates type‑safe Dart bindings for Rust functions, enabling direct cross‑language calls between Flutter and a Rust backend. It provides an async FFI adapter that transforms synchronous Rust functions into Dart async methods with background thread management, a cross‑language object manager that wraps persistent Rust structs as Dart objects preserving state across calls, and trait object interop that converts Rust traits into Dart abstract classes for seamless bidirectional use. The project handles the full lifecycle of integrat

Anchor is a framework for building, testing, and deploying Solana smart contracts. It provides a Rust domain-specific language (DSL) that uses macro‑based instruction dispatch and derive‑macro account validation to automate ownership checks, discriminator enforcement, and serialization before instruction execution. The framework also includes a CLI toolchain for scaffolding, compiling, and managing contracts across workspaces, a test harness that simulates program execution with account manipulation and compute unit benchmarking, and an IDL generator that emits a standardized JSON interface fr

Fury 是一个多语言二进制序列化框架，旨在编码领域对象和复杂图，以促进跨语言数据交换。它包括一个接口定义语言（IDL）编译器，可将模式定义转换为多种语言中的惯用原生类型和序列化样板代码。 该框架通过零拷贝二进制读取器脱颖而出，该读取器允许在不反序列化整个对象的情况下访问特定字段，以及一个保留循环引用和引用完整性的对象图序列化器。它还具有一个数据转换器，可将基于行的二进制数据转换为用于分析工作负载的列式 Apache Arrow 格式。 该框架涵盖了广泛的功能领域，包括用于前向和后向兼容性的元数据驱动模式演进、用于消除运行时反射的构建时 AOT 编译过程，以及通过基于白名单的类型验证进行的安全反序列化。它还为通过 gRPC 进行的高性能远程过程调用提供了集成。

rkyv 是一个用于 Rust 的零拷贝反序列化框架，提供了一种用于内存映射数据归档的二进制序列化格式。它允许将复杂的数据结构映射到字节，并直接从缓冲区访问，而无需分配新内存或复制数据。 该项目支持多态类型和特征对象的序列化，在二进制形式中保持其动态行为和结构。它利用相对指针寻址和字节对齐的结构打包，确保数据无论加载到内存中的何处都保持有效。 该框架通过直接内存映射涵盖了高性能数据持久化和低延迟状态管理。它为应用程序提供了零拷贝数据访问和在需要时进行完整数据类型重建的机制。

redb is an embedded key-value store and ACID-compliant storage engine. It functions as a persistent storage system for saving and retrieving data as key-value pairs within a tree structure. The engine is built as an MVCC transactional database, utilizing multi-version concurrency control to manage simultaneous reads and writes without blocking. It employs a single-writer multi-reader model to ensure data consistency while allowing multiple threads to access the store. The system provides persistent state management and atomic transaction management to prevent data corruption during crashes.

liburing is a C library that provides a low-level wrapper for the Linux kernel io_uring interface. It serves as a programming interface for executing non-blocking disk and network operations and abstracts the system calls required to communicate with the Linux kernel. The library focuses on reducing system call overhead and context switching for high-throughput data processing. It implements mechanisms for shared ring buffers, zero-copy buffer registration, and fixed-file descriptor mapping to minimize internal lookup and reference counting overhead. The project covers asynchronous input and