16 个仓库
Tools for reading and parsing binary data formats without full deserialization.
Distinguishing note: Focuses on zero-copy or partial-read access patterns for binary buffers, distinct from general-purpose database drivers.
Explore 16 awesome GitHub repositories matching data & databases · Binary Data Accessors. Refine with filters or upvote what's useful.
FlatBuffers is a cross-platform serialization library designed for performance-critical applications that require efficient, zero-copy data access. By organizing data in a structured binary format, it allows applications to read and write complex data structures directly from memory-mapped buffers without the need for intermediate parsing or temporary object allocation. The project distinguishes itself through a schema-driven approach that balances high-performance access with long-term data evolution. It utilizes a unique memory layout featuring relative offsets and inline fixed-size structu
Reading specific fields from large binary buffers without the need for full deserialization or temporary object allocation.
This project is a comprehensive JavaScript programming tutorial and language reference. It serves as a web development education resource providing instruction on modern language fundamentals, object-oriented design, and advanced asynchronous programming patterns. The resource functions as both a frontend development guide and a technical reference. It covers core language features such as closures, prototypes, promises, and typed arrays, while providing practical lessons on managing browser data and handling network requests. The content spans several key capability areas, including browser
Explains how to read and interpret bytes from raw binary buffers at specific offsets.
The ECMAScript specification is the formal standard defining the syntax, semantics, and execution model that all JavaScript implementations must follow. It establishes the official language rules through a combination of formal grammar and step-by-step algorithmic prose. The project manages the technical evolution of the language via a consensus-driven governance framework and a staged proposal pipeline. This process tracks features from initial design through expert reviewer sign-off and committee approval. A specification-as-code toolchain compiles these formal definitions and algorithmic d
Provides views that allow reading and writing multiple numeric types directly to an underlying binary buffer.
Thrift 是一个跨语言远程过程调用(RPC)框架和数据序列化协议。它提供了一种接口定义语言(IDL)来以中立格式指定数据类型和服务接口,从而实现跨多种编程语言的客户端和服务器代码的自动化生成。 该项目作为一个多语言服务通信器,使用分层软件栈来确保互操作性通信。它专注于实现跨语言远程过程调用,并将复杂数据结构转换为标准化格式以实现高效的网络传输。 该框架包括用于管理 API 版本兼容性的功能,以支持客户端和服务器之间的非原子更新。它还提供了用于验证数据完整性并确保服务在不同语言实现中行为一致的机制。
Supports non-atomic updates by ignoring unknown fields during binary deserialization, ensuring backward and forward compatibility.
Sled is an embedded key-value store and ACID-compliant database designed for high-performance data persistence. It functions as a log-structured storage engine that organizes data using B+ trees to support efficient range queries and prefix scans. The engine implements a zero-copy data store model, utilizing epoch-based reclamation to provide direct references to cached values without memory allocations. It distinguishes itself through a combination of write-ahead logging, page cache optimizations to reduce write amplification on flash storage, and serializable transactions for atomic multi-k
Allows reconstructing structured data from byte slices using specific configurations.
Sui is a blockchain platform featuring an object-centric state model and resource-oriented smart contracts. It utilizes parallel transaction execution to increase network throughput and supports programmable transaction blocks that bundle multiple operations into single atomic units. The platform distinguishes itself with a capability-based access control system and zero-knowledge login mechanisms, enabling users to authenticate via identity providers without seed phrases. It also implements deterministic object addressing to allow predictable state lookups and supports the creation of soulbo
Implements binary deserialization to reconstruct structured data from BCS serialized byte slices.
ArduinoJson is a C++ library for parsing and manipulating JSON data and MessagePack binary streams on microcontrollers with limited memory and processing power. It provides the core primitives necessary for embedded data serialization and parsing, enabling devices to exchange structured data over serial or network interfaces. The library is distinguished by its focus on microcontroller memory management, employing strategies such as pool-based allocation, string deduplication, and non-owning string views to minimize RAM usage. It further optimizes for constrained environments by allowing cons
Provides direct reading and writing of raw binary data embedded within MessagePack streams.
MessagePack-CSharp is a high-performance binary serialization library for .NET applications that converts object graphs into the MessagePack format. It functions as a C# data serialization toolkit and a polymorphic binary encoder capable of handling abstract classes and interfaces using union keys to identify concrete derived types. The library provides a binary format transcoder to transform binary data into human-readable JSON for debugging. It supports ahead-of-time formatter generation to avoid runtime overhead and implements LZ4 binary compression to reduce the size of serialized data.
Reconstructs structured object graphs from raw binary blobs using both typed and dynamic deserialization.
MessagePack-CSharp is a high-performance binary serializer for .NET that converts C# objects to and from the compact MessagePack format. It uses compile-time source generation to produce AOT-safe formatters and resolvers, eliminating runtime reflection and enabling ahead-of-time compilation scenarios. The serializer encodes object fields as integer indices instead of string keys, producing compact binary output with deterministic field ordering, and provides stack-allocated reader and writer structs for direct encoding and decoding of MessagePack primitives without heap allocations. The libra
Deserializes binary data with missing or extra fields by ignoring unknown keys and applying defaults.
protobuf-net 是一个二进制序列化框架和 .NET 库,实现了 Protocol Buffers 规范。它作为一个基于模式的序列化工具和契约优先的数据映射器,将复杂的对象图转换为紧凑的二进制格式,用于存储和网络传输。 该库通过减小有效载荷大小和处理时间,实现了跨平台数据交换和高性能网络通信。它特别支持对象层次结构持久化,允许序列化和重构包含继承和派生类型的复杂类结构。 该项目涵盖了核心功能,包括二进制数据序列化和反序列化、模式驱动的类生成,以及通过唯一标识符管理类型继承。它提供了通过静态属性或动态运行时配置映射数据结构的机制。
Reconstructs complex .NET objects from raw binary streams to restore original application state.
这是一个 Protocol Buffers Swift 库和框架,为序列化和反序列化结构化数据提供运行时支持。它包含一个代码生成器,可将模式文件转换为类型安全的 Swift 结构,以及一个用于将数据转换为紧凑流的二进制序列化框架。 该库充当 JSON 数据映射器,将消息转换为标准 JSON 表示,以促进跨平台数据交换。它利用模式驱动的方法确保 Swift 应用与其他语言编写的服务之间的数据建模一致性。 该框架涵盖二进制和 JSON 数据的序列化与反序列化,利用基于接口的运行时库来处理生成的各种消息格式。
Reconstructs structured data from compact binary streams using predefined protobuf schemas.
Prost 是一个 Rust 语言的 Protocol Buffers 实现,作为二进制序列化框架和代码生成器使用。它将模式定义转换为符合 Rust 习惯的结构体(structs)和枚举(enums),并提供将结构化数据编码和解码为 Protocol Buffers 二进制格式所需的逻辑。 该项目的特色在于其 no-std 实现,使其能够在缺乏标准库的嵌入式或内核环境中运行。它还通过发出文件描述符集(file descriptor sets)充当模式内省器,从而支持在运行时对原始模式定义进行程序化分析。 该库涵盖了广泛的功能,包括用于接口建模的服务 trait 生成、对已知类型的支持,以及用于映射手动类型定义的程序宏注解。它还提供了依赖管理和模糊测试工具,以确保序列化的稳定性。
Transforms binary streams back into structured Rust data objects through efficient deserialization.
Fury 是一个多语言二进制序列化框架,旨在编码领域对象和复杂图,以促进跨语言数据交换。它包括一个接口定义语言(IDL)编译器,可将模式定义转换为多种语言中的惯用原生类型和序列化样板代码。 该框架通过零拷贝二进制读取器脱颖而出,该读取器允许在不反序列化整个对象的情况下访问特定字段,以及一个保留循环引用和引用完整性的对象图序列化器。它还具有一个数据转换器,可将基于行的二进制数据转换为用于分析工作负载的列式 Apache Arrow 格式。 该框架涵盖了广泛的功能领域,包括用于前向和后向兼容性的元数据驱动模式演进、用于消除运行时反射的构建时 AOT 编译过程,以及通过基于白名单的类型验证进行的安全反序列化。它还为通过 gRPC 进行的高性能远程过程调用提供了集成。
Reconstructs original typed objects from binary payloads using a shared schema to ensure type safety.
fastjson2 is a high-performance Java library used for serializing and deserializing Java objects to and from JSON strings and byte arrays. It functions as a binary JSON serializer for encoding objects into optimized binary formats to reduce payload size and increase processing speed. The project includes a JSONPath query engine for extracting specific data and fields from JSON documents using standardized path expressions and filters. It also serves as a JSON schema validator, providing utilities to generate data contracts and validate JSON input against defined schemas to ensure structural c
Reconstructs Java objects from binary data using embedded type information for automatic resolution.
Fory is a cross-language serialization framework and binary data serializer designed to convert complex object graphs into a compact binary format for high-performance data exchange. It includes an IDL-based schema compiler to transform interface definition language files into type-safe native data models and a schema evolution manager to maintain forward and backward compatibility. The project features a zero-copy data access layer that allows reading specific fields from binary rows without deserializing the entire object. It supports dual-mode serialization, enabling a toggle between a por
Reconstructs native language objects from binary streams based on a shared schema.
NBTExplorer is a viewer and editor for Named Binary Tag data. It functions as a processor for the hierarchical binary tag files used within Minecraft save files and game configuration. The tool enables the exploration and modification of game world settings, player data, and entity states. It supports reading and updating structured binary data across standard, uncompressed, and region-based file formats. The project provides capabilities for binary tag deserialization and in-place modification. It handles data through buffered stream reading, zlib-based decompression, and region-file indexi
Implements binary deserialization to convert raw NBT streams into structured tree objects.