Awesome GitHub RepositoriesData Serialization

Protocol Buffers ist ein sprachneutraler, plattformunabhängiger Mechanismus zur Serialisierung strukturierter Daten. Es bietet eine schema-gesteuerte Toolchain, die deklarative Datendefinitionen in typsicheren Quellcode kompiliert und so eine konsistente Kommunikation und stark typisierte API-Verträge über Dienste hinweg ermöglicht, die in verschiedenen Programmiersprachen geschrieben sind. Das Projekt zeichnet sich durch ein hocheffizientes binäres Wire-Format aus, das tag-basiertes Encoding und eine Komprimierung von Ganzzahlen mit variabler Breite nutzt, um Payload-Größe und Verarbeitungs-Overhead zu minimieren. Es unterstützt ein robustes, evolutionäres Schema-Management, das es Entwicklern ermöglicht, Datenstrukturen inkrementell zu aktualisieren und dabei Abwärts- und Aufwärtskompatibilität zu wahren. Dies wird zusätzlich durch ein versioniertes Edition-System unterstützt, das Feature-Sets und Serialisierungslogik über verteilte Softwarekomponenten hinweg verwaltet. Über die binäre Kern-Serialisierung hinaus umfasst das Projekt Funktionen für die kanonische JSON-Konvertierung mit Schema-Validierung, granulare Symbol-Sichtbarkeitskontrolle und Feld-Präsenzverfolgung, um zwischen Standard- und nicht gesetzten Werten zu unterscheiden. Es bietet zudem spezialisierte Optimierungen, wie Arena-basiertes Speichermanagement für C++-Implementierungen, um die Performance bei der Erstellung und Bereinigung komplexer Nachrichtenbäume zu verbessern.

FlatBuffers is a cross-platform serialization library designed for performance-critical applications that require efficient, zero-copy data access. By organizing data in a structured binary format, it allows applications to read and write complex data structures directly from memory-mapped buffers without the need for intermediate parsing or temporary object allocation. The project distinguishes itself through a schema-driven approach that balances high-performance access with long-term data evolution. It utilizes a unique memory layout featuring relative offsets and inline fixed-size structu

Gson is a Java library designed for the serialization and deserialization of objects into structured text formats. It functions as a reflection-based data mapper, inspecting class structures at runtime to automatically convert memory-resident objects into data representations and reconstruct them back into typed language objects. The library distinguishes itself through a modular type-adapter pattern that allows for custom conversion rules for complex or nested structures. It also provides robust support for production environments by generating build-time metadata and configuration rules, wh

This project is a high-performance JSON library for Go, serving as a replacement for the standard encoding/json package. It provides a serialization engine for encoding and decoding data structures, alongside a schemaless parser for extracting specific values from JSON blobs using paths. The library distinguishes itself through a focus on reducing CPU and memory overhead. It implements a high-performance encoder that utilizes buffer reuse and reduced float precision to increase data throughput. Broad capability areas include data serialization, high-performance JSON processing, and schema-le

Moshi is a JSON serialization library and parser for Kotlin and Java. It functions as a reflectionless JSON encoder that converts typed objects to JSON strings and parses JSON data back into language objects. The library distinguishes itself through compile-time adapter generation, which removes the performance overhead associated with runtime reflection. It also provides a polymorphic JSON mapper that uses type identifiers to resolve and instantiate specific subclasses of a common base type. The framework supports custom adapter definitions for specialized type conversion, including nullabi

This project is a Protocol Buffers implementation for Go, providing a binary serialization framework to convert native data structures into a compact binary format for efficient network transmission and storage. It functions as a language bindings generator, utilizing a compiler plugin to create Go source code from platform-neutral protocol buffer definitions. The implementation includes a JSON data mapper that transforms structured binary messages into JSON format to facilitate compatibility with web services and external APIs. It also enables cross-language data exchange by using a common s

Jackson is a Java data binding framework and multi-format data serializer used to translate data structures into native language objects. It functions as a JSON data binding library and a streaming parser that reads and writes data as discrete tokens to process large datasets with minimal memory. The project distinguishes itself through a bytecode serialization accelerator that replaces standard reflection with generated bytecode to increase data binding speed. It employs a module-based extensibility model to support a wide range of formats beyond JSON, including XML, YAML, CSV, TOML, and bin

orjson is a high-performance Python library for serializing and deserializing JSON data. It functions as both a JSON parsing library and a serialization engine, converting data between native Python objects and UTF-8 encoded bytes. The project provides specialized support for converting complex Python data structures, including dataclasses, datetime objects, and NumPy arrays and scalars, into JSON format. It also allows for the insertion of pre-serialized JSON blobs into documents to maintain processing speed. The library includes capabilities for memory-efficient deserialization through key

mapstructure is a reflection-based library for the bidirectional encoding and decoding of generic map data into native typed Go structures. It functions as a tool for transforming loosely typed map values into specific structs and encoding those structs back into maps. The library utilizes Go reflection to map generic data keys to struct fields during runtime. It supports the transformation of untyped data from dynamic sources or configuration files into type-safe native structures. The mapping process includes recursive traversal of nested maps and slices, as well as the use of struct tags

MessagePack is a binary object serialization library and a cross-platform data exchange format. It serves as a binary alternative to JSON, converting structured data into a space-efficient binary representation for network transmission and storage. The system provides a standardized format for swapping complex data types across different programming languages and architectures. It allows for the definition of custom data type encoding by pairing application-specific information with specialized serialization markers. The library handles the encoding and decoding of diverse data types, includ

Marshmallow is a Python data serialization library and validation framework. It functions as a bridge that transforms complex application objects into primitive data types for storage or network transmission and vice versa. The library utilizes a schema-based approach to validate that incoming data conforms to specific types and constraints. It employs a two-way transformation pipeline consisting of separate load and dump phases to handle deserialization and serialization. The framework supports class-based schema definitions with declarative field validation and recursive schema nesting for

Kryo is a Java binary serialization framework and object cloning library used to convert Java objects into a compact binary format and restore them from bytes. It provides a system for creating deep or shallow copies of complex object graphs without requiring binary conversion. The project utilizes variable-length integer encoding and integer-ID class registration to reduce the size of serialized data. It handles complex object graphs through circular reference tracking to prevent data duplication and infinite loops. To maintain long-term data viability, it includes schema evolution managemen

yaml-cpp is a C++ library for parsing and emitting YAML 1.2 documents. It provides a complete YAML processing pipeline, from reading YAML content into a traversable node tree to writing in-memory data structures back as YAML text. The library represents parsed YAML as a mutable tree of typed nodes, supporting scalars, sequences, maps, and aliases. It uses a recursive-descent parser to build this node tree, and a stream-based emitter to generate YAML output incrementally. Template-based type conversion enables compile-time serialization between YAML nodes and C++ types, including support for c

This project is a Protocol Buffers Go compiler and code generation framework that translates schema definitions into optimized Go structures and reflection-free serialization methods. It includes a gRPC service generator for producing client and server communication code and a serialization performance toolkit for generating automated benchmarks and test suites. The framework features a plugin-based system for transforming data definitions into source code with custom field mapping and struct tag injection. It allows for custom type mapping and struct type overriding to link data fields to sp

yyjson is a high-performance C library for parsing, serializing, and manipulating JSON data structures. It functions as a memory-efficient data tool that utilizes custom memory pools and contiguous block allocation to minimize fragmentation and overhead. The project distinguishes itself as a SIMD JSON processor, using hardware-accelerated vector instructions for rapid string validation and parsing. It provides a toolkit for querying, modifying, and patching nested JSON objects and arrays through pointer-based access and path expressions. The library covers a broad range of data manipulation

faster JSON serialization for Go

crates-io: https://crates.io/crates/protobuf crates-io-shields: https://img.shields.io/crates/v/protobuf.svg docs-rs: https://docs.rs/protobuf docs-rs-shields: https://img.shields.io/badge/docs.rs-rustdoc-green.svg actions:…

Bond was a cross-platform framework for working with schematized data. The open-source project ended on March 31, 2025.

API DocumentationAPI documentation

documentation