Awesome GitHub RepositoriesBinary Data Formats

This library is a header-only C++ framework designed for the serialization, parsing, and manipulation of structured data. It provides a recursive variant data model that represents arbitrary structures as native types, allowing developers to navigate, modify, and patch hierarchical data using standard pointer paths and declarative update patterns. The project distinguishes itself through its template-based architecture, which enables type-safe data structures and custom type mapping at compile time. It offers a unified interface for binary format abstraction, supporting multiple compact repre

This project is a comprehensive platform for quantitative investment research, machine learning, and algorithmic trading. It provides an end-to-end environment for developing, testing, and executing financial strategies, supporting the entire lifecycle from data ingestion and feature engineering to model training and backtesting. The system is distinguished by its configuration-driven workflow orchestration, which allows researchers to automate complex pipelines and manage experiments through declarative files. It features a high-performance data infrastructure that utilizes custom binary for

ET is a C# game server framework and distributed actor model runtime designed for large-scale multiplayer environments. It provides a comprehensive toolkit for building distributed game backends, incorporating a multiplayer network transport layer and a specialized suite for game AI and pathfinding. The framework is distinguished by its use of a distributed actor model to scale processing across multiple threads and servers, utilizing isolated actors for state management and messaging. It features a unified codebase architecture that allows shared logic between the server and client, enabling

This project is an educational resource and a collection of instructional materials for performing data manipulation and statistical analysis using Python. It provides a comprehensive set of guides and code examples for using the Pandas, NumPy, and Matplotlib libraries to analyze structured data. The resource includes a dedicated guide for reshaping, cleaning, and aggregating tabular data and time series via Pandas, alongside a reference for high-performance vectorized operations and linear algebra using NumPy. It also features tutorials for creating publication-quality charts, distribution p

The mongo-go-driver is a Go library for building applications that integrate with a MongoDB document store. It enables the storage and retrieval of flexible document data by providing a bridge between Go backends and the database. The driver implements specialized capabilities for semantic vector search, allowing the handling and execution of high-dimensional vector data for similarity-based retrieval. It also supports full-text search via linguistic analysis and programmatic search index management. The project covers a broad range of database operations, including document-based CRUD, bulk

asyncpg is an asynchronous database driver and binary protocol client for PostgreSQL. It provides a non-blocking interface for executing SQL statements, streaming result sets, and managing data transfer between an application and a PostgreSQL database. The driver implements the PostgreSQL binary protocol directly to facilitate efficient data transfer and type conversion. It includes a connection pool to maintain and reuse open database connections, reducing the latency associated with repeated handshakes. The project covers a broad range of database integration capabilities, including atomic

MessagePack is a binary object serialization library and a cross-platform data exchange format. It serves as a binary alternative to JSON, converting structured data into a space-efficient binary representation for network transmission and storage. The system provides a standardized format for swapping complex data types across different programming languages and architectures. It allows for the definition of custom data type encoding by pairing application-specific information with specialized serialization markers. The library handles the encoding and decoding of diverse data types, includ

MessagePack-CSharp is a high-performance binary serializer for .NET that converts C# objects to and from the compact MessagePack format. It uses compile-time source generation to produce AOT-safe formatters and resolvers, eliminating runtime reflection and enabling ahead-of-time compilation scenarios. The serializer encodes object fields as integer indices instead of string keys, producing compact binary output with deterministic field ordering, and provides stack-allocated reader and writer structs for direct encoding and decoding of MessagePack primitives without heap allocations. The libra

Racket 是一种通用的、多范式编程语言，属于 Lisp 家族，专为语言创建而设计。它作为一个语言工作台，通过灵活的宏和模块系统，为设计和实现自定义编程语言提供了一个平台。 该系统的特色在于提供了一套全面的语义工程套件，允许构建专门的语言子集和教育层。它包括用于自定义语言设计的工具，如词法分析器和解析器生成，以及在读取时定义模块扩展规则和动态语言选择的能力。 该项目提供了一个集成开发环境，内置编辑器、可视化调试器和软件包管理器。其功能范围扩展到涵盖 2D 图形渲染、二进制数据处理、SQL 和演绎数据库集成以及图形用户界面构建的通用标准库。 该环境支持将源代码编译为独立的二进制可执行文件以进行分发。

pgx 是一个用于使用 Rust 语言开发高性能 PostgreSQL 扩展的框架和工具套件。它提供了一个底层 API 包装器，用于与内部数据库内存上下文、日志系统和核心执行 API 进行交互，从而能够直接在数据库引擎内实现自定义的数据库功能和逻辑。 该项目的特色在于其专用的构建工具和命令行接口，用于管理从环境初始化到二进制打包的扩展开发生命周期。它包含一个类型映射器，可将语言结构转换为数据库复合类型，并自动生成相应的 SQL 模式定义。 该框架涵盖了广泛的功能面，包括用户定义函数映射、二进制协议集成，以及多版本目标支持，以确保跨不同数据库版本的兼容性。它还提供了专门的内存管理包装器来处理指针并防止数据库环境中的内存泄漏。

Cereal 是一个 C++ 序列化库和对象持久化工具，用于将数据类型和容器转换为存储或传输格式。它以仅头文件（header-only）库的形式实现，允许直接包含在源代码中，无需编译二进制文件。 该库支持多种数据表示形式，包括二进制、XML 和 JSON。它提供了定义自定义归档（archive）的能力，从而能够开发专门的输出格式来控制数据的编码和存储方式。 该系统通过基于模板的序列化和递归树遍历来处理复杂对象的转换。这些能力支持应用程序状态持久化以及进程间的二进制数据交换。

Kaitai Struct 是一个声明式二进制规范框架，提供跨多种编程语言的自动化解析器和序列化器生成。它使用一种可移植的领域特定语言来描述二进制文件和网络数据包的布局，并将这些规范转换为用于读取和写入二进制数据的源代码。 该系统通过将单一结构定义翻译为包括 C++、C#、Go、Java、JavaScript、Python 和 Rust 在内的多种语言的原生源代码，实现了跨语言数据处理。这种方法允许在不同平台上实现相同的二进制数据规范，而无需手动重写解析逻辑。 该框架涵盖二进制数据建模和处理，包括管理用于偏移导航的读取指针以及应用解压和解密等数据转换的能力。它还提供格式规范调试工具，以验证二进制数据布局与其声明性描述之间的映射。

meta-rules-dat is a collection of binary-encoded network datasets used to identify and categorize traffic for routing on resource-constrained devices. It provides a structured domain categorization list and a geographic IP routing dataset to map network traffic to specific countries or service providers. The project utilizes trie-based lookup data and compact binary serialization to enable high-performance prefix matching and fast domain-to-category resolution. To minimize memory and storage overhead, it employs stripped-down GeoIP mapping that removes non-essential metadata. The datasets co

Odin Serializer 是一个为 Unity 游戏引擎设计的框架，用于将复杂的对象图转换为持久化的二进制或 JSON 格式。它提供了一个用于保存和加载复杂类结构的系统，包括嵌套泛型和多态类型，同时保持数据的结构完整性。 该框架通过其对提前（AOT）编译的支持脱颖而出，在构建过程中生成必要的类型元数据，以确保在限制运行时反射的平台上功能正常。它直接与 Unity 引擎生命周期集成，允许序列化引擎特定的类型，并通过唯一的持久化标识符管理外部资产引用。 用户可以定义细粒度的序列化策略，以控制在转换过程中包含或排除哪些类成员。此功能允许通过确保仅处理所需信息来优化数据存储和传输，而底层引擎处理复杂继承层次结构的递归映射。

This project provides a collection of command-line administrative utilities designed for managing MongoDB database instances. It serves as the primary interface for performing database backups, restoring binary snapshots, and executing data migration tasks between database instances and external systems. The tools facilitate the movement of information by importing and exporting collections across various formats, including JSON, CSV, TSV, and binary files. Beyond migration, the suite includes specialized utilities for inspecting binary data files in a human-readable format and managing large