Awesome GitHub RepositoriesAvro Decoding

fq 是一个命令行二进制数据处理器，用于将原始字节流和位级数据解码、转换和分析为结构化格式。它作为一个函数式二进制查询引擎，允许对二进制结构进行过滤和映射，同时也是一个转换器，将复杂的二进制 Blob 和专有文件格式转换为标准的 JSON、YAML 或 XML。 该工具作为低级位操作器，能够对原始文件执行位级切片、位运算和加密哈希。它还作为一个网络协议分析器，能够重组碎片化的 TCP 流并解密 TLS 流量以进行应用级检查。 该项目涵盖了二进制解析和数据转换方面的广泛功能，包括对自定义解码器定义的支持，以及对 Mach-O、ASN1 BER 和 Avro OCF 等多种专业格式的支持。它提供了用于二进制树搜索、结构化文本解码以及二进制与文本格式之间双向序列化的工具。 用户可以通过命令行界面和用于实时查询测试的交互式 REPL 与系统进行交互。

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

Materialize is a streaming SQL database that continuously ingests live data from sources such as Kafka, Redpanda, PostgreSQL, and MySQL, and incrementally maintains materialized views. It provides a PostgreSQL-compatible query engine that accepts standard SQL over the PostgreSQL wire protocol, enabling any existing SQL client or BI tool to query real-time data. The system also includes a Model Context Protocol (MCP) server that exposes live materialized view data to AI agents, providing fresh context without polling. Materialize distinguishes itself through its ability to offer configurable c

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

CloudEvents is an open specification for describing event data in a common format across cloud platforms and services. It defines a standard structure and set of metadata attributes for events, enabling interoperability across different systems so producers and consumers can exchange events without custom translation. The specification provides a protocol-agnostic serialization framework that maps CloudEvents attributes and payloads to multiple serialization formats including JSON, Avro, and Protobuf, and defines transport bindings for mapping events onto protocols like HTTP, AMQP, Kafka, MQTT

kcat 是一个用于 Apache Kafka 的命令行接口客户端，用于使用原生线路协议生产、消费和调试消息。它提供了一套用于与 Kafka 集群交互的工具，包括用于检查集群元数据的协议调试器和用于处理原子消息批次的事务管理器。 该项目具有一个专门的 Avro 模式解码器，通过与远程模式注册表或本地文件集成，将二进制编码的消息转换为人类可读的 JSON。此外，它还包含一个内存模拟器，允许通过模拟短暂的 Broker 行为来测试生产者和消费者逻辑，而无需外部基础设施。 该工具集涵盖了广泛的消息传递操作，包括平衡消费者组支持、基于时间戳的偏移量查找以及来自标准输入的事务性数据流。它还提供了用于连接安全配置和集群元数据检查的实用程序。

Racket 是一种通用的、多范式编程语言，属于 Lisp 家族，专为语言创建而设计。它作为一个语言工作台，通过灵活的宏和模块系统，为设计和实现自定义编程语言提供了一个平台。 该系统的特色在于提供了一套全面的语义工程套件，允许构建专门的语言子集和教育层。它包括用于自定义语言设计的工具，如词法分析器和解析器生成，以及在读取时定义模块扩展规则和动态语言选择的能力。 该项目提供了一个集成开发环境，内置编辑器、可视化调试器和软件包管理器。其功能范围扩展到涵盖 2D 图形渲染、二进制数据处理、SQL 和演绎数据库集成以及图形用户界面构建的通用标准库。 该环境支持将源代码编译为独立的二进制可执行文件以进行分发。

Arroyo is a high-performance stream processing platform built in Rust. It executes continuous SQL queries on streaming data with event-time semantics, enabling accurate windowed aggregations, joins, and stateful computations on unbounded event streams. The platform uses native Rust execution for high throughput and low latency, with periodic checkpointing for exactly-once fault tolerance and horizontal scaling across distributed workers. The system integrates deeply with Kafka for reading and writing topics with exactly-once delivery and supports change data capture (CDC) from MySQL and Postg

AKHQ is a web-based management interface for Apache Kafka, providing a centralized platform for administering clusters, topics, and consumer groups. It serves as a comprehensive monitoring and administration tool that includes a Kafka Connect manager and a ksqlDB administration interface. The platform distinguishes itself through extensive schema registry integration, allowing users to browse and decode Avro, Protobuf, and JSON messages using Confluent, Tibco, or AWS Glue registries. It also features a granular security model with role-based access control, sensitive data masking, and support