Awesome GitHub RepositoriesConcurrent Data Processors

This library is a collection of generic utilities for the Go programming language designed to simplify the manipulation of slices and maps. It provides a functional toolkit that enables developers to perform data transformations, such as filtering, mapping, and reducing, while maintaining strict type safety through the use of language-level generics. The project distinguishes itself by offering a dual approach to data processing that balances functional programming patterns with performance-oriented execution. It supports both immutable functional pipelines for predictable state transitions a

brpc is a high-performance C++ RPC framework and network programming library designed for building distributed systems. It functions as a multi-protocol RPC server capable of hosting and detecting multiple communication protocols, including gRPC, Thrift, HTTP, Redis, and Memcached, on a single TCP port. The project distinguishes itself through high-throughput data transport and memory efficiency, utilizing RDMA-based transport to bypass the kernel TCP stack and zero-copy memory management to eliminate data duplication. It also implements the Raft algorithm for consensus-based state replicatio

Badger is an embeddable key-value store written in Go that provides persistent data storage for byte keys and values. It is a persistent database that utilizes a tiered LSM tree storage model to optimize disk storage and retrieval efficiency. The system features an ACID transaction engine that ensures data integrity through serializable snapshot isolation and multi-version concurrency control. It also provides an encrypted key-value store with data-at-rest encryption and a managed encrypted key registry to secure stored information. The engine covers a broad set of capabilities including hig

Dolphin is a multimodal layout analyzer and image-to-structure converter that transforms photographed or digital document images into machine-readable structured data. It functions as an LLM document parser, utilizing vision-language models to simultaneously predict spatial layout and text content. The system is designed as a concurrent document processor, employing parallel document parsing to process multiple elements across distributed compute nodes. This high-throughput approach reduces the total time required to convert large volumes of images into structured formats. The project covers

whatsmeow is a Go library that implements the WhatsApp multidevice protocol, providing direct access to the binary node serialization, WebSocket-based connection management, and Signal‑Protocol encryption that underpin WhatsApp’s messaging infrastructure. It maintains persistent, low-latency connections across multiple devices, handles session credentials independently for each device, and processes incoming events through an event‑driven dispatch system for messages, receipts, and presence updates. The library differentiates itself through comprehensive client‑side state management: it cache

RxGo 是一个函数式响应式编程库，也是 ReactiveX 的 Go 语言实现。它作为一个异步流处理工具包，旨在利用可观察模式协调基于事件的程序和数据流。 该库支持构建异步处理管道，以转换、过滤和组合事件序列。其特色在于使用函数式操作符来组合这些管道，并提供了管理并发执行的机制。 该工具包涵盖了广泛的流编排功能，包括数据聚合、多流组合以及将流转换为静态数据结构。它内置了对错误恢复、用于调节数据生产速度的背压控制以及用于跨 CPU 核心并行处理的工作池的支持。

FirebaseChat 是一个实时聊天应用程序和无服务器消息客户端，可在客户端之间即时同步对话和消息。它为基于云托管 NoSQL 文档存储的消息系统提供了用户界面。 该项目通过集成 Firebase 后端实现了实时聊天系统，支持即时消息工作流和跨连接客户端的数据同步。 应用程序使用无模式 NoSQL 存储和基于文档的数据建模来管理数据。它利用持久连接进行数据库同步，并通过事件驱动的消息分发和客户端状态投影来处理状态更新。

Streem 是一种基于流的编程语言和数据流水线编排器。它提供了一种用于定义并发数据流的领域特定语言（DSL），允许用户通过一系列转换和过滤单个流元素的运算，将数据源链接到目的地。 该系统使用自定义脚本语法来定义数据流连接和流水线定义。这允许编排并发数据处理，其中多个流水线阶段同时执行，以在系统中移动数据元素。 该平台涵盖函数式数据转换和基于阶段的组合，在项目通过链接运算的顺序链时应用特定函数来修改或过滤项目。

Ultrajson 是一个高性能 Python JSON 库和基于 C 的解析器，旨在最大限度地减少解析开销。它提供了 JSON 规范的 C 实现，以实现数据的快速编码和解码。 该项目作为一个并发 JSON 处理器，支持多线程序列化和反序列化，以提高数据吞吐量并防止阻塞执行。 其功能涵盖 Python 数据序列化和用于实时数据管道的高性能 JSON 处理。这包括解析 JSON 数据并将原生对象序列化为 JSON 字符串。

Concurrent-map is a lock-striped hash map and sharded concurrent cache for Go, designed as a high-performance key-value store that enables thread-safe parallel reads and writes with minimal blocking. It replaces a single global mutex with per-shard locking, using hash-based key distribution to assign entries to independent segments, allowing multiple goroutines to operate simultaneously without race conditions. The library achieves its performance through fine-grained locking and a lock-free read path, where each shard operates independently with its own lock, enabling parallel reads and writ

SSH.NET is a .NET library that implements the SSH-2 protocol for encrypted remote connections and secure file transfers. It provides a complete SSH-2 protocol stack implementation with a channel multiplexing engine that manages multiple concurrent channels over a single connection, supporting simultaneous shell sessions, remote command execution, SFTP transfers, and port forwarding tunnels. The library includes a pluggable authentication pipeline supporting password, public key, certificate, keyboard-interactive, and multi-factor authentication combinations. The library distinguishes itself t

OpenMQTTGateway 是一个专为 ESP32 和 ESP8266 设备设计的多协议物联网网关和硬件接口。它充当桥梁，将蓝牙低功耗（BLE）、射频（RF）和红外（IR）信号转换为标准化的 MQTT 消息，供家庭自动化系统使用。 该项目为 433MHz、315MHz 和 868MHz 无线电信号以及 LoRa 和红外命令提供了特定的桥接功能。它包含一个蓝牙存在检测器，通过监控设备广播来跟踪人员或物体的位置与接近度。 该系统通过被动扫描和异步无线电轮询处理物联网传感器聚合，并通过硬件特定的有效载荷解码将原始二进制数据转换为 JSON 格式。它还支持双向命令路由，以管理连接的无线设备和智能电源插座。

Melody is a WebSocket server framework designed to upgrade HTTP connections into bidirectional streams. It functions as a session manager and binary streaming library that facilitates real-time data exchange between a server and multiple remote clients. The project features a broadcast engine that optimizes multi-client transmissions through wire-representation caching. It includes a dedicated session management system for attaching custom state to persistent network connections and an upgrade handler that validates handshake headers. The framework covers broad capability areas including eve

该项目是一个教育资源，旨在帮助开发者掌握 Go 中的并发执行和同步模式。它提供了一个结构化的动手编码练习和测试驱动挑战集合，专注于多线程软件工程实践的实际应用。 该仓库强调 Go 并发的核心机制，包括基于通道的通信、互斥锁和生命周期协调。它涵盖了诸如管理后台任务、实现优雅的服务关闭以及强制执行速率限制或执行时间约束等基本模式。通过完成这些练习，用户可以学习保护共享内存结构并在并发操作期间防止竞态条件。 这些材料包含利用运行时检测来验证并发逻辑正确性的自动化测试工具。这些练习引导学习者掌握协调多个协程的细微差别，确保共享资源在保持系统吞吐量的同时保持线程安全。