15 个仓库
Models that process data as discrete packets for efficient manipulation.
Explore 15 awesome GitHub repositories matching data & databases · Packet-Based Stream Processors. Refine with filters or upvote what's useful.
FFmpeg is a cross-platform multimedia framework designed for the recording, conversion, and streaming of audio and video content. It functions as a comprehensive toolkit that provides both a command-line utility for direct media manipulation and a collection of low-level libraries for integration into custom applications. At its core, the project utilizes a packet-based stream engine and a format-agnostic abstraction layer to handle diverse media standards, containers, and network protocols. The framework distinguishes itself through a modular, graph-based filter execution model that allows f
Manages media data as discrete, timestamped packets to facilitate efficient routing and manipulation across the pipeline.
MediaPipe is a cross-platform machine learning framework designed for building and deploying pipelines that process live and streaming media. It provides a system for connecting processing components into custom machine learning chains to analyze real-time audio and video streams. The framework includes a suite of pre-trained models for tasks such as hand, face, and pose tracking, along with tools for retraining and customizing these models with specific datasets. It also features a dedicated benchmarker for measuring the execution speed and accuracy of machine learning models directly within
Uses typed data packets to communicate between pipeline components, ensuring synchronized media frames and timestamps.
PyMySQL is a MySQL database connector and SQL database driver for Python environments. It serves as a client library that enables Python applications to establish network connections and interact with MySQL database servers. The library is a pure Python implementation of the MySQL client-server wire protocol. This design allows for MySQL integration and data management without requiring native C extensions. The project provides capabilities for database connectivity, query execution, and secure access control. It follows the DB-API 2.0 specification to provide a consistent interface for mana
Implements buffer-based parsing to decompose raw binary network streams into discrete MySQL protocol packets.
libhv is a high-performance C/C++ network library and event-driven I/O framework used to build TCP, UDP, SSL, HTTP, WebSocket, and MQTT clients and servers. It provides a non-blocking event loop for managing network sockets, timers, and system signals across multiple threads. The project is distinguished by its integrated support for specialized network roles, including a full HTTP web server with RESTful routing and middleware, an MQTT messaging client for IoT communication, and the ability to implement SOCKS5 and HTTP proxies. It also features a reliable UDP implementation to ensure ordered
Reconstructs application messages from raw byte streams using fixed-length, delimiter-based, or length-field parsing.
Mineflayer is a Node.js game automation framework and JavaScript API for creating programmable Minecraft bots. It functions as a headless game client that implements the Minecraft protocol to connect external applications to game servers without a graphical user interface. The project enables the automation of character actions, including movement, combat, and interaction with the game world. It provides specialized systems for calculating navigation routes through 3D environments, manipulating blocks to build or mine, and managing user account authentication and session tokens. The framewor
Parses raw TCP streams into structured JavaScript objects according to the Minecraft binary protocol.
Trippy is a network diagnostic tool and terminal user interface that combines ping and traceroute functionality to analyze network paths and performance. It provides a network packet parser and path analyzer to detect packet loss, jitter, and multipath routing through continuous network probing. The project is distinguished by its support for specialized tracing strategies, including Paris and Dublin, to ensure consistent path discovery. It identifies network address translation devices and utilizes heuristic analysis to distinguish genuine packet loss from rate-limiting. Broad capability ar
Decodes raw packet wire formats and ICMP extensions into structured diagnostic data.
esptool 是一个基于 Python 的串口实用程序,专为 Espressif 片上系统(SoC)设备的固件烧录、硬件熔丝配置和配置供应而设计。它作为一个串口固件烧录器和硬件配置工具,通过串口与芯片交互,以实现程序执行和设备设置。 该工具提供了管理 Espressif 硬件的专门功能,包括设置永久电子熔丝以锁定安全设置和定义硬件级配置的能力。它还具有一个串口通信桥,可将网络流量路由至串口,从而授予对设备控制台的远程访问权限。 该项目广泛涵盖了嵌入式设备诊断和底层芯片交互。它通过二进制镜像烧录管理固件部署,并通过加密完整性验证确保安全性。
Implements a byte-level packet protocol with headers and footers to synchronize host and bootloader communication.
ssh2 is a JavaScript implementation of the SSH2 protocol for Node.js, providing the core components necessary to create secure clients and servers. It enables the establishment of authenticated sessions for remote server automation and secure communication. The project distinguishes itself by providing a complete suite of tools for identity management, including utilities for generating and parsing cryptographic key pairs and integrating with external authentication agents. It also functions as a tunneling proxy capable of routing TCP, HTTP, X11, and SOCKSv5 network traffic through encrypted
Implements length-prefixed packet framing to synchronize encrypted data streams over TCP.
This project provides a technical specification and reference manual for the MQTT 3.1.1 standard, serving as a translated Chinese guide for implementing this lightweight IoT messaging protocol in resource-constrained device networks. The documentation covers the fundamental mechanisms of the publish-subscribe model, including topic-based routing with wildcard filtering, retained message storage, and the configuration of last will and testament messages. It details the protocol's approach to reliability through various quality of service levels, ranging from single-level acknowledgments to exa
Assigns unique identifiers to packets to guarantee reliable delivery and acknowledgement across service levels.
MQTTnet 是一个 .NET 库,提供了一个用于实现 MQTT 客户端和代理(broker)的框架。它包括一个用于托管中央消息枢纽的代理实现,用于在连接的客户端之间路由消息,以及一个用于发布和订阅消息的客户端库。 该项目支持 TCP 和 WebSocket 传输层,通过传输层安全(TLS)实现安全的数据传输。它允许开发自定义中间件,以在消息通过服务器时进行拦截、转换或验证。 能力领域包括分布式消息路由、具有服务质量(QoS)队列的持久连接管理,以及用于客户端凭据验证和主题订阅限制的安全原语。
Transforms raw byte streams into structured protocol frames through a sequential processing pipeline.
go-mysql 是一个用于 MySQL 数据库集成的工具包,提供了一个原生客户端驱动程序,以及实现自定义 MySQL 兼容服务器或代理的功能。它使应用程序能够通过标准 SQL 接口模式与 MySQL 服务器进行通信。 该项目通过流式传输二进制日志事件,为实时数据同步提供了专门支持。这允许捕获数据库变更并将其推送到外部数据存储、搜索引擎或缓存中,以保持一致性。 该库包括用于内存高效的大型数据集处理工具,利用查询结果流式传输和回调机制来单独处理行。它还涵盖了底层协议解析、有状态连接池和自定义数据包响应逻辑。
Provides low-level protocol packet parsers to decode raw bytes into structured MySQL packets.
go-mysql 是一个为 Go 编程语言设计的工具包,旨在实现 MySQL 协议处理程序、二进制日志复制系统、模拟服务器和模式解析工具。它提供了通过原生网络协议和二进制日志流与 MySQL 服务器交互所需的库。 该项目通过解析和解码底层网络数据包,支持创建自定义数据库代理或模拟服务器。它还包括一个用于流式传输和处理二进制日志以在不同存储系统间同步数据的复制工具,以及一个用于提取和分析数据库结构定义的模式解析器。 该工具包涵盖了广泛的数据库集成功能,包括连接管理、查询执行以及实时数据复制工作流的管理。它还支持模式迁移的自动化以及将记录同步到外部存储系统。
Decomposes raw binary network streams into protocol-specific MySQL packets for direct server communication.
Engine.io 是一个实时传输层和跨浏览器通信引擎。它作为一种底层双向通信库,旨在管理实时应用所需的底层网络连接和数据交换。 该库提供了一种 WebSocket 传输机制,并在双向协议不可用时自动回退到 HTTP 长轮询。它是 Socket.IO 的核心基础设施,支持从标准 HTTP 握手升级为持久的 TCP 连接。 其功能包括用于检测连接中断的心跳监控、用于管理连接生命周期的状态机,以及用于消息序列化的基于数据包的帧编码。该系统抽象了传输层,以确保在不同 Web 浏览器和设备环境中实现一致的数据流传输。
Defines structural headers and markers to frame data packets for consistent message sequencing and delivery.
该项目是一个用于 Node.js 的 MySQL 数据库驱动程序,通过原生通信协议建立网络连接并执行 SQL 查询。它作为异步 SQL 客户端,提供基于 Promise 的查询接口和对 async/await 模式的支持,以管理非阻塞数据库操作。 该库包含完整的 MySQL 协议实现,可作为构建自定义数据库服务器、代理或客户端的工具包。它还作为连接池管理器和注入预防工具,利用预处理语句和参数化查询来保护数据库交互。 该项目涵盖了广泛的数据库集成功能,包括原子事务协调、二进制日志流式传输以及用于批量导入的表格数据流式传输。它通过自动连接清理管理资源生命周期,并提供多种身份验证机制,如密码哈希验证和安全握手协商。 连接性进一步通过 SSL 加密、网络流量压缩以及通过自定义双工流传输路由流量的能力得到支持。
Implements the MySQL wire protocol to decode raw binary network streams into structured data packets.
Luvit 是一个集成了 libuv 的 Lua 事件驱动运行时,提供非阻塞 I/O 和异步系统操作。它作为一个异步网络框架和执行环境,利用即时 (JIT) 编译实现高性能脚本处理。 该平台通过 TCP 集群管理器在多个 CPU 核心之间分配网络流量,该管理器在子进程之间共享套接字句柄。它为构建 HTTP 服务器提供了专门的功能,通过流处理器解码传入的请求流并编码响应。 该运行时支持通用的事件驱动应用程序开发、异步网络编程,以及集成编译后的二进制模块以扩展底层语言功能。用户可以通过读取-求值-输出循环 (REPL) 与环境交互,执行独立脚本,或通过命令行评估代码片段。
Implements stream-based processing to decode incoming HTTP requests and encode outgoing responses.