54 Repos
Mechanisms and interfaces for transferring data between systems or memory spaces with minimal overhead.
Distinguishing note: Focuses on high-performance data transfer and memory sharing rather than general database management or storage.
Explore 54 awesome GitHub repositories matching data & databases · Data Exchange Protocols. Refine with filters or upvote what's useful.
ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring. The platform distinguishes itself through ad
Implements zero-copy shared memory buffers to eliminate serialization overhead during data transfer between database and analytical frameworks.
DeepSeek-TUI is an AI coding agent orchestrator and framework designed to automate complex programming tasks. It functions as a harness for coordinating AI models that can read source code, edit files, and execute shell commands through automated agent workflows. The system is distinguished by its multi-agent coordination capabilities, which allow for the spawning of parallel sub-agents to handle concurrent investigations or implementation slices. It employs autonomous goal-seeking loops to pursue objectives across multiple turns and utilizes a tool integration gateway to connect models to ex
Provides a standardized exchange protocol for connecting agent harnesses to remote servers for tool capability sharing.
IPFS is a peer-to-peer hypermedia protocol and content-addressed storage system that identifies data by cryptographic hashes rather than network locations. It enables the creation of a decentralized web by organizing files and directories as directed acyclic graphs of linked content identifiers. The project differentiates itself through the use of a distributed hash table for locating peers and a system of signed records to map human-readable names to changing content. It also provides HTTP gateways that translate standard web requests into peer-to-peer queries, allowing decentralized data to
Implements a dedicated protocol for requesting and sending data blocks between peers.
Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep
Facilitates data exchange between active workflows or external processes without interrupting the execution flow.
This is a comprehensive library of code examples and reference implementations for building cross-platform user interfaces with Flutter. The project provides a collection of demo applications and guides designed to illustrate the implementation of design patterns, animation techniques, and testing workflows. The repository features specific demonstrations for native integration, including examples of embedding modules into existing native applications, using platform channels, and bridging native code with the framework. It also serves as an animation reference, providing implementations for
Communicates between different frontend frameworks using custom events and interop methods to share state controllers.
This project is a pure JavaScript database driver for Node.js that implements the native MySQL binary protocol. It serves as a comprehensive connector for managing persistent network links to MySQL servers, enabling applications to execute queries, manage transactions, and handle complex data operations without requiring external middleware. The driver distinguishes itself through its integrated support for connection pooling and distributed database routing. It maintains managed sets of reusable network sockets to optimize resource usage under high request volumes, while simultaneously provi
Processes database operations using non-blocking callbacks to handle concurrent requests without stalling the main application thread.
ET is a C# game server framework and distributed actor model runtime designed for large-scale multiplayer environments. It provides a comprehensive toolkit for building distributed game backends, incorporating a multiplayer network transport layer and a specialized suite for game AI and pathfinding. The framework is distinguished by its use of a distributed actor model to scale processing across multiple threads and servers, utilizing isolated actors for state management and messaging. It features a unified codebase architecture that allows shared logic between the server and client, enabling
Enables isolated execution units to exchange data via a dedicated messaging mechanism to coordinate distributed tasks.
WebDriverIO is a Node.js test automation framework used for automating functional tests across web browsers and mobile applications. It acts as a WebDriver protocol client that manages remote browser sessions and executes commands against WebDriver and Appium servers to perform end-to-end testing. The framework is distinguished by its ability to control both native and hybrid mobile applications and its support for running automated suites across local machines, remote grids, and cloud device providers. It includes specialized capabilities for coordinating multi-browser interactions and estab
Facilitates the exchange of plain objects between the main process and worker threads using unique string keys.
DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met
Moves large volumes of data across multiple GPUs to maintain performance during expert-parallel model training.
This repository is a comprehensive collection of reference implementations and sample libraries for the Universal Windows Platform. It provides practical examples of how to use Windows Runtime APIs to build cross-device applications, including detailed guidance on XAML-based declarative user interfaces and DirectX-integrated rendering. The project distinguishes itself by providing a wide array of hardware integration suites, covering low-level communication with USB, Serial, I2C, SPI, and GPIO peripherals. It includes specialized implementations for mixed reality holographic rendering, advanc
Enables the execution of scripts within a web view to retrieve results and handle events from hosted scripts.
Actix is a Rust actor framework and concurrent programming toolkit designed for building applications that manage state through an asynchronous messaging system. It provides a model where independent actors serve as autonomous units of state and logic, communicating via strongly typed messages sent to unique addresses. The framework distinguishes itself by isolating state within these actors, allowing internal data to be mutated safely during message handling without the use of locks or mutexes. It employs an asynchronous mailbox system to buffer incoming requests and uses supervision strateg
Enables strongly typed data exchange between isolated actors using unique addresses.
wasm-bindgen is a glue code generator and interoperability tool that facilitates high-level communication and data exchange between compiled Rust modules and JavaScript environments. It functions as a memory bridge and type definition generator, allowing the exchange of complex data types and the calling of functions across execution boundaries. The project distinguishes itself by automating the conversion of data types and function signatures using attribute-based bindings. It generates the necessary JavaScript and WebAssembly wrappers to manage linear memory mapping and creates automatic Ty
Enables the exchange of complex objects, strings, and classes beyond simple numeric types.
jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti
Writes and reads video frames from a Data Distribution Service databus for external application integration.
Lnd is a full implementation of the Lightning Network protocol, functioning as a Bitcoin Layer 2 daemon that manages payment channels and settles transactions on the Bitcoin blockchain. It serves as an off-chain payment processor and a cryptographic wallet manager, enabling the execution of instant, scalable transactions through a network node. The project distinguishes itself through a focus on secure node networking and programmatic control. It provides gRPC and REST API servers for automating payment workflows and utilizes macaroon-based authorization to delegate granular permissions via c
Distributes work across actors using configurable strategies to balance computational load.
L-ink Card is a hardware implementation that combines an electronic ink display controller with an IC card emulator. The system utilizes the ISO 15693 protocol to enable wireless data exchange and energy harvesting from mobile devices, allowing the device to operate without a battery. The project features a multi-identifier storage system and a physical selector switch to simulate various access cards and security tokens. It also supports the transfer of bitmap data from a companion mobile application to update the bistable e-ink screen with custom images, text, or digital business card profi
Implements a protocol for transferring small packets of data between embedded hardware and smartphones.
Eel is a framework for creating desktop applications using a Python backend and a web-based frontend. It acts as a bidirectional bridge between Python and JavaScript, allowing developers to build graphical user interfaces with HTML and JavaScript that communicate with local system logic. The project facilitates two-way communication by enabling the exposure of Python functions to the browser and allowing the backend to trigger JavaScript functions. It uses a local server to render web interfaces as standalone desktop windows and provides tools to bundle the Python code and web assets into a s
Enables retrieving values from another process by pausing execution until the result is received.
ml5-library is a JavaScript machine learning library that functions as a browser-based inference engine. It provides a high-level wrapper for implementing neural networks and data models, allowing users to execute machine learning predictions directly on the client side. The library simplifies the integration of machine learning into web applications and creative coding projects by removing the requirement for deep mathematical expertise. It specifically enables web-based image classification through the use of pretrained deep learning models to identify and label objects within images. The
Implements asynchronous callback functions to handle results from long-running model predictions without blocking the browser.
MISP is an open-source threat intelligence sharing platform designed for collecting, storing, and distributing structured threat indicators and intelligence. At its core, it provides a distributed synchronization protocol for transferring events between instances, an attribute-based correlation engine that links matching indicators across events, and a REST API with an OpenAPI specification for programmatic access to threat data. The platform uses formal data formats for JSON, taxonomy, galaxy, and object templates to enable compatibility across tools and communities. The platform distinguish
Implements bidirectional TAXII exchange for receiving and pushing threat intelligence data.
Pony is an open-source, actor-model, capabilities-secure, high performance programming language
Provides exclusive asynchronous message passing between actors, with the runtime handling message queues and delivery.
go-plugin ist ein Framework zur Implementierung von Plugin-Architekturen, bei denen Erweiterungen als separate Prozesse ausgeführt werden und über ein Go-RPC-Plugin-System kommunizieren. Es fungiert als Inter-Process-Communication-Bibliothek und Lebenszyklus-Manager, der es Host-Anwendungen ermöglicht, externe binäre Plugins zu entdecken, zu starten und zu überwachen. Das Framework ermöglicht sprachübergreifende Plugin-Systeme, wodurch Plugins, die in verschiedenen Sprachen geschrieben wurden, über standardisierte RPC-Protokolle in einen Host integriert werden können. Es verfügt über eine bidirektionale RPC-Brücke, die es sowohl dem Host als auch dem Plugin ermöglicht, Schnittstellenimplementierungen für komplexen Datenaustausch und Callbacks auszutauschen. Das System bietet Prozessisolierung durch die Ausführung von Plugins als unabhängige Subprozesse, wodurch sichergestellt wird, dass Plugin-Abstürze die Host-Anwendung nicht beenden. Es deckt Sicherheit durch gegenseitige TLS-Authentifizierung und Integritätsprüfung ab sowie Lebenszyklus-Orchestrierung, einschließlich zustandsbehafteter Prozess-Wiederanbindung, um die Konnektivität während Host-Upgrades aufrechtzuerhalten. Zusätzliche Funktionen umfassen binäre Entdeckung, Protokoll-Versionierung und die Synchronisation von Plugin-Logs und Output-Streams zurück zum Host.
Manages the exchange of complex arguments and return values through specialized connections and raw data streams.