28 Repos
Use of protocol buffers for efficient binary serialization and cross-language data communication.
Distinct from Serialization Protocols: Existing candidates focus on JSON mapping or raw buffers, not the general use of Protobuf for distributed node communication
Explore 28 awesome GitHub repositories matching data & databases · Protobuf Serialization. Refine with filters or upvote what's useful.
Hyperledger Fabric is a permissioned blockchain framework and enterprise blockchain platform designed for maintaining shared, immutable records of transactions across a network of authorized participants. It provides the infrastructure to build and manage private distributed ledgers that prioritize data confidentiality and scalability. The platform is distinguished by a modular architecture and a pluggable consensus engine. This design allows internal system implementations to be swapped with custom modules to adapt the ledger's behavior and consensus mechanisms to specific technical or indus
Uses protocol buffers to generate source code for consistent data communication and serialization between nodes.
grpcurl is a command-line interface and client tool for invoking gRPC methods and inspecting service definitions. It serves as a server debugger and reflection client, allowing users to call remote procedures and verify server responses from the terminal using JSON or protobuf formats. The tool distinguishes itself through the ability to discover available services and methods dynamically via a server reflection service or by loading local protobuf schema files. It can extract service descriptions from a server for local use and print specific schema symbol definitions. It covers a broad ran
Ships a tool to list services and describe protobuf message structures using server reflection or local files.
grpc-rust is a native gRPC framework for Rust designed for building high-performance remote procedure call clients and servers. It provides an asynchronous communication stack and a protocol buffers implementation for encoding, decoding, and generating type-safe code from service definitions. The project enables the implementation of unary and bi-directional data streaming over the HTTP/2 protocol. It includes specialized support for gRPC-Web integration, allowing browser-based clients to communicate with services through protocol translation. The infrastructure covers a broad range of distr
Uses protocol buffers for efficient binary serialization and deserialization of messages.
Pomelo is a distributed Node.js game server framework designed for building scalable multiplayer game backends and real-time web applications. It employs a multi-process architecture to distribute application logic horizontally across a cluster of machines, enabling high concurrency and balanced network traffic. The framework utilizes a WebSocket communication layer to manage bidirectional data exchange between diverse web clients and server clusters. It incorporates a remote procedure call system for triggering functions across remote server processes and uses Protocol Buffers for binary ser
Uses Protocol Buffers to compress message payloads into a compact binary format.
protobuf.js is a JavaScript and TypeScript library for encoding and decoding structured data using the Protocol Buffers binary format. It functions as a dynamic parser that can load and process schemas at runtime, a JSON transcoder for converting messages to human-readable formats, and a framework for building transport-agnostic remote procedure call clients. The project distinguishes itself by offering both dynamic message handling and a static code generator that produces TypeScript declarations and JavaScript modules to reduce runtime overhead. It also provides a reflection API for definin
Imports service descriptors via reflection, source files, or binary sets to define message structures.
RoadRunner is a high-performance application server and process manager designed to serve PHP applications using a persistent worker model. It eliminates bootload overhead and initialization time by keeping application processes alive between requests, acting as a protocol-agnostic proxy that routes traffic to a pool of supervised workers. The server is built with a plugin-based modular architecture, allowing it to be extended with custom Go plugins and compiled into tailored binaries. It distinguishes itself by providing a unified execution model for a wide array of communication protocols,
Allows the retrieval of shared protobuf definitions at runtime for handling dynamic messages.
pysc2 ist eine Python-Schnittstelle und ein Simulationsframework, das die StarCraft II-Game-Engine mit Machine-Learning-Agenten verbindet. Es fungiert als API-Wrapper, der Spielinterna als eine Reihe von Beobachtungen und Aktionen offenlegt und eine Reinforcement-Learning-Umgebung für Forschung und Training bereitstellt. Das Framework enthält Werkzeuge für die Analyse von Spiel-Replays, um Daten und Aktionssequenzen aus aufgezeichneten Matches für prädiktive Modellierungen zu extrahieren. Es bietet zudem eine Agenten-Simulationsumgebung, um die Leistung einzelner oder konkurrierender künstlicher Intelligenz-Agenten auszuführen und zu bewerten. Das System handhabt die Konfiguration von Spielkarten, Verhaltensanalysen und bietet eine manuelle Steuerungsschnittstelle zum Debuggen von Agentenverhalten. Es transformiert den Spielzustand in mehrdimensionale Tensoren und verwendet ein Remote-Procedure-Call-Framework, um die Kommunikation zwischen dem Client und der Game-Engine zu verwalten.
Uses Protocol Buffers to serialize observations and actions for consistent, strictly typed cross-language data transfer.
tensorboardX is a machine learning experiment tracking library used to log metrics and visual data from training processes. It enables the creation of event files that store scalars, images, audio, and graphs for monitoring model performance and behavior. The project provides framework-agnostic logging, allowing users to write visualization data from PyTorch, NumPy, or Chainer. It decouples data recording from specific deep learning engines by using a standardized set of writers to generate binary protobuf files. The library supports model visualization and training data analysis, including
Uses protocol buffers to ensure cross-platform compatibility between the logging library and the visualization viewer.
This project is an embedded operating system for creating decentralized peer-to-peer communication networks using long-range radio hardware. It provides an off-grid messaging protocol for transmitting text and GPS coordinates without relying on cellular or internet infrastructure. The firmware implements a secure communication framework using shared keys and high-grade encryption to protect data transmitted over the air. It includes a LoRaWAN hardware interface to manage radio parameters and an MQTT radio gateway to bridge local mesh traffic to external networks. The system covers a broad ra
Utilizes Protobuf for compact binary serialization to optimize message transmission over low-bandwidth radio links.
Twirp ist ein Protocol-Buffers-RPC-Framework, das für die Implementierung von typsicherer, verteilter Microservices-Kommunikation über HTTP entwickelt wurde. Es fungiert als Framework für die Microservices-Kommunikation und als API-Codegenerator, der stark typisierte Client-Stubs und Server-Interfaces aus Schemadefinitionen erstellt, um manuellen Netzwerk-Boilerplate-Code zu eliminieren. Das Framework stellt die Kompatibilität mit bestehender Web-Infrastruktur sicher, indem es Remote Procedure Calls über Standard-HTTP-Transport ausführt. Es verwendet schema-gesteuerte Verträge als Single Source of Truth, um sprachübergreifende Interoperabilität zu ermöglichen und Geschäftslogik vom Netzwerktransport zu entkoppeln. Das System deckt mehrere Kernbereiche ab, einschließlich interface-gesteuertem Routing, Daten-Serialisierung in mehreren Formaten (unterstützt sowohl JSON als auch Binärdaten) und Request-Lifecycle-Middleware für übergreifende Belange. Es bietet zudem ein standardisiertes Fehlermeldesystem, um Dienstfehler in maschinenlesbare Codes zu übersetzen, und enthält Tools zur Überprüfung der Client-Kompatibilität.
Supports both JSON and binary serialization to match client requirements and optimize data exchange.
Loads game configuration and content from spreadsheet formats using custom schema definitions.
TensorFlow Serving is a high-performance machine learning inference server designed to deploy TensorFlow models to production environments. It functions as a complete serving system that executes predictions on input data through a graph executor, providing network endpoints that eliminate the need for a separate runtime environment for client applications. The system is distinguished by its model version manager, which organizes and selects specific model versions within a directory hierarchy. It uses a filesystem watcher to detect new model versions and trigger automatic updates without int
Uses protocol buffers to define strict input and output signatures for efficient data exchange between clients and models.
yolotf ist ein Framework für Objekterkennung, das Tools zur Konvertierung von Darknet-Modellkonfigurationen und -Gewichten in TensorFlow-Graphen bereitstellt. Es enthält einen TensorFlow-Modell-Trainer für das Training neuer Erkennungsmodelle oder das Fine-Tuning bestehender Gewichte mit benutzerdefinierten Datensätzen. Das Projekt bietet einen Mobile-Model-Exporter, der Graphdefinitionen und Metadaten in Protobuf-Dateien für das Deployment auf mobilen Geräten serialisiert. Das Framework unterstützt die Objekterkennungs-Inferenz auf Bildern und Videos, um Objekte zu identifizieren und Bounding-Box-Koordinaten zu exportieren. Es verwaltet den Modellzustand durch Weight-Mapping-Übersetzung und checkpoint-basiertes Training, um die Wiederherstellung von Gewichten und Optimizer-Zuständen zu ermöglichen.
Serializes trained model graphs and metadata into protobuf files for mobile deployment.
grpcui ist ein browserbasiertes interaktives Web-Interface und API-Client für die Ausführung von Remote Procedure Calls (RPC) an gRPC-Server. Es fungiert als HTTP-zu-gRPC-Gateway, das JSON-Payloads in binäre Protobuf-Nachrichten übersetzt, sodass Benutzer Netzwerkanfragen auslösen und Antworten über eine visuelle Weboberfläche inspizieren können. Das Tool bietet Funktionen zum Browsen und Entdecken von Schemata, indem es Server-Reflection abfragt oder lokale Protobuf-Deskriptordateien lädt, um verfügbare Services und Methodensignaturen zu identifizieren. Es nutzt diese Definitionen, um automatisch dynamische Eingabeformulare für die Konstruktion von Anfragedaten zu generieren, inklusive Unterstützung für komplexe verschachtelte Strukturen. Das Projekt deckt API-Tests und Client-Debugging ab und ermöglicht die Überprüfung von Antwort-Bodies, Status-Codes und Metadaten-Headern. Es verwaltet zudem die sichere Konnektivität über eine Transportschicht, die Plaintext, TLS und gegenseitige Authentifizierung mit konfigurierbaren Client-Zertifikaten unterstützt.
Resolves communication endpoints by loading service definitions from server reflection, source files, or binary descriptors.
CloudEvents is an open specification for describing event data in a common format across cloud platforms and services. It defines a standard structure and set of metadata attributes for events, enabling interoperability across different systems so producers and consumers can exchange events without custom translation. The specification provides a protocol-agnostic serialization framework that maps CloudEvents attributes and payloads to multiple serialization formats including JSON, Avro, and Protobuf, and defines transport bindings for mapping events onto protocols like HTTP, AMQP, Kafka, MQTT
Supports string, binary, and protobuf message payloads within a CloudEvent envelope.
stc is a TypeScript type checker implemented entirely in Rust, designed to validate type correctness in TypeScript source code with native performance and memory safety. It operates as a no-emitting type validator, meaning it checks types only and does not produce JavaScript output, reducing overhead compared to a full compiler. The checker is built around a zero-false-positive design, reporting only errors that are provably certain from the type system, prioritizing correctness over speculative warnings. The tool distinguishes itself through parallel file processing, analyzing source files c
Loads and processes typing packages that use namespaces and globals.
CRI-O is an open-source container runtime that implements the Kubernetes Container Runtime Interface (CRI) to manage container images, pods, and containers on cluster nodes using OCI-compatible runtimes. It serves as a node-level container manager that handles image pulling, container lifecycle, and resource monitoring for Kubernetes clusters, running containers according to the Open Container Initiative specifications. The runtime distinguishes itself through live configuration reloading that applies changes to runtime definitions, registry mirrors, and TLS certificates without restarting th
Prevents panics when serializing ListContainer and ListPodSandbox responses using protocol buffers.
Perfetto is a platform for system-level performance tracing and analysis on Linux and Android. It combines a high-throughput trace recorder, a SQL-based query engine, and a browser-based visualizer into a single toolchain. The platform covers CPU scheduling and call-stack profiling, native and Java heap memory allocation tracking, GPU and graphics events, and system-wide counters such as CPU frequency and power consumption. The architecture decouples trace recording from offline analysis, using a compact protobuf format for event encoding and columnar storage for efficient SQL queries. The we
Encodes trace events in a compact, extensible binary format using Protocol Buffers.
ServiceStack ist ein hochperformantes .NET-Webframework für den Bau typsicherer APIs unter Verwendung stark typisierter Request- und Response-Objekte. Es fungiert als nachrichtenbasierte API-Engine, die Geschäftslogik von der Transportschicht entkoppelt, wodurch Services über mehrere Protokolle wie HTTP, gRPC und verschiedene Message-Queue-Provider bereitgestellt werden können. Das Framework zeichnet sich durch seinen typsicheren API-Generator aus, der native Client-SDKs und Data Transfer Objects (DTOs) aus Service-Metadaten über mehrere Sprachen hinweg produziert. Es enthält zudem ein verteiltes Service-Gateway für Microservices-Orchestration, ein Code-First-ORM zur direkten Übersetzung von C#-Objekten in Datenbankdatensätze sowie ein zentralisiertes Identitäts- und Zugriffsmanagementsystem für sicheren tokenbasierten Zugriff. Die breitere Funktionspalette deckt asynchrones Messaging und Echtzeit-Event-Streaming durch Pub-Sub und Server-Sent Events ab. Es bietet umfassende Unterstützung für Datenserialisierung in Formaten wie JSON, XML, ProtoBuf und MessagePack, neben integrierten Authentifizierungs-Flows wie JWT, API-Keys und Step-up-Authentifizierung. Zusätzliches Tooling umfasst automatisierte CRUD-API-Generierung, Hintergrund-Job-Ausführung und Vorlagen für administrative Dashboards.
Utilizes protocol buffers to transmit data in a compact binary wire format for high-performance serialization.
Valhalla is an open-source routing engine that calculates optimal paths and travel times using OpenStreetMap data. It is built around a tiled routing graph framework, allowing map data to be organized into small geographic tiles for efficient regional updates and offline routing capability. The project distinguishes itself through a multimodal routing server that combines automobile, pedestrian, bicycle, and public transit modes into single journeys. It includes a GPS trace matching engine to align noisy coordinates to the most probable road network paths and an isochrone and matrix generator
Generates protocol buffer files from route requests to provide consistent input for pinpoint testing.