dora-rs/dora
Dora
Dora is a robotics dataflow framework and distributed orchestrator used to build and manage processing pipelines. It enables the deployment of robotics workloads across clusters with remote node execution and provides a real-time data pipeline for predictable performance.
The system is distinguished by its support for multi-language nodes written in Rust, Python, C, or C++ that interoperate within a single dataflow. It utilizes a zero-copy shared-memory transport and columnar formats to minimize latency for large payloads, and it includes bidirectional bridges to integrate with external ecosystems such as ROS2 and MAVLink.
The framework covers a broad range of capabilities including declarative node orchestration via directed graphs, real-time execution tuning through CPU pinning and memory locking, and system observability via live stream inspection and data record and replay. It also provides distributed execution tracing and node fault tolerance through restart policies and circuit breakers.
Features
- Distributed Robot Middleware - Provides a distributed, graph-based middleware for deploying and managing robotics workloads across machine clusters.
- Processor Graph Dataflow Models - Defines processing pipelines as a directed graph of typed nodes for routing and transforming data streams.
- Real-Time Data Processors - Provides a low-latency processing engine for real-time data streams with predictable performance for robotics.
- Shared Memory Data Exchange - Moves large data payloads between processes using shared memory and zero-copy formats to eliminate serialization overhead.
- Hybrid Transport Layers - Automatically switches between shared memory for local nodes and network sockets for remote nodes.
- Declarative Node Orchestration - Uses YAML configurations and versioned lockfiles to resolve dependencies and deploy nodes across multiple machines.
- Distributed Job Execution - Executes nodes across a cluster of machines using automatic network fallback and remote management.
- Robotics Dataflow Orchestration - Builds processing pipelines for robots using directed graphs that connect nodes written in different programming languages.
- ROS2 Ecosystem Integrations - Bridges dataflows bidirectionally with ROS2 topics, services, and actions to integrate with existing robotics software.
- Bidirectional Protocol Bridging - Translates internal dataflow messages to and from external robotics standards like ROS2 and MAVLink.
- ROS2 Protocol Bridges - Provides bidirectional bridges to integrate internal dataflows with external robotics ecosystems like ROS2.
- ROS2 Bridges - Provides a bidirectional communication layer connecting internal dataflows with ROS2 topics, services, and actions.
- Low Latency Messaging - Moves large payloads between processes using shared memory and zero-copy formats to minimize delivery delay.
- Real-Time Execution Tuning - Achieves predictable latency by pinning nodes to specific CPU cores and utilizing memory locking.
- Real-Time Resource Pinning - Assigns nodes to specific CPU cores and locks memory to minimize jitter and ensure predictable execution latency.
- Multi-Language Binding Layers - Enables nodes written in Rust, Python, C, or C++ to interoperate within a single dataflow.
- Multi-Language Node Runtimes - Supports writing processing nodes in Rust, Python, C, or C++ that interoperate within a single dataflow.
- Dataflow Frameworks - Implements a dataflow framework that models computation as a directed graph of multi-language nodes.
- Real-Time Jitter Reduction - Reduces system jitter and latency by pinning nodes to CPU cores and using memory locking.
- MAVLink Protocol Bridges - Implements a bidirectional communication bridge to integrate MAVLink-based telemetry and drone commands into dataflow graphs.
- Live Dataflow Graph Modifiers - Allows changing node connections in a running dataflow via a command line interface without restarting.
- Declarative Pipeline Definitions - Allows defining processing graphs in YAML or Python where multi-language nodes connect via typed inputs.
- Fault Tolerance Implementation - Implements restart policies and circuit breakers to ensure automatic recovery after node failures.
- Communication and Integration Patterns - Implements diverse interaction styles including pub/sub and request-reply services for inter-component messaging.
- Data Stream Inspection Tools - Provides tools to analyze topic frequency and print real-time message content for debugging.
- Robotics Data Recording - Provides capabilities for capturing robotics message streams to files and replaying them for offline analysis and debugging.
- Hot Reconfigurations - Allows updating node connections and graph structures in a running system via CLI without restarts.
- Distributed Tracing - Collects structured logs and spans to visualize data movement across distributed nodes.
- System Health Monitoring - Tracks distributed traces, resource metrics, and node health via a terminal-based interface.
- Industrial Automation - Dataflow framework for robotics and multi-AI applications.
- Robotics Middleware - High-performance middleware for AI-based robotic applications.
