# openmind/om1

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/openmind-om1).**

2,636 stars · 957 forks · Python · mit

## Links

- GitHub: https://github.com/OpenMind/OM1
- Homepage: https://openmind.org
- awesome-repositories: https://awesome-repositories.com/repository/openmind-om1.md

## Topics

`llm` `multiagent` `robotics` `ros2` `zenoh`

## Description

OM1 is a multimodal AI agent runtime and orchestration framework designed to connect large language models to physical robot hardware and sensors. It provides an execution environment that processes audio, video, and sensor data to drive autonomous decisions and actions in real-world settings.

The system integrates a robotics SLAM and navigation stack with a hardware abstraction layer, allowing high-level AI commands to be translated into low-level motor and actuator instructions. It distinguishes itself by incorporating blockchain-based governance to enforce immutable operational rules and providing agents with economic agency through digital wallet integration and internet payment processing.

The framework covers a broad set of robotic capabilities, including autonomous path planning, obstacle avoidance, and environmental mapping. It also features multimodal perception tools for speech and vision processing, a physics-accurate robot simulator for testing agent logic, and a middleware pipeline for monitoring and transforming data flow.

The project is implemented in Python and includes utilities for code validation and stability testing of custom system extensions.

## Tags

### Artificial Intelligence & ML

- [Multimodal AI Orchestrators](https://awesome-repositories.com/f/artificial-intelligence-ml/multimodal-ai-orchestrators.md) — Orchestrates multiple AI models and data streams to execute complex autonomous workflows for robotic agents. ([source](https://cdn.jsdelivr.net/gh/openmind/om1@main/README.md))
- [Robotic Task Orchestration](https://awesome-repositories.com/f/artificial-intelligence-ml/robotic-task-orchestration.md) — Translates sensor data and user intent into physical movements and speech by coordinating AI models and hardware. ([source](https://docs.openmind.com/full-autonomy-guidelines/architecture_overview))
- [Agent Action Orchestrators](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-action-orchestrators.md) — Maps AI-driven decisions and model function calls to concrete operational tasks and hardware actions. ([source](https://docs.openmind.com/core-concepts/concepts))
- [Autonomous Economic Agency](https://awesome-repositories.com/f/artificial-intelligence-ml/autonomous-economic-agency.md) — Integrates digital wallets and blockchain governance to allow agents to execute payments and follow immutable rules.
- [Conversational Voice Processing](https://awesome-repositories.com/f/artificial-intelligence-ml/conversational-voice-processing.md) — Streams audio to text and synthesizes text back to audio for low-latency conversational communication. ([source](https://docs.openmind.com/full-autonomy-guidelines/architecture_overview))
- [Language Model Orchestrators](https://awesome-repositories.com/f/artificial-intelligence-ml/language-model-orchestrators.md) — Routes tasks across a hierarchy of local and cloud models to balance immediate reaction and long-term planning.
- [Agentic AI Orchestrators](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-deployment-and-serving/local-and-on-device-inference/edge-ai-model-deployment/agentic-ai-orchestrators.md) — Implements an orchestration layer that balances local and cloud AI models for autonomous multimodal robotic operations. ([source](https://docs.openmind.com/developing/2_architecture))
- [Multimodal Runtimes](https://awesome-repositories.com/f/artificial-intelligence-ml/multimodal-models/multimodal-runtimes.md) — Implements an execution environment optimized for AI agents processing multimodal sensor data.
- [Sensor Fusion](https://awesome-repositories.com/f/artificial-intelligence-ml/sensor-fusion.md) — Combines disparate short-form sensor descriptions into a unified situational paragraph for AI decision-making. ([source](https://docs.openmind.com/developing/2_architecture))
- [Sensor-to-Text Conversion](https://awesome-repositories.com/f/artificial-intelligence-ml/sensor-to-text-conversion.md) — Transforms raw visual, audio, and spatial inputs into natural language descriptions using recognition models. ([source](https://docs.openmind.com/developing/2_architecture))
- [Text-to-Speech Conversions](https://awesome-repositories.com/f/artificial-intelligence-ml/speech-and-text-conversion/text-to-speech-conversions.md) — Transforms written text into spoken audio using integrated speech synthesis models. ([source](https://docs.openmind.com/api-reference/introduction))
- [Speech-to-Text Conversions](https://awesome-repositories.com/f/artificial-intelligence-ml/speech-to-text-conversions.md) — Transcribes spoken audio into written text using automated speech recognition models. ([source](https://docs.openmind.com/api-reference/introduction))
- [Operational Mode Extensions](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-extensibility-frameworks/operational-mode-extensions.md) — Implements new operational modes to change how an agent interacts with its environment. ([source](https://docs.openmind.com/developer-cookbook/introduction))
- [Localization Recovery](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-recovery-triggers/localization-recovery.md) — Detects localization failures and automatically triggers a re-localization process without human intervention. ([source](https://docs.openmind.com/full-autonomy-guidelines/localization))
- [Voice-Activated Triggers](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/conversational-voice-interaction/voice-agents/voice-activity-detection/voice-activated-triggers.md) — Transitions the system into specific operational modes when predefined keywords are detected in speech. ([source](https://docs.openmind.com/modes-and-lifecycle/mode_selection))
- [Unified Provider Interfaces](https://awesome-repositories.com/f/artificial-intelligence-ml/cloud-ai-integrations/unified-provider-interfaces.md) — Unifies connectivity between various local and cloud-based language model providers through a single interface. ([source](https://docs.openmind.com/api-reference/introduction))
- [External Tool Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/external-service-integrations/external-knowledge-integrators/external-tool-integrations.md) — Integrates third-party services and APIs via standardized protocols to enable dynamic tool discovery and execution by agents. ([source](https://docs.openmind.com/mcp/mcp-integration))
- [MCP Protocol Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/external-tool-integrations/mcp-protocol-integrations.md) — Uses the Model Context Protocol to connect the agent runtime to standardized external tools and services. ([source](https://docs.openmind.com/developing))
- [Prompt Assembly Systems](https://awesome-repositories.com/f/artificial-intelligence-ml/prompt-assembly-systems.md) — Dynamically assembles system prompts and conversation history to provide necessary context for language model reasoning. ([source](https://docs.openmind.com/full-autonomy-guidelines/architecture_overview))
- [Social Robotics Interaction](https://awesome-repositories.com/f/artificial-intelligence-ml/social-robotics-interaction.md) — Facilitates natural dialogue and user communication through focused interaction modes and face detection. ([source](https://docs.openmind.com/modes-and-lifecycle/modes))
- [Tool Call Interception Middleware](https://awesome-repositories.com/f/artificial-intelligence-ml/tool-call-interception-middleware.md) — Implements a middleware pipeline to intercept and transform data flow between AI inputs and agent actions.

### Hardware & IoT

- [Navigation Frameworks](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/motion-planning-control/navigation-frameworks.md) — Provides a comprehensive navigation stack including path planning, SLAM, and automated charging workflows. ([source](https://cdn.jsdelivr.net/gh/openmind/om1@main/README.md))
- [Obstacle Avoidance Systems](https://awesome-repositories.com/f/hardware-iot/autonomous-flight-controllers/obstacle-avoidance-systems.md) — Calculates potential paths and selects valid trajectories by checking for objects within a safety radius using LiDAR. ([source](https://docs.openmind.com/robotics/motion_planning_lidara1m8))
- [Collision Avoidance Sensors](https://awesome-repositories.com/f/hardware-iot/connectivity-iot/proximity-services/collision-avoidance-sensors.md) — Triggers immediate retreat and rotational maneuvers when physical impact switches are activated. ([source](https://docs.openmind.com/robotics/motion_planning_turtlebot4))
- [Robotics And Autonomous Systems](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems.md) — Provides an integrated stack for SLAM-based environmental mapping and autonomous path planning for robots.
- [Environmental Mapping Techniques](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/localization-mapping/environmental-mapping-techniques.md) — Saves, lists, deletes, and marks specific coordinates as locations within stored environmental maps. ([source](https://docs.openmind.com/full-autonomy-guidelines/api_endpoints))
- [SLAM Algorithms](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/localization-mapping/slam-algorithms.md) — Implements simultaneous localization and mapping (SLAM) for autonomous navigation and obstacle avoidance.
- [Autonomy Middleware](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/autonomy-middleware.md) — Manages the full autonomy lifecycle, allowing agents to operate independently across the robotic hardware stack. ([source](https://docs.openmind.com/))
- [Robotics and Control](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotics-and-control.md) — Converts high-level movement intents into low-level velocity commands for motor controllers. ([source](https://docs.openmind.com/full-autonomy-guidelines/architecture_overview))
- [Language and Robotics Integration](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotics-and-control/language-and-robotics-integration.md) — Provides a runtime for connecting large language models to physical robot hardware and sensors.
- [Hardware Actuator Translation](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotics-and-control/language-and-robotics-integration/hardware-actuator-translation.md) — Interfaces AI runtimes with physical actuators and sensors to translate decisions into movement.
- [Robot Platform Integration](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotics-and-control/language-and-robotics-integration/robot-platform-integration.md) — Connects AI runtimes to embedded robotic platforms including humanoids and quadrupeds. ([source](https://docs.openmind.com/developer-cookbook/om1-integration-with-different-machines))
- [Robotics Middleware](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-middleware.md) — Provides the communication middleware necessary for data exchange between AI agents and robot hardware. ([source](https://docs.openmind.com/core-concepts/middleware))
- [Hardware Abstraction Layers](https://awesome-repositories.com/f/hardware-iot/hardware-abstraction-layers.md) — Provides a plugin-based hardware abstraction layer to interface with diverse robotic sensors and actuators agnostically.
- [Path Navigation](https://awesome-repositories.com/f/hardware-iot/path-navigation.md) — Moves between defined points within a mapped area while avoiding obstacles using pathfinding. ([source](https://docs.openmind.com/modes-and-lifecycle/modes))
- [Position Estimation](https://awesome-repositories.com/f/hardware-iot/position-estimation.md) — Combines visual place recognition and LiDAR scan matching to establish coordinates within a known map. ([source](https://docs.openmind.com/full-autonomy-guidelines/localization))
- [Real-Time SLAM](https://awesome-repositories.com/f/hardware-iot/real-time-slam.md) — Builds occupancy grid maps of physical spaces while tracking position in real-time using SLAM. ([source](https://docs.openmind.com/full-autonomy-guidelines/api_endpoints))
- [Robot Localization Tracking](https://awesome-repositories.com/f/hardware-iot/robot-localization-tracking.md) — Calculates real-time GPS location, altitude, velocity, and magnetic heading to enable outdoor navigation. ([source](https://docs.openmind.com/robotics/gps_compass))
- [Autonomous Maintenance Routines](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/autonomous-maintenance-routines.md) — Triggers autonomous docking for charging and manages the execution of predefined patrol routines. ([source](https://docs.openmind.com/full-autonomy-guidelines/api_endpoints))
- [Hardware Agnostic Interfaces](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotics-and-control/hardware-agnostic-interfaces.md) — Implements a hardware-agnostic interface to control various robot types through standard compatibility. ([source](https://docs.openmind.com/full-autonomy-guidelines/brainpack_introduction))
- [High-Precision Localization](https://awesome-repositories.com/f/hardware-iot/high-precision-localization.md) — Processes correction messages via compatible receivers to achieve centimeter-level localization accuracy. ([source](https://docs.openmind.com/robotics/gps_compass))
- [Robotics Visualization Tools](https://awesome-repositories.com/f/hardware-iot/integration-performance/hardware-interfacing-integration/hardware-integration/device-sensors/remote-sensor-queries/sensor-data-visualizers/robotics-visualization-tools.md) — Renders a real-time visual interface on a local display to communicate current activity and internal state. ([source](https://docs.openmind.com/full-autonomy-guidelines/architecture_overview))
- [Operational Mode Overrides](https://awesome-repositories.com/f/hardware-iot/operational-mode-overrides.md) — Allows changing the active operational mode via a remote administrative portal to override autonomous behavior. ([source](https://docs.openmind.com/modes-and-lifecycle/mode_selection))
- [Blind Spot Filtering](https://awesome-repositories.com/f/hardware-iot/sensor-noise-filtering/blind-spot-filtering.md) — Ignores fixed obstructions in the sensor field of view to prevent erroneous collision alerts. ([source](https://docs.openmind.com/robotics/motion_planning_lidara1m8))
- [Movement Drift Correction](https://awesome-repositories.com/f/hardware-iot/sensor-noise-filtering/movement-drift-correction.md) — Maintains a stable position estimate during navigation by filtering sensor noise and correcting for drift. ([source](https://docs.openmind.com/full-autonomy-guidelines/localization))
- [Target Following](https://awesome-repositories.com/f/hardware-iot/target-following.md) — Tracks a specific human target and maintains a set distance while following their movement. ([source](https://docs.openmind.com/full-autonomy-guidelines/premium_features))

### Part of an Awesome List

- [Multimodal Fusion](https://awesome-repositories.com/f/awesome-lists/ai/multimodal-fusion.md) — Fuses disparate sensor data and multimodal streams into a unified situational context for model reasoning.
- [Obstacle-Aware Exploration](https://awesome-repositories.com/f/awesome-lists/ai/navigation-and-exploration/obstacle-aware-exploration.md) — Uses laser scan data to detect objects and generate motion commands to navigate environments without collisions. ([source](https://docs.openmind.com/robotics/motion_planning_unitree_go2))
- [Natural Language Actuator Mapping](https://awesome-repositories.com/f/awesome-lists/devtools/robotics-and-hardware/natural-language-actuator-mapping.md) — Converts high-level natural language commands into specific servo and actuator instructions. ([source](https://docs.openmind.com/developing/2_architecture))
- [Robotics Simulators](https://awesome-repositories.com/f/awesome-lists/ai/robotics-simulators.md) — Ships a physics-accurate virtual environment for testing AI agent behaviors before physical deployment.
- [Robotic Sensor Simulation](https://awesome-repositories.com/f/awesome-lists/devtools/hardware-simulation/sensor-data-simulation/robotic-sensor-simulation.md) — Generates synthetic depth data from simulated hardware to test perception systems using LiDAR simulation. ([source](https://docs.openmind.com/simulators/isaac-sim))
- [Robotics Simulators](https://awesome-repositories.com/f/awesome-lists/devtools/robotics-simulators.md) — Provides a virtual environment with physics simulation to test AI agent behaviors and hardware interactions before physical deployment. ([source](https://docs.openmind.com/simulators/isaac-sim))

### Data & Databases

- [Data Integration](https://awesome-repositories.com/f/data-databases/data-integration-synchronization/data-integration.md) — Ingests information from web APIs, social media, cameras, and sensors to inform agent behavior. ([source](https://docs.openmind.com/developing/0_introduction))
- [Agent Memory Management](https://awesome-repositories.com/f/data-databases/session-management/agent-memory-management.md) — Stores short-term session data or persists long-term memory to maintain context across multiple interactions. ([source](https://docs.openmind.com/api-reference/api_pricing))
- [Multimodal Data Buses](https://awesome-repositories.com/f/data-databases/multimodal-data-storage/multimodal-data-buses.md) — Collects and structures natural language descriptions from various sensors into a centralized data bus. ([source](https://docs.openmind.com/developing/2_architecture))

### Networking & Communication

- [Multimodal Data Streams](https://awesome-repositories.com/f/networking-communication/bidirectional-streaming-protocols/multimodal-data-streams.md) — Publishes real-time audio and video streams to a central server using standard network protocols. ([source](https://docs.openmind.com/robotics/media_server))
- [Cryptocurrency Payment Processing](https://awesome-repositories.com/f/networking-communication/global-peer-to-peer-payments/cryptocurrency-payment-processing.md) — Enables autonomous agents with economic agency by processing outgoing cryptocurrency transactions for goods and services. ([source](https://docs.openmind.com/robotics/coinbase-x402))
- [Pub-Sub Systems](https://awesome-repositories.com/f/networking-communication/pub-sub-systems.md) — Synchronizes real-time data streams and stored system state using a unified publish-subscribe messaging interface.
- [Unified Data Stream Synchronization](https://awesome-repositories.com/f/networking-communication/unified-data-stream-synchronization.md) — Synchronizes data in motion, data at rest, and computations using a unified pub/sub/query protocol. ([source](https://docs.openmind.com/robotics/zenoh))
- [Voice and Vision Processing](https://awesome-repositories.com/f/networking-communication/voice-and-vision-processing.md) — Processes real-time audio and video streams to enable natural language interaction and environmental awareness.

### Operating Systems & Systems Programming

- [Hardware Abstraction Layers](https://awesome-repositories.com/f/operating-systems-systems-programming/hardware-interfacing-drivers/hardware-abstraction-layers.md) — Provides a plugin-based hardware abstraction layer to normalize interfaces across diverse robotic components.
- [Robot Lifecycle Orchestration](https://awesome-repositories.com/f/operating-systems-systems-programming/kernel-core-internals/process-and-memory-management/memory-management/process-lifecycle-orchestrators/process-lifecycle-managers/robot-lifecycle-orchestration.md) — Controls the transition between functional states by executing initialization, active, and cleanup logic. ([source](https://docs.openmind.com/modes-and-lifecycle/lifecycle))
- [Operational Mode Automations](https://awesome-repositories.com/f/operating-systems-systems-programming/operational-mode-automations.md) — Automatically switches operational modes based on context, elapsed time, or task completion. ([source](https://docs.openmind.com/modes-and-lifecycle/mode_selection))

### Software Engineering & Architecture

- [Smart Contract Governance](https://awesome-repositories.com/f/software-engineering-architecture/blockchain-governance-models/smart-contract-governance.md) — Enforces immutable operational rules for AI agents by retrieving constraints directly from blockchain smart contracts.
- [Blockchain Governance Models](https://awesome-repositories.com/f/software-engineering-architecture/blockchain-governance-models.md) — Enforces immutable operational rules and ensures transparency through blockchain-based smart contract governance.
- [Tool Isolation](https://awesome-repositories.com/f/software-engineering-architecture/execution-control/execution-modes/tool-operation-modes/tool-isolation.md) — Assigns specific external tool sets to different operational modes to ensure agents only access relevant capabilities. ([source](https://docs.openmind.com/mcp/mcp-integration))
- [Request Interception Middleware](https://awesome-repositories.com/f/software-engineering-architecture/request-interception-middleware.md) — Implements a middleware stack to modify or monitor data flow between inputs and actions. ([source](https://docs.openmind.com/core-concepts/concepts))
- [Agent Input Plugins](https://awesome-repositories.com/f/software-engineering-architecture/software-architecture/architectural-patterns/plugin-module-systems/modular-plugin-architectures/plugin-based-architectures/plugin-based-architectures/input-data-processing-plugins/agent-input-plugins.md) — Provides a plugin system for agents to receive and process data from diverse external information sources. ([source](https://docs.openmind.com/developer-cookbook/introduction))
- [Behavioral State Transition Logic](https://awesome-repositories.com/f/software-engineering-architecture/state-machine-logic/lightweight-state-machines/error-state-machines/behavioral-state-transition-logic.md) — Defines priority-based logic for switching between operational states based on user input or context. ([source](https://docs.openmind.com/modes-and-lifecycle/transition_rules))

### Development Tools & Productivity

- [Behavior Configuration](https://awesome-repositories.com/f/development-tools-productivity/behavior-configuration.md) — Defines capabilities, input-action combinations, and system prompts using configuration files to determine interaction patterns. ([source](https://cdn.jsdelivr.net/gh/openmind/om1@main/README.md))
- [Runtime Behavior Configuration](https://awesome-repositories.com/f/development-tools-productivity/configuration-parsers/runtime-behavior-configuration.md) — Customizes operational parameters and settings to adapt the runtime to different deployment environments. ([source](https://docs.openmind.com/core-concepts/concepts))

### Game Development

- [Autonomous AI Agent Simulations](https://awesome-repositories.com/f/game-development/simulation-engines/interactive-simulations/autonomous-ai-agent-simulations.md) — Runs a virtual environment to test agent logic and movements before deploying to physical hardware. ([source](https://docs.openmind.com/simulators/gazebo))
- [Physical Interaction Simulations](https://awesome-repositories.com/f/game-development/simulation-engines/interactive-simulations/autonomous-ai-agent-simulations/physical-interaction-simulations.md) — Tests agent logic and physical interactions within virtual environments before deploying code to real hardware.

### Graphics & Multimedia

- [Depth Data Capture](https://awesome-repositories.com/f/graphics-multimedia/media-capture-utilities/depth-data-capture.md) — Integrates depth-sensing hardware to provide spatial data for AI agent environmental awareness. ([source](https://docs.openmind.com/full-autonomy-guidelines/nvidia_thor))

### Security & Cryptography

- [Wallet Transaction Monitoring](https://awesome-repositories.com/f/security-cryptography/private-wallet-management/cryptocurrency-wallets/wallet-transaction-monitoring.md) — Tracks cryptocurrency wallet balances and detects new transactions to trigger autonomous economic actions. ([source](https://docs.openmind.com/robotics/coinbase_hackathon))
- [Behavioral Mode Switching](https://awesome-repositories.com/f/security-cryptography/role-based-access-control/workspace-role-assignments/ai-agent-role-assignments/behavioral-mode-switching.md) — Transitions the system between distinct operational states to adjust how it perceives the environment and prioritizes tasks. ([source](https://docs.openmind.com/modes-and-lifecycle/modes))
- [Operational Mode Switching](https://awesome-repositories.com/f/security-cryptography/role-based-access-control/workspace-role-assignments/ai-agent-role-assignments/behavioral-role-switching/operational-mode-switching.md) — Transitions the runtime between distinct operational modes to adjust how the agent perceives and prioritizes tasks.

### Testing & Quality Assurance

- [Agent Behavior Simulation](https://awesome-repositories.com/f/testing-quality-assurance/api-endpoint-testing/simulated-request-testing/llm-behavior-simulators/agent-behavior-simulation.md) — Executes agent behaviors within dedicated simulator instances for testing and validation. ([source](https://docs.openmind.com/api-reference/api_pricing))

### Web Development

- [Physical Area Patrolling](https://awesome-repositories.com/f/web-development/web-automation-scraping/browser-interaction-primitives/browser-navigation/ai-driven-navigation/ai-driven-extraction/infrastructure-patrols/physical-area-patrolling.md) — Performs scheduled monitoring routines to detect unusual activity or movement and report status logs. ([source](https://docs.openmind.com/modes-and-lifecycle/modes))