Physical-Intelligence/openpi

Features

• Action Inference - Generates direct physical control commands for robots by processing visual and textual inputs through VLA models.
• Distributed Training - Spreads training workloads across multiple GPUs or nodes to reduce fine-tuning time for large models.
• Distributed Training Accelerators - Implements distributed GPU training to accelerate the fine-tuning of large vision-language-action models.
• Robotic Model Trainers - Functions as a distributed trainer for fine-tuning large vision-language-action models across multiple compute nodes.
• VLA - Adapts base vision-language-action model checkpoints to specific robot platforms using task-specific datasets.
• Robotics Frameworks - Provides a comprehensive framework for executing and fine-tuning vision-language-action models for robotic systems.
• Remote Action Streaming - Provides a websocket server for streaming model-generated control actions to physical robot hardware in real-time.
• Language and Robotics Integration - Generates physical control movements for robots by integrating vision and language inputs through pre-trained models.
• Robot Action Streaming Servers - Implements a websocket-based server that streams model-generated control actions to physical robots in real-time.
• Remote Inference Streaming - Uses persistent websocket connections to stream real-time robot control signals from remote inference servers.
• Remote Inference Streaming - Separates neural network computation from robot hardware by streaming inference to a dedicated remote server.
• Remote Hardware Controls - Sends real-time control actions to physical hardware via websocket connections from a remote inference server.
• Modular Checkpoint Adaptation - Enables the adaptation of general base models to specific robot platforms using modular checkpoint updates.
• Model Format Converters - Provides a tool for transforming robotic model checkpoints between different framework formats.
• Model Interoperability Formats - Converts model checkpoints between different framework formats to ensure compatibility across robotic environments.
• Weight Conversion Utilities - Provides utilities for transforming model checkpoints between different tensor formats for cross-environment compatibility.
• Distributed Learning - Supports training large robotic models across multiple GPUs and compute nodes to accelerate development.
• Embodied Foundation Models - Open-world generalist model for robotic control.
• Foundation Models - Flow-based vision-language-action models for robotic manipulation.
• Robotics Foundation Models - Flow-based vision-language-action model for general robot control.

Star history