Vector-Wangel/XLeRobot
XLeRobot
XLeRobot 是一个具身智能机器人平台和硬件生态系统，专为开发和部署自主机器人而设计。它集成了双臂移动机器人平台、基于大语言模型（LLM）的机器人控制器、基于物理的仿真环境以及遥操作界面，将自然语言指令转化为物理动作。
该项目强调使用 3D 打印和经济实惠的组件进行低成本机器人制造，以创建带有可互换手臂和夹爪的移动底座。它具有专门的遥操作工作流，允许通过 VR 界面和游戏控制器进行远程硬件控制，从而记录同步的传感器数据以进行模仿学习。
该系统涵盖了广泛的功能，包括视觉-语言-动作模型的训练与部署、闭环视觉跟踪，以及用于验证 AI 策略的基于物理引擎的仿真。它还提供了用于记录训练数据集和管理手动操作的实时外设控制工具。
Features
- Vision-Language-Action Models - Implements vision-language-action models that process multimodal inputs to predict robotic joint actions.
- Dual-Arm Mobile Robots - Provides a physical platform featuring a wheeled base and two robotic arms for AI manipulation tasks.
- Agentic LLM Frameworks - Integrates large language models to translate high-level natural language instructions into physical robot actions.
- Autonomous Task Execution - Executes reinforcement learning policies to perform complex tasks independently on robotic hardware.
- Teleoperation-Based Imitation Learning - Records synchronized camera streams and joint positions via teleoperation to create imitation learning datasets.
- Vision-Language-Action Training - Trains vision-language-action models and transformers on recorded datasets to automate complex manipulation tasks.
- Embodied AI Platforms - Provides an integrated platform for embodied intelligence research, combining AI models with physical hardware.
- Robotics Simulators - Provides a physics-based virtual space for verifying AI policies and robot movements before physical deployment.
- Policy Training Data Collection - Provides tools for recording physical movements and camera data to train neural network policies.
- Physics Simulation Environments - Provides a physics-based simulation environment to verify AI motion policies before hardware deployment.
- Teleoperation Systems - Ships a system for recording training data and controlling hardware remotely via VR or game controllers.
- Language and Robotics Integration - Translates natural language instructions into physical robotic actions using large language models.
- Object Detection and Tracking - Employs algorithms for identifying and maintaining the trajectory of objects to guide physical interactions.
- Edge AI Model Deployment - Deploys trained AI policies directly onto physical robot hardware for autonomous manipulation.
- Robot Policy Validations in Simulation - Tests trained AI policies within physically accurate virtual environments before physical deployment.
- VR Robotic Interfaces - Interfaces virtual reality systems with hardware for intuitive remote manual operation in simulated and physical environments.
- Modular Physical Architectures - Implements a modular physical architecture with a wheeled base and interchangeable arms and grippers.
- Mobile Base Integration - Sets up a stable two-wheeled platform with motors to enable physical movement in a workspace.
- Real-Time Actuator Mapping - Maps inputs from VR systems or game controllers to robot actuators for low-latency manual manipulation.
- Low-Cost Robot Fabrication - Utilizes 3D printing and affordable components to construct low-cost mobile dual-arm robots.
- Open-Source Robot Arm Hardware - Provides 3D-printable designs and assembly guides for creating affordable robotic arm manipulators.
- Mobile Robot Chassis - Assembles a hardware platform with a wheeled base and dual arms for testing embodied AI.
- Robot Hardware Customization - Provides guidelines and construction methods for the robot chassis, including wheel and shell integration.
- Robot Joint Control - Allows manual operation of joints and end-effectors using keyboards, game controllers, or VR hardware.
- Remote Hardware Controls - Operates robotic hardware in real-time using connected peripherals to minimize network latency.
- Visual Tracking Controllers - Uses real-time camera feedback and computer vision models to adjust motor movements based on object position.
