haosulab/ManiSkill
ManiSkill
ManiSkill is a GPU-accelerated robot simulation framework designed for training robotic manipulation skills, benchmarking learning algorithms, and generating synthetic datasets. It serves as a reinforcement learning environment where robot control policies can be developed and evaluated using parallelized physics and rendering on the GPU.
The platform is distinguished by its ability to perform sim-to-real transfer, allowing policies trained in virtual environments to be deployed onto physical robotic hardware. It features ray-traced parallel rendering for producing high-frame-rate RGBD and segmentation data, alongside a domain randomization pipeline that varies physical properties and visual textures to increase model robustness.
The framework covers rigid-body manipulation simulation across multiple robot embodiments and provides a suite of standardized tasks for benchmarking. It includes tools for synthetic dataset generation, heterogeneous parallel simulation of unique scenes, and the application of pre-tuned reinforcement and imitation learning baselines.
Command-line utilities are provided for managing simulation assets and expert demonstration data.
Features
- GPU-Accelerated Robot Simulators - Provides a GPU-accelerated physics simulation environment specifically designed for training and testing robotic manipulation policies.
- Articulated Body Simulators - Executes GPU-parallelized rigid-body simulations of articulated robot chains and grippers for manipulation tasks.
- Reinforcement Learning Environments - Serves as a simulation suite for developing and evaluating robot control policies through reinforcement learning.
- Parallel Simulation Environments - Simulates multiple unique environments in parallel to accelerate experience gathering for reinforcement learning.
- Imitation and Reinforcement Learning Toolkits - Provides a toolkit for training robotic behaviors using both imitation learning and reinforcement learning baselines.
- Sim-to-Real Robot Policy Trainings - Translates control policies trained in virtual environments into executable commands for physical robotic hardware.
- Policy Deployments - Transitions trained robotic control policies to physical hardware for real-world execution.
- Domain Randomizations - Varies physical properties and visual textures across simulations to improve the robustness of learned policies.
- Sim-to-Real Transfer - Bridges the gap between simulated training environments and physical hardware deployment for robotic policies.
- Sim-to-Real Transfer - Offers a framework for training robotic behaviors in simulation and deploying those policies to physical hardware.
- High-Quality Scene Rendering - Generates high-quality visual output of simulations using ray-traced shaders in both graphical and headless modes.
- Parallel Ray Tracing - Generates high-frame-rate RGBD and segmentation data using GPU-accelerated parallel ray tracing across multiple scenes.
- Synthetic Dataset Generators - Automatically generates synthetic RGBD and segmentation datasets across parallel simulations for training vision models.
- Robot Embodiment Registrations - Supports switching between different robot hardware by loading pre-configured models with standardized collision and controller data.
- Scalable Robot Policy Evaluations - Enables scalable and repeatable evaluation of trained robot policies within accelerated simulation environments.
- Robot Policy Validations in Simulation - Validates trained robot policies in physically accurate virtual environments prior to physical hardware deployment.
- Environment Definitions - Provides an object-oriented interface for creating simulation environments and managing GPU memory for robotic tasks.
- Synthetic Robotic Dataset Generators - Produces high-frame-rate RGBD and segmentation data across diverse parallelized virtual scenes for dataset generation.
- Robot Model Libraries - Loads pre-configured robot models including collision data and controllers for simulation environments.
- Embodiment Management - Enables switching between different robot models to evaluate skill generalization across hardware configurations.
- Robotics Benchmarks - Provides a collection of standardized manipulation environments to benchmark the performance of robot learning algorithms.
- Robotic Sensor Simulation - Simulates sensor limitations by toggling between ground-truth state data and restricted visual observations.
- Parallel Visual Data Capture - Captures high-frame-rate RGBD and segmentation data using GPU parallelization for dataset generation.
- Sensor Observation Toggles - Toggles between raw ground-truth state data and simulated sensor observations to control model information.
- Simulation and Benchmarking - GPU-parallelized simulation for generalizable embodied AI.
- Simulation Frameworks - GPU-parallelized simulation for generalizable embodied AI.
- Simulation Engines - Robot simulation and manipulation learning package.
