10 Repos
Tools for executing benchmarks and evaluation scripts on physical and simulated robotic hardware.
Distinct from Performance Evaluation Tools: Distinct from general performance tools: focuses on robotic policy rollout and hardware evaluation.
Explore 10 awesome GitHub repositories matching artificial intelligence & ml · Robotic Policy Evaluators. Refine with filters or upvote what's useful.
This project is a comprehensive research platform designed for the end-to-end lifecycle of robotic learning. It provides a modular framework for training neural network policies—specifically through imitation and reinforcement learning—and deploying them onto physical robotic hardware. By offering a unified interface for hardware abstraction, the platform decouples high-level control logic from the specific sensors and actuators of diverse robotic systems. The framework distinguishes itself through a standardized approach to data and policy management. It utilizes a consistent schema for reco
Executes standardized benchmarks to test trained policies in simulated and physical environments.
Isaac Lab is an open-source framework for training robot policies in physically simulated environments, supporting both single-agent and multi-agent reinforcement learning. It is built on an Omniverse-PhysX simulation backend that models rigid bodies, articulated systems, deformable objects, and sensors, and provides a task-based environment configuration system where each training environment is defined as a modular class specifying observation spaces, action spaces, reward functions, and termination conditions. The framework distinguishes itself through an RL-library abstraction layer that
Loads a saved checkpoint and runs the policy across multiple parallel simulation environments to observe behavior.
Provides a framework for scalable, repeatable evaluation of trained robot policies within a simulated environment.
OpenVLA is a vision-language-action model and framework designed for general-purpose robotic manipulation. It provides a robotic policy training framework and a control inference engine that map visual and textual inputs to robotic control actions, enabling zero-shot instruction following on hardware. The project includes a robotics dataset pipeline for standardizing diverse trajectory data and managing dataset mixtures. It supports large-scale model training through distributed GPU compute and sharded data parallelism, alongside parameter-efficient adaptation for fine-tuning models to new ta
Includes tools for evaluating robotic policies through benchmarks in real-world environments and simulations.
Diffusion Policy ist ein Robot-Learning-Framework, das Diffusionsmodelle verwendet, um visuelle Beobachtungen auf präzise Aktionstrajektorien abzubilden. Es fungiert als Imitation-Learning-Toolkit und Visuomotor-Policy-Learner und bietet ein System zum Training neuronaler Netze, die menschliches Verhalten durch die Generierung robotischer Bewegungen basierend auf Bild- und Sensordaten replizieren. Das Framework verwendet einen konditionalen Denoising-Prozess, um Sequenzen robotischer Bewegungen zu samplen, was es ermöglicht, multimodale Aktionsverteilungen zu handhaben, bei denen mehrere gültige Trajektorien für einen einzelnen Zustand existieren können. Es nutzt score-basiertes Aktionsmodellieren und einen Roboter-Aktionsgenerator, um präzise Verhaltensweisen für komplexe physische Aufgaben zu erzeugen. Das Projekt deckt eine umfassende Robotik-Datenpipeline ab, einschließlich der Sammlung von menschengeführten Demonstrationen und der Speicherung groß angelegter Trajektorien in komprimierten, gechunkten Formaten. Seine Steuerungsfunktionen umfassen Receding-Horizon-Control via Sliding-Window-Ausführung und asynchrone Aktionsbereitstellung an Hardware-Controller, um hochfrequente Beobachtungsschleifen aufrechtzuerhalten. Das System enthält Tools für das Experiment-Management, wie Workspace-Encapsulation und Modell-Checkpointing, sowie Policy-Evaluierung zum Testen auf physischer oder simulierter Hardware.
Provides tools for executing benchmarks and logging performance metrics on physical and simulated robotic hardware.
evo ist ein Python-Framework zur Evaluierung von SLAM-Algorithmen, Roboterodometrie und Trajektoriendaten. Es dient als Analysebibliothek zur Messung von Drift und Präzision durch die Berechnung absoluter und relativer Posenfehler zwischen geschätzten Pfaden und Ground-Truth-Referenzen. Das Projekt bietet ein geometrisches Ausrichtungs-Framework zur Korrektur von Rotation, Translation und Skalierung zwischen räumlichen Trajektorien, um eine konsistente Fehlermessung zu gewährleisten. Es enthält spezialisierte Werkzeuge für die Odometrie-Driftanalyse und die Verarbeitung von Robotikdaten, einschließlich der Möglichkeit, Trajektorieninformationen aus ROS-Bagfiles zu extrahieren. Die Software deckt ein breites Spektrum an Funktionen ab, einschließlich 2D- und 3D-Trajektorienvisualisierung mit Unterstützung für geografische Kartenkacheln und ROS-Karten-Overlays. Zusätzliche Funktionen umfassen Zeitstempel-Synchronisation, räumliche Transformationen sowie die Möglichkeit, Trajektoriendaten in verschiedenen Industriestandardformaten zu filtern oder zu exportieren.
Calculates absolute and relative pose errors by comparing estimated trajectories against recorded ground-truth data.
Legged Gym is a high-performance simulation platform and toolkit engineered for training autonomous robotic agents in complex, physics-based environments. It provides a comprehensive framework for developing legged locomotion control policies, enabling robots to learn movement strategies for navigating uneven terrain and managing physical disturbances through reinforcement learning. The platform distinguishes itself by utilizing hardware-accelerated physics and headless execution to maximize computational throughput during training. It incorporates a domain randomization pipeline that injects
Facilitates policy execution by running saved reinforcement learning models within simulation environments to evaluate robotic performance and physical behavior.
AgiBot-World is a suite of software pipelines and tools designed for robotic policy training, dataset standardization, embodiment transfer, and performance benchmarking. It provides infrastructure for developing bimanual manipulation policies using foundation models and human-reference trajectory data. The project features a robot embodiment transfer suite that adapts pre-trained models to different robot bodies without requiring new multi-embodiment training data. It also includes a specialized evaluation framework for validating vision-language-action models through open-loop testing and ph
Implements a platform for measuring accuracy and generalization in robotic manipulation tasks using large scale datasets.
ManiSkill is a GPU-accelerated robot simulation framework designed for training robotic manipulation skills, benchmarking learning algorithms, and generating synthetic datasets. It serves as a reinforcement learning environment where robot control policies can be developed and evaluated using parallelized physics and rendering on the GPU. The platform is distinguished by its ability to perform sim-to-real transfer, allowing policies trained in virtual environments to be deployed onto physical robotic hardware. It features ray-traced parallel rendering for producing high-frame-rate RGBD and se
Enables scalable and repeatable evaluation of trained robot policies within accelerated simulation environments.
Omnizart is a deep learning framework designed for automatic music transcription and music information retrieval. It functions as a toolkit for analyzing polyphonic audio recordings to extract structured musical information, including notes, chord progressions, drum events, and rhythmic patterns. The system provides a modular pipeline that orchestrates the entire lifecycle of audio analysis, from initial feature extraction and data preparation to model inference. Users can apply pre-trained models to transcribe audio directly or utilize the included utilities to train and fine-tune neural net
Compares automated transcription outputs against verified ground truth data to quantify accuracy.