# real-stanford/diffusion_policy **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/real-stanford-diffusion-policy).** _How this analysis was created: the description and tags below were written by an AI model that read this project's README and public documentation pages; stars, license and language come straight from the GitHub API. The model does not read the source code._ 4,299 stars · 792 forks · Python · MIT ## Links - GitHub: https://github.com/real-stanford/diffusion_policy - Homepage: https://diffusion-policy.cs.columbia.edu/ - awesome-repositories: https://awesome-repositories.com/repository/real-stanford-diffusion-policy.md ## Topics `robotics` ## Description Diffusion Policy is a robot learning framework that uses diffusion models to map visual observations to precise action trajectories. It functions as an imitation learning toolkit and visuomotor policy learner, providing a system to train neural networks that replicate human behavior by generating robotic movements based on image and sensor data. The framework employs a conditional denoising process to sample sequences of robotic movements, allowing it to handle multimodal action distributions where multiple valid trajectories may exist for a single state. It utilizes score-based action modeling and a robot action generator to produce precise behaviors for complex physical tasks. The project covers a comprehensive robotic data pipeline, including the collection of human-led demonstrations and the storage of large-scale trajectories in compressed, chunked formats. Its control capabilities include receding horizon control via sliding-window execution and asynchronous action delivery to hardware controllers to maintain high-frequency observation loops. The system includes tools for experiment management, such as workspace encapsulation and model checkpointing, as well as policy evaluation for testing on physical or simulated hardware. ## Tags ### Artificial Intelligence & ML - [Diffusion Policy Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/diffusion-policy-learning.md) — Uses diffusion-based neural networks to model complex action distributions for stable and flexible robotic behavior. - [Robotic Behavior Execution](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-execution-runtimes/robotic-behavior-execution.md) — Translates AI inference results into timed sequences of physical robotic behaviors while maintaining high-frequency observation loops. - [Diffusion Conditioning Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/diffusion-conditioning-architectures.md) — Uses conditional denoising diffusion architectures to map visual observations to precise robotic action sequences. - [Diffusion Policy Training](https://awesome-repositories.com/f/artificial-intelligence-ml/distributed-training-scaling-utilities/diffusion-policy-training.md) — Trains neural networks mapping visual observations to robotic actions using a diffusion-based approach. ([source](https://diffusion-policy.cs.columbia.edu/data/experiments/image/pusht/diffusion_policy_cnn/config.yaml)) - [Score-Based Action Models](https://awesome-repositories.com/f/artificial-intelligence-ml/large-action-models/score-based-action-models.md) — Learns the gradient of the action distribution to represent multiple valid movement trajectories for a single state. - [Diffusion Model Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/pytorch-training-frameworks/diffusion-model-frameworks.md) — Provides a specialized framework for training and sampling from diffusion models to generate robotic action sequences. - [Imitation and Reinforcement Learning Toolkits](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-training/robot-policy-trainers/imitation-and-reinforcement-learning-toolkits.md) — Ships a comprehensive toolkit for collecting human demonstrations and training robot policies via imitation learning. - [Imitation Learning Trainings](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-training/robot-policy-trainers/scalable-robot-policy-trainings/imitation-learning-trainings.md) — Trains robot policies from demonstration data to map camera images to precise action trajectories. ([source](https://cdn.jsdelivr.net/gh/real-stanford/diffusion_policy@main/README.md)) - [Robot Action Model Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/robot-action-model-frameworks.md) — Implements a system for training and deploying neural networks that generate precise robotic control signals. - [Robotic Control Policies](https://awesome-repositories.com/f/artificial-intelligence-ml/robotic-control-policies.md) — Maps sensory visual inputs to action tokens to achieve precise visuomotor control for complex physical tasks. - [Robotic Policy Evaluators](https://awesome-repositories.com/f/artificial-intelligence-ml/performance-evaluation-tools/robotic-policy-evaluators.md) — Provides tools for executing benchmarks and logging performance metrics on physical and simulated robotic hardware. ([source](https://cdn.jsdelivr.net/gh/real-stanford/diffusion_policy@main/README.md)) - [Robust Visuomotor Policies](https://awesome-repositories.com/f/artificial-intelligence-ml/robust-visuomotor-policies.md) — Executes learned tasks while resisting external disruptions such as visual occlusions or physical interference. ([source](https://diffusion-policy.cs.columbia.edu/)) ### Part of an Awesome List - [Robotic Policy Learning](https://awesome-repositories.com/f/awesome-lists/ai/robotic-policy-learning.md) — Uses diffusion models to map visual observations to precise action trajectories for robotic control. ([source](https://diffusion-policy.cs.columbia.edu/)) ### Hardware & IoT - [Action Trajectory Sampling](https://awesome-repositories.com/f/hardware-iot/action-trajectory-sampling.md) — Performs intricate physical maneuvers by sampling action trajectories from a learned diffusion process. ([source](https://diffusion-policy.cs.columbia.edu/data/experiments/low_dim/pusht/diffusion_policy_cnn/train_0/checkpoints/epoch=0550-test_mean_score=0.969.ckpt](https://diffusion-policy.cs.columbia.edu/data/experiments/low_dim/pusht/diffusion_policy_cnn/train_0/checkpoints/epoch=0550-test_mean_score=0.969.ckpt)) - [Multimodal Action Prediction](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotics-and-control/multimodal-action-prediction.md) — Handles multimodal action distributions to manage multiple valid robotic trajectories for a single state. ([source](https://diffusion-policy.cs.columbia.edu/)) - [Receding Horizon Control](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotics-and-control/receding-horizon-control.md) — Optimizes action sequences over a sliding time window during inference to maintain physical stability. ([source](https://diffusion-policy.cs.columbia.edu/)) - [Demonstration Collection Systems](https://awesome-repositories.com/f/hardware-iot/embedded-robotics/robotics-autonomous-systems/robotics-drones/robotic-tooling/teleoperation-systems/demonstration-collection-systems.md) — Ships a system for capturing high-quality human-led manipulation data via teleoperation to create training datasets. ([source](https://cdn.jsdelivr.net/gh/real-stanford/diffusion_policy@main/README.md)) - [Shared Memory Buffers](https://awesome-repositories.com/f/hardware-iot/integration-performance/hardware-interfacing-integration/hardware-integration/device-sensors/real-time-sensor-streaming/shared-memory-buffers.md) — Captures high-frequency camera data into shared memory to minimize processing delays and serialization overhead. ([source](https://cdn.jsdelivr.net/gh/real-stanford/diffusion_policy@main/README.md)) ### Data & Databases - [Data Pipeline Configurations](https://awesome-repositories.com/f/data-databases/data-pipeline-configurations.md) — Provides configuration schemas to define how multimodal sensor and image data are batched and shuffled for training. ([source](https://diffusion-policy.cs.columbia.edu/data/experiments/image/pusht/diffusion_policy_cnn/config.yaml)) - [Trajectory Data Pipelines](https://awesome-repositories.com/f/data-databases/robotic-dataset-catalogs/trajectory-data-pipelines.md) — Provides a data pipeline for standardizing and managing high-frequency robotic trajectory datasets. - [Shared Memory Buffers](https://awesome-repositories.com/f/data-databases/shared-memory-buffers.md) — Employs shared-memory buffering to capture high-frequency camera data and reduce processing latency. - [Chunked File Storages](https://awesome-repositories.com/f/data-databases/storage-abstraction/file-storage-services/chunked-file-storages.md) — Implements a compressed, chunked storage system for handling large-scale robotic demonstration datasets efficiently on disk. - [Telemetry Data Pipelines](https://awesome-repositories.com/f/data-databases/telemetry-data-pipelines.md) — Provides a telemetry data pipeline for batching and streaming high-resolution image and sensor data for training. ### Scientific & Mathematical Computing - [Sliding Window Optimizations](https://awesome-repositories.com/f/scientific-mathematical-computing/sliding-window-optimizations.md) — Utilizes sliding-window numerical optimizations to maintain stability and consistency during physical robot movements. ### System Administration & Monitoring - [Trajectory Experience Storage](https://awesome-repositories.com/f/system-administration-monitoring/audit-logs/agent-trajectory-logs/trajectory-experience-storage.md) — Implements a storage system for recording and indexing large-scale robotic state-action trajectories. ([source](https://cdn.jsdelivr.net/gh/real-stanford/diffusion_policy@main/README.md))