# pytorch/pytorch

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/pytorch-pytorch).**

100,814 stars · 28,028 forks · Python · NOASSERTION

## Links

- GitHub: https://github.com/pytorch/pytorch
- Homepage: https://pytorch.org
- awesome-repositories: https://awesome-repositories.com/repository/pytorch-pytorch.md

## Topics

`autograd` `deep-learning` `gpu` `machine-learning` `neural-network` `numpy` `python` `tensor`

## Description

PyTorch is a machine learning framework centered on a GPU-ready tensor library that supports multi-dimensional array operations across both CPU and accelerator hardware. It provides a foundational infrastructure for mathematical computation and dynamic neural network construction, utilizing a tape-based automatic differentiation system that allows for flexible, non-static graph execution.

The framework is designed for deep integration with Python, enabling natural usage alongside standard scientific computing ecosystems. It distinguishes itself through a comprehensive distributed training suite that includes data-parallel, model-parallel, and sharding primitives, alongside a just-in-time compilation infrastructure. Developers can extend the library by registering custom operators written in Python, C++, or CUDA, ensuring these components compose directly with the core automatic differentiation and execution pipelines.

Beyond its core tensor and neural network modules, the project includes extensive tooling for data ingestion, performance profiling, and memory analysis. It provides specialized utilities for audio processing, including feature extraction and speech recognition, as well as a distributed remote procedure call framework for managing complex, multi-node computational workloads.

Installation instructions are available for various hardware backends and build-time configurations to support specific environment requirements.

## Tags

### Artificial Intelligence & ML

- [Hardware-Accelerated](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries/tensor-libraries/hardware-accelerated.md) — Accelerates multi-dimensional array operations by leveraging native GPU and specialized hardware support. ([source](https://cdn.jsdelivr.net/gh/pytorch/pytorch@main/README.md))
- [Distributed Training Primitives](https://awesome-repositories.com/f/artificial-intelligence-ml/distributed-training-frameworks/distributed-training-primitives.md) — Scales computations across multiple nodes and devices using primitives designed for data-parallel and model-parallel training. ([source](https://pytorch.org/tutorials/))
- [Neural Network Components](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/neural-network-components.md) — Organizes neural network architectures through modular base classes and container types for custom layer management. ([source](https://pytorch.org/docs/stable/nn.html))
- [Automatic Differentiation Systems](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/automatic-differentiation-systems.md) — Implements a dynamic, tape-based mechanism to compute gradients for flexible neural network training. ([source](https://cdn.jsdelivr.net/gh/pytorch/pytorch@main/README.md))
- [ATen](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries/tensor-libraries/aten.md) — Serves as a foundational tensor and mathematical operation layer for executing high-level computational kernels. ([source](https://github.com/pytorch/pytorch/blob/main/GLOSSARY.md))
- [Mathematical Operations](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries/tensor-libraries/mathematical-operations.md) — Executes a broad range of numerical routines, including linear algebra and pointwise operations, directly on tensors. ([source](https://pytorch.org/docs/stable/torch.html))
- [Fully Sharded Data Parallelism](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-training-and-tuning/distributed-and-scaling-strategies/distributed-learning/fully-sharded-data-parallelism.md) — Shards model parameters, gradients, and optimizer states across processes to enable memory-efficient distributed training. ([source](https://pytorch.org/tutorials/))
- [Functional Autograd](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/automatic-differentiation-systems/functional-autograd.md) — Exposes functional APIs to compute higher-order derivatives like Jacobians and Hessians on arbitrary mathematical functions. ([source](https://pytorch.org/docs/stable/autograd.html))
- [Convolution Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers.md) — Provides a robust suite of 1D, 2D, and 3D convolution layers tailored for image and signal processing. ([source](https://pytorch.org/docs/stable/nn.html))
- [Normalization Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/normalization-layers.md) — Stabilizes training convergence by automatically inferring input shapes for standard normalization layers. ([source](https://pytorch.org/docs/stable/nn.html))
- [Recurrent Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/recurrent-layers.md) — Processes sequential data through modular implementations of recurrent architectures like LSTMs and GRUs. ([source](https://pytorch.org/docs/stable/nn.html))
- [Native Extension Interfaces](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/hardware-acceleration/native-extension-interfaces.md) — Enables high-performance execution by integrating custom C++, CUDA, or SYCL code directly into the computational graph. ([source](https://pytorch.org/tutorials/advanced/cpp_extension.html))
- [Autograd Graph Inspection Tools](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/automatic-differentiation-systems/autograd-graph-inspection-tools.md) — Hooks into automatic differentiation graphs to extract node metadata and interpose custom logic during the backward pass. ([source](https://pytorch.org/docs/stable/autograd.html))
- [Forward-Mode Differentiation](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/automatic-differentiation-systems/forward-mode-differentiation.md) — Computes directional derivatives by propagating tangents through functions via a dedicated forward-mode differentiation API. ([source](https://pytorch.org/docs/stable/autograd.html))

### Programming Languages & Runtimes

- [Operation Kernels](https://awesome-repositories.com/f/programming-languages-runtimes/runtime-execution-environments/runtime-environments/runtimes/graph-symbolic-execution-engines/operation-kernels.md) — Bundles a comprehensive library of native and compound operation kernels for execution on CPUs and accelerators. ([source](https://github.com/pytorch/pytorch/blob/main/GLOSSARY.md))
- [Python Bindings](https://awesome-repositories.com/f/programming-languages-runtimes/language-interoperability/interoperability/python-bindings.md) — Integrates deeply with the Python ecosystem to allow seamless interoperability with standard scientific computing libraries. ([source](https://cdn.jsdelivr.net/gh/pytorch/pytorch@main/README.md))
- [Just-In-Time Compilers](https://awesome-repositories.com/f/programming-languages-runtimes/runtime-execution-environments/runtime-environments/execution-engines/just-in-time-compilers.md) — Optimizes function execution at runtime through source code scripting or tracing via just-in-time compilation. ([source](https://github.com/pytorch/pytorch/blob/main/GLOSSARY.md))

### Data & Databases

- [Batched Data Loading](https://awesome-repositories.com/f/data-databases/data-pipeline-orchestration/data-engineering-pipelines/batched-data-loading.md) — Automates the collation of data samples into tensors using customizable functions for complex data structures. ([source](https://pytorch.org/docs/stable/data.html))
- [Parallel Data Loaders](https://awesome-repositories.com/f/data-databases/data-pipeline-orchestration/data-engineering-pipelines/parallel-data-loaders.md) — Prevents training bottlenecks by fetching and preprocessing data through multi-process parallelism. ([source](https://pytorch.org/docs/stable/data.html))
- [Dataset Abstractions](https://awesome-repositories.com/f/data-databases/data-pipeline-orchestration/data-engineering-pipelines/dataset-abstractions.md) — Supports both index-based and streaming data access patterns through flexible dataset interface abstractions. ([source](https://pytorch.org/docs/stable/data.html))
- [Serialization Utilities](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-persistence-storage/data-storage/serialization-utilities.md) — Persists tensors and complex data structures to disk through native loading and saving mechanisms. ([source](https://pytorch.org/docs/stable/torch.html))
- [Data Samplers](https://awesome-repositories.com/f/data-databases/data-pipeline-orchestration/data-engineering-pipelines/data-samplers.md) — Manages data index sequences to enable custom batching and shuffling strategies during model training. ([source](https://pytorch.org/docs/stable/data.html))

### Networking & Communication

- [Pipeline Parallelism Strategies](https://awesome-repositories.com/f/networking-communication/distributed-systems-p2p/distributed-computing/model-parallelism-techniques/pipeline-parallelism-strategies.md) — Partitions large models across multiple devices to bypass single-device memory constraints during parallel execution. ([source](https://pytorch.org/tutorials/))
- [Remote Procedure Call Frameworks](https://awesome-repositories.com/f/networking-communication/distributed-systems-p2p/distributed-computing/remote-procedure-call-frameworks.md) — Facilitates distributed function execution, object referencing, and asynchronous task management across networked system components. ([source](https://pytorch.org/tutorials/))

### Part of an Awesome List

- [Artificial Intelligence](https://awesome-repositories.com/f/awesome-lists/ai/artificial-intelligence.md) — Deep learning framework with strong GPU acceleration.
- [Deep Learning](https://awesome-repositories.com/f/awesome-lists/ai/deep-learning.md) — Core library for tensors and dynamic neural networks.
- [Deep Learning Ecosystems](https://awesome-repositories.com/f/awesome-lists/ai/deep-learning-ecosystems.md) — Core deep learning library for research and production.
- [Deep Learning Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/deep-learning-frameworks.md) — Core library for tensor computation and deep learning.
- [General Machine Learning](https://awesome-repositories.com/f/awesome-lists/ai/general-machine-learning.md) — Framework for tensors and dynamic neural networks.
- [Large Language Models](https://awesome-repositories.com/f/awesome-lists/ai/large-language-models.md) — Core framework for dynamic neural networks and GPU acceleration.
- [Machine Learning](https://awesome-repositories.com/f/awesome-lists/ai/machine-learning.md) — Deep learning framework for research and production.
- [Machine Learning and AI](https://awesome-repositories.com/f/awesome-lists/ai/machine-learning-and-ai.md) — Tensor and neural network framework with GPU acceleration.
- [Machine Learning Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/machine-learning-frameworks.md) — Flexible deep learning framework for research and production.
- [Machine Learning Libraries](https://awesome-repositories.com/f/awesome-lists/ai/machine-learning-libraries.md) — Tensors and dynamic neural networks with GPU acceleration.
- [Data Science and Databases](https://awesome-repositories.com/f/awesome-lists/data/data-science-and-databases.md) — Deep learning framework with GPU acceleration.
- [Computation and Optimization](https://awesome-repositories.com/f/awesome-lists/devtools/computation-and-optimization.md) — Core library for developing and training deep learning models.
- [Python Projects](https://awesome-repositories.com/f/awesome-lists/devtools/python-projects.md) — Listed in the “Python Projects” section of the Awesome For Beginners awesome list.
- [Scientific Computing Libraries](https://awesome-repositories.com/f/awesome-lists/devtools/scientific-computing-libraries.md) — Dynamic neural network library with GPU acceleration.
- [PyTorch Utilities](https://awesome-repositories.com/f/awesome-lists/more/pytorch-utilities.md) — Listed in the “PyTorch Utilities” section of the The Incredible Pytorch awesome list.

### Development Tools & Productivity

- [Custom Operator Interfaces](https://awesome-repositories.com/f/development-tools-productivity/developer-utilities-libraries/extensibility-frameworks/custom-operator-interfaces.md) — Registers custom user-defined operators to ensure seamless integration with automatic differentiation and compilation pipelines. ([source](https://pytorch.org/tutorials/advanced/cpp_extension.html))
- [CPU Profilers](https://awesome-repositories.com/f/development-tools-productivity/debugging-profiling-testing/debugging-diagnostics/performance-resource-profilers/cpu-profilers.md) — Tracks operator execution time and memory usage to pinpoint performance bottlenecks across various hardware backends. ([source](https://pytorch.org/tutorials/))
- [Memory Profiling](https://awesome-repositories.com/f/development-tools-productivity/debugging-profiling-testing/debugging-diagnostics/performance-resource-profilers/memory-profiling.md) — Analyzes memory allocation patterns to detect fragmentation and optimize resource consumption during intensive computations. ([source](https://pytorch.org/tutorials/))

### Testing & Quality Assurance

- [Execution Profilers](https://awesome-repositories.com/f/testing-quality-assurance/performance-testing-analysis/performance-diagnostics/execution-profilers.md) — Visualizes hardware utilization, operator latency, and memory metrics to provide a comprehensive view of runtime performance. ([source](https://pytorch.org/tutorials/))