facebookresearch/maeArchived
Mae
This is a PyTorch library and framework for self-supervised vision learning. It provides an implementation of masked autoencoders and vision transformers designed to learn image representations by reconstructing masked image patches from unlabeled data.
The project features a distributed training pipeline that scales workloads across multiple GPU nodes. This infrastructure includes multi-node orchestration and gradient accumulation to manage large batch sizes and coordinate resource requests across clusters.
The toolkit covers a complete workflow from self-supervised masked pre-training to downstream task fine-tuning. It includes capabilities for representation quality evaluation through linear probing analysis and performance metric calculation on validation datasets.
Features
- Masked Image Modeling - Implements masked image modeling to learn visual representations by predicting missing image patches.
- Masked Autoencoding - Implements a masked autoencoder architecture for self-supervised pre-training of vision transformers via image patch reconstruction.
- Distributed Training - Scales deep learning workloads across multiple compute nodes to handle large-scale vision datasets.
- Data-Parallel Training - Provides distributed data-parallel training to synchronize gradients across multiple GPU nodes.
- Distributed Training - Ships a distributed training pipeline for scaling models across multiple GPU nodes.
- Asymmetric Encoder-Decoders - Implements an asymmetric encoder-decoder architecture to efficiently reconstruct masked image patches.
- Image Patch Embedders - Converts images into flattened vector sequences using patch-based embedding modules.
- Vision Transformer Pre-training - Provides a full pipeline for pre-training vision transformers using self-supervised masked modeling.
- Masked Autoencoder Implementations - Provides a complete PyTorch implementation of masked autoencoders for vision transformers.
- Self-Supervised Vision Representation Trainers - Trains vision transformers to learn image representations from unlabeled data using self-supervised techniques.
- Vision Transformers - Implements a transformer architecture designed for processing image data as sequences.
- Training Node Orchestration - Orchestrates training workloads across multiple compute nodes and GPUs using a cluster scheduler.
- Linear Classifiers - Evaluates learned feature quality by training linear classifiers on frozen encoder embeddings.
- Vision Model Fine-Tuning - Enables fine-tuning of pre-trained vision transformers for specific downstream tasks.
- Representation Evaluation Tools - Assesses representation quality through linear probing and validation accuracy metrics.
- Gradient Accumulation Strategies - Uses gradient accumulation strategies to simulate larger batch sizes on limited hardware.
- Self-Supervised Learning - Masked autoencoder pretraining for visual representation learning.
- Self-Supervised Pretraining - Scalable masked autoencoder framework for vision learners.
- Vision Backbones and Classification - Scalable masked autoencoders for vision learning.
