Swin-Transformer is a deep learning framework designed for training and deploying hierarchical vision transformer models. It serves as a research library and toolkit for computer vision tasks, providing the infrastructure to build models that replace standard convolution operations with sliding window self-attention mechanisms. By utilizing a multi-scale feature hierarchy, the framework enables the processing of visual data at varying resolutions and spatial scales. The project distinguishes itself through its implementation of shifted window partitioning, which facilitates global information
DeiT is a PyTorch vision transformer framework designed for image classification. It implements a transformer-based architecture that processes images as sequences of flattened patches using self-attention layers and position-aware sequence modeling instead of convolutional filters. The project focuses on data-efficient training through a knowledge distillation framework. This system allows a student model to mimic the soft labels of a high-performance teacher model to improve accuracy and generalization, particularly when training on smaller datasets. The library covers the full development
LeViT a Vision Transformer in ConvNet's Clothing for Faster Inference
Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, Seong Joon Oh | Paper
The main features of naver-ai/pit are: Representation Learning, Vision Backbones and Classification, Vision Transformers.
Open-source alternatives to naver-ai/pit include: microsoft/swin-transformer — Swin-Transformer is a deep learning framework designed for training and deploying hierarchical vision transformer… meituan-automl/cpvt — We propose a conditional positional encoding (CPE) scheme for vision Transformers. Unlike previous fixed or learnable… facebookresearch/deit — DeiT is a PyTorch vision transformer framework designed for image classification. It implements a transformer-based… facebookresearch/levit — LeViT a Vision Transformer in ConvNet's Clothing for Faster Inference. ibm/crossvit — Official implementation of CrossViT. https://arxiv.org/abs/2103.14899. microsoft/cream — This is a collection of our NAS and Vision Transformer work.
