This is a collection of our NAS and Vision Transformer work.
DeiT is a PyTorch vision transformer framework designed for image classification. It implements a transformer-based architecture that processes images as sequences of flattened patches using self-attention layers and position-aware sequence modeling instead of convolutional filters. The project focuses on data-efficient training through a knowledge distillation framework. This system allows a student model to mimic the soft labels of a high-performance teacher model to improve accuracy and generalization, particularly when training on smaller datasets. The library covers the full development
GhostNet provides a set of efficient AI model architectures and neural network design patterns designed to reduce computation and memory overhead. It serves as a computer vision backbone and a lightweight vision transformer, optimizing the balance between predictive accuracy and inference speed. The project focuses on reducing resource consumption for deployment on mobile devices and edge hardware. It achieves this through the use of lightweight vision transformer implementations and architectures that minimize the total number of parameters. The codebase covers a range of capabilities for i
This is a pytorch implementation for the Visformer models. This project is based on the training code in DeiT and the tools in timm.
The main features of danczs/visformer are: Efficient Vision Architectures, Efficient Vision Transformers, Vision Backbones and Classification.
Open-source alternatives to danczs/visformer include: ibm/crossvit — Official implementation of CrossViT. https://arxiv.org/abs/2103.14899. microsoft/cream — This is a collection of our NAS and Vision Transformer work. facebookresearch/deit — DeiT is a PyTorch vision transformer framework designed for image classification. It implements a transformer-based… huawei-noah/ghostnet — GhostNet provides a set of efficient AI model architectures and neural network design patterns designed to reduce… chenglin-yang/lvt. br-idl/paddlevit — :robot: PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+.
