Open-source alternatives to ToMe

This repository contains PyTorch code for ConViT. It builds on code from the Data-Efficient Vision Transformer and from timm.

DeiT is a PyTorch vision transformer framework designed for image classification. It implements a transformer-based architecture that processes images as sequences of flattened patches using self-attention layers and position-aware sequence modeling instead of convolutional filters. The project focuses on data-efficient training through a knowledge distillation framework. This system allows a student model to mimic the soft labels of a high-performance teacher model to improve accuracy and generalization, particularly when training on smaller datasets. The library covers the full development

Efficient-AI-Backbones is a lightweight neural network library and computer vision model zoo. It provides a collection of optimized deep learning backbones designed to minimize computational overhead and memory usage for artificial intelligence tasks. The project implements specialized architectures such as GhostNet and MLP to reduce processing requirements. It features a modular backbone design and the distribution of pretrained weights to accelerate the development and deployment of vision models. The library covers efficient neural network design and edge device AI optimization. Its capab

Official implementation of CrossViT. https://arxiv.org/abs/2103.14899

This is a pytorch implementation for the Visformer models. This project is based on the training code in DeiT and the tools in timm.

This is a collection of our NAS and Vision Transformer work.

This project provides the source code for the vision longformer paper.

Efficient vision foundation models for high-resolution generation and perception.