Projects sharing features with Cream

LeViT a Vision Transformer in ConvNet's Clothing for Faster Inference

This is a pytorch implementation for the Visformer models. This project is based on the training code in DeiT and the tools in timm.

This repository contains the PyTorch training and evaluation code for LocalViT.

This repo is an official implementation for "Dynamic Grained Encoder for Vision Transformers" (NeurIPS2021) on PyTorch framework.

(2022/05/10) Code of ResTV2 is released! ResTv2 simplifies the EMSA structure in ResTv1 (i.e., eliminating the multi-head interaction part) and employs an upsample operation to reconstruct the lost medium- and high-frequency information caused by the downsampling operation.

This repository contains PyTorch code for ConViT. It builds on code from the Data-Efficient Vision Transformer and from timm.

This project provides the source code for the vision longformer paper.

GhostNet provides a set of efficient AI model architectures and neural network design patterns designed to reduce computation and memory overhead. It serves as a computer vision backbone and a lightweight vision transformer, optimizing the balance between predictive accuracy and inference speed. The project focuses on reducing resource consumption for deployment on mobile devices and edge hardware. It achieves this through the use of lightweight vision transformer implementations and architectures that minimize the total number of parameters. The codebase covers a range of capabilities for i

We propose a conditional positional encoding (CPE) scheme for vision Transformers. Unlike previous fixed or learnable positional encodings, which are pre-defined and independent of input tokens, CPE is dynamically generated and conditioned on the local neighborhood of the input tokens. As a…

This is an official implementation of CvT: Introducing Convolutions to Vision Transformers.

CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped, CVPR 2022

Efficient-AI-Backbones is a lightweight neural network library and computer vision model zoo. It provides a collection of optimized deep learning backbones designed to minimize computational overhead and memory usage for artificial intelligence tasks. The project implements specialized architectures such as GhostNet and MLP to reduce processing requirements. It features a modular backbone design and the distribution of pretrained weights to accelerate the development and deployment of vision models. The library covers efficient neural network design and edge device AI optimization. Its capab

This repository contains the official implementation of the research paper, "FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization" ICCV 2023