Awesome GitHub RepositoriesVision Transformers

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This library provides a comprehensive collection of modular building blocks and research-backed architectures for implementing vision transformers within the PyTorch framework. It serves as a centralized repository for constructing, training, and analyzing attention-based models, offering a wide array of specialized variants designed for image classification and visual representation learning. The project distinguishes itself through a focus on architectural efficiency and flexibility, supporting diverse input formats including non-square images and volumetric data like video. It incorporates

This project is a comprehensive deep learning framework and educational platform designed for constructing, training, and evaluating neural network architectures. It provides a modular environment for building models through tensor operations and automatic differentiation, supporting a wide range of tasks from image classification and object detection to sequential data processing. Beyond its core technical capabilities, the project distinguishes itself by integrating professional career development resources directly into its learning ecosystem. It offers structured guidance, resume reviews,

Qwen3-VL is a multimodal vision-language model designed to process and reason across images, videos, and text. It functions as a computer vision framework capable of identifying objects, extracting structured data from documents, and interpreting spatial elements within visual media. The system operates as an automated user interface interaction agent, interpreting screen data to navigate software and mobile applications. By utilizing a unified transformer architecture, it performs complex visual reasoning to execute user-defined tasks without manual input. Beyond interface navigation, the m

LaTeX-OCR is a specialized optical character recognition system designed to identify and transcribe complex mathematical symbols and their spatial relationships from images. It functions as a machine learning engine that converts visual representations of equations into structured LaTeX code for use in technical documentation and academic typesetting. The project utilizes a hierarchical vision-based encoding and autoregressive sequence decoding architecture to process input images and generate mathematical notation token by token. Beyond its core recognition capabilities, the system provides

Swin-Transformer is a deep learning framework designed for training and deploying hierarchical vision transformer models. It serves as a research library and toolkit for computer vision tasks, providing the infrastructure to build models that replace standard convolution operations with sliding window self-attention mechanisms. By utilizing a multi-scale feature hierarchy, the framework enables the processing of visual data at varying resolutions and spatial scales. The project distinguishes itself through its implementation of shifted window partitioning, which facilitates global information

PaddleDetection is an object detection framework designed for the end-to-end development, training, and deployment of computer vision models. It provides a comprehensive library of modular neural network architectures and pipelines that support object detection, instance segmentation, and multi-object tracking tasks. The project distinguishes itself through a configuration-driven approach that decouples model components like backbones and heads, allowing for the flexible assembly of custom vision workflows. It incorporates advanced techniques such as anchor-free detection logic, joint detecti

This project is a research library and toolkit for deep learning computer vision, focused on implementing transformer and mixer-based architectures for image classification. It processes visual data by converting images into sequences of patches, allowing standard attention mechanisms to capture global dependencies without relying on traditional convolutional operations. The framework distinguishes itself through its support for multimodal embedding analysis, which maps images and text into a shared latent vector space. This capability enables zero-shot classification and cross-modal retrieva

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This is a PyTorch library and framework for self-supervised vision learning. It provides an implementation of masked autoencoders and vision transformers designed to learn image representations by reconstructing masked image patches from unlabeled data. The project features a distributed training pipeline that scales workloads across multiple GPU nodes. This infrastructure includes multi-node orchestration and gradient accumulation to manage large batch sizes and coordinate resource requests across clusters. The toolkit covers a complete workflow from self-supervised masked pre-training to d

This project is a PyTorch vision transformer framework designed for self-supervised learning. It implements a model that trains visual representations using a momentum teacher and self-distillation without the need for labeled data. The library functions as an image feature extractor and visual attention visualizer, allowing for the generation of high-dimensional vectors and the rendering of self-attention maps as heatmaps or videos to analyze model focus. It provides comprehensive tools for downstream vision evaluation, including linear probe classification, k-nearest neighbor categorizatio

Donut is an OCR-free document transformer and end-to-end document parser. It functions as a neural network that converts unstructured document images directly into structured data or text without the use of an external optical character recognition engine. The project includes a synthetic document generator to create artificial images and ground-truth labels for training. It employs a transformer model to perform visual question answering and document image classification based on visual layout and text. The system covers several document understanding capabilities, including structured info

x-transformers est une bibliothèque PyTorch et un toolkit de recherche pour construire des architectures transformer. Il fournit un framework modulaire pour implémenter la recherche transformer expérimentale, incluant une suite de mécanismes d'attention avancés, des outils de modélisation de séquences longues et un framework pour les vision transformers. Le projet se distingue par son accent sur les composants haute performance et économes en mémoire, tels que Flash Attention avec des noyaux tuilés et l'attention multi-requêtes. Il implémente également des méthodes spécialisées pour étendre les fenêtres de contexte, incluant la récurrence de séquence et les plongements positionnels rotatifs. La bibliothèque couvre un large éventail de capacités architecturales, incluant divers schémas de normalisation pour stabiliser l'entraînement, des réseaux feedforward à portes et des topologies de couches personnalisées comme les réseaux Macaron. Elle prend en charge les constructions d'encodeur et de décodeur, fournissant des outils pour la génération de séquence autorégressive et les tâches vision-langage comme la légende d'image.

MobileSAM est un segmenteur d'image léger et un modèle de vision promptable conçu pour l'isolation rapide d'objets sur du matériel aux ressources limitées. Il fonctionne comme un outil de masquage d'image automatique capable de détecter et d'isoler des objets distincts sur une image entière sans saisie manuelle. Le système permet le masquage d'objets basé sur des prompts en utilisant des points de coordonnées ou des boîtes englobantes pour générer des masques précis. Il prend également en charge la segmentation d'image de tous les objets via un échantillonnage de prompts conscient des objets pour identifier chaque élément distinct dans une scène. Pour faciliter le déploiement mobile et en périphérie (edge), le modèle est compatible avec l'exportation ONNX, permettant au modèle de vision de s'exécuter sur divers runtimes matériels multiplateformes.

BLIP is a vision-language model framework that combines contrastive, matching, and language modeling objectives to align images with text. Built on a multimodal encoder-decoder architecture, it supports distributed data-parallel training with cosine learning rate scheduling and sliding-window metric tracking for training stability. The framework provides capabilities for image captioning, visual question answering, and cross-modal retrieval, scoring semantic alignment between images and text through learned embeddings. It includes toolkits for fine-tuning pre-trained models on custom datasets

Sapiens est un modèle de vision humaine haute résolution conçu pour des tâches de vision par ordinateur centrées sur l'humain de haute précision. Il fonctionne comme une suite d'outils pour estimer la pose humaine, la profondeur et la géométrie de surface. Le projet utilise une architecture vision transformer comme backbone pour effectuer plusieurs tâches via un encodeur partagé. Cette architecture permet la prédiction simultanée des structures squelettiques, des emplacements des articulations et de la distance entre une caméra et un sujet humain. Les capacités du modèle couvrent la segmentation des parties du corps humain pour isoler les régions anatomiques des arrière-plans et la prédiction des normales de surface pour récupérer les détails géométriques 3D à partir d'images 2D. Ces tâches sont soutenues par un framework d'apprentissage multi-tâches qui utilise la régression au niveau des pixels et le masquage par segmentation sémantique.

This is an image segmentation framework and masking toolkit for constructing binary and multi-class neural network architectures. It serves as a deep learning encoder wrapper that integrates pre-trained convolutional neural network architectures into semantic segmentation models. The library enables the use of pre-trained backbones to isolate complex patterns and leverages transfer learning to accelerate training. It provides a collection of overlap-based loss functions and precision metrics specifically designed to evaluate and refine the accuracy of image masks. The toolkit covers the full

Roboflow Sports is a sports video analysis system that combines object detection and tracking with bird's-eye field visualization. Its core pipeline detects and tracks players, referees, and balls across video frames, then maps those tracked positions onto a radar-style overhead view of the playing field. The system goes beyond basic detection by localizing field boundaries and key landmarks such as pitch lines and corners, enabling spatial mapping of player positions relative to the field geometry. It classifies detected players by team affiliation through visual feature extraction and clust

DeiT est un framework de vision transformer PyTorch conçu pour la classification d'images. Il implémente une architecture basée sur des transformers qui traite les images comme des séquences de patchs aplatis en utilisant des couches d'auto-attention et une modélisation de séquence sensible à la position au lieu de filtres convolutifs. Le projet se concentre sur l'entraînement efficace en données grâce à un framework de distillation de connaissances. Ce système permet à un modèle étudiant d'imiter les soft labels d'un modèle enseignant haute performance pour améliorer la précision et la généralisation, particulièrement lors de l'entraînement sur des jeux de données plus petits. La bibliothèque couvre le cycle de vie complet du développement, incluant l'entraînement à la classification d'images, l'optimisation de la perte d'entropie croisée et le déploiement de poids pré-entraînés pour l'inférence. Elle inclut également un outil de benchmarking pour évaluer les performances et la précision du modèle par rapport aux jeux de données standard.

This project is a research framework and toolkit designed for training large-scale vision transformers and multimodal language models. It provides a comprehensive suite for vision-language pretraining, enabling the development of models that map images and text into shared latent spaces. The framework is distinguished by its capabilities in high-fidelity image generation and multimodal research, utilizing normalizing flows and variational autoencoders to produce images from text prompts or class labels. It supports the development of both generative and contrastive models, allowing for a wide