Awesome GitHub RepositoriesVision Transformers

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This library provides a comprehensive collection of modular building blocks and research-backed architectures for implementing vision transformers within the PyTorch framework. It serves as a centralized repository for constructing, training, and analyzing attention-based models, offering a wide array of specialized variants designed for image classification and visual representation learning. The project distinguishes itself through a focus on architectural efficiency and flexibility, supporting diverse input formats including non-square images and volumetric data like video. It incorporates

This project is a comprehensive deep learning framework and educational platform designed for constructing, training, and evaluating neural network architectures. It provides a modular environment for building models through tensor operations and automatic differentiation, supporting a wide range of tasks from image classification and object detection to sequential data processing. Beyond its core technical capabilities, the project distinguishes itself by integrating professional career development resources directly into its learning ecosystem. It offers structured guidance, resume reviews,

Qwen3-VL is a multimodal vision-language model designed to process and reason across images, videos, and text. It functions as a computer vision framework capable of identifying objects, extracting structured data from documents, and interpreting spatial elements within visual media. The system operates as an automated user interface interaction agent, interpreting screen data to navigate software and mobile applications. By utilizing a unified transformer architecture, it performs complex visual reasoning to execute user-defined tasks without manual input. Beyond interface navigation, the m

LaTeX-OCR is a specialized optical character recognition system designed to identify and transcribe complex mathematical symbols and their spatial relationships from images. It functions as a machine learning engine that converts visual representations of equations into structured LaTeX code for use in technical documentation and academic typesetting. The project utilizes a hierarchical vision-based encoding and autoregressive sequence decoding architecture to process input images and generate mathematical notation token by token. Beyond its core recognition capabilities, the system provides

Swin-Transformer is a deep learning framework designed for training and deploying hierarchical vision transformer models. It serves as a research library and toolkit for computer vision tasks, providing the infrastructure to build models that replace standard convolution operations with sliding window self-attention mechanisms. By utilizing a multi-scale feature hierarchy, the framework enables the processing of visual data at varying resolutions and spatial scales. The project distinguishes itself through its implementation of shifted window partitioning, which facilitates global information

PaddleDetection is an object detection framework designed for the end-to-end development, training, and deployment of computer vision models. It provides a comprehensive library of modular neural network architectures and pipelines that support object detection, instance segmentation, and multi-object tracking tasks. The project distinguishes itself through a configuration-driven approach that decouples model components like backbones and heads, allowing for the flexible assembly of custom vision workflows. It incorporates advanced techniques such as anchor-free detection logic, joint detecti

This project is a research library and toolkit for deep learning computer vision, focused on implementing transformer and mixer-based architectures for image classification. It processes visual data by converting images into sequences of patches, allowing standard attention mechanisms to capture global dependencies without relying on traditional convolutional operations. The framework distinguishes itself through its support for multimodal embedding analysis, which maps images and text into a shared latent vector space. This capability enables zero-shot classification and cross-modal retrieva

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This is a PyTorch library and framework for self-supervised vision learning. It provides an implementation of masked autoencoders and vision transformers designed to learn image representations by reconstructing masked image patches from unlabeled data. The project features a distributed training pipeline that scales workloads across multiple GPU nodes. This infrastructure includes multi-node orchestration and gradient accumulation to manage large batch sizes and coordinate resource requests across clusters. The toolkit covers a complete workflow from self-supervised masked pre-training to d

This project is a PyTorch vision transformer framework designed for self-supervised learning. It implements a model that trains visual representations using a momentum teacher and self-distillation without the need for labeled data. The library functions as an image feature extractor and visual attention visualizer, allowing for the generation of high-dimensional vectors and the rendering of self-attention maps as heatmaps or videos to analyze model focus. It provides comprehensive tools for downstream vision evaluation, including linear probe classification, k-nearest neighbor categorizatio

Donut is an OCR-free document transformer and end-to-end document parser. It functions as a neural network that converts unstructured document images directly into structured data or text without the use of an external optical character recognition engine. The project includes a synthetic document generator to create artificial images and ground-truth labels for training. It employs a transformer model to perform visual question answering and document image classification based on visual layout and text. The system covers several document understanding capabilities, including structured info

x-transformers 是一个用于构建 Transformer 架构的 PyTorch 库和研究工具包。它提供了一个模块化框架来实现实验性 Transformer 研究，包括一套先进的注意力机制、长序列建模工具以及视觉 Transformer 框架。 该项目以专注于内存高效和高性能组件而著称，例如带有分块内核的 Flash Attention 和多查询注意力。它还实现了用于扩展上下文窗口的专门方法，包括序列循环和旋转位置嵌入。 该库涵盖了广泛的架构功能，包括用于稳定训练的各种归一化方案、门控前馈网络以及 Macaron 网络等自定义层拓扑。它支持编码器和解码器构造，为自回归序列生成和图像描述等视觉-语言任务提供工具。

MobileSAM 是一个轻量级图像分割器和可提示视觉模型，专为资源受限硬件上的快速对象隔离而设计。它具有自动图像遮罩功能，无需人工输入即可检测并隔离整个图像中的不同对象。 该系统支持使用坐标点或边界框进行基于提示的对象遮罩，以生成精确的掩码。它还支持通过对象感知提示采样进行全对象图像分割，以识别场景中的每一个不同对象。 为了便于移动端和边缘侧部署，该模型兼容 ONNX 导出，允许视觉模型在各种跨平台硬件运行时上运行。

BLIP is a vision-language model framework that combines contrastive, matching, and language modeling objectives to align images with text. Built on a multimodal encoder-decoder architecture, it supports distributed data-parallel training with cosine learning rate scheduling and sliding-window metric tracking for training stability. The framework provides capabilities for image captioning, visual question answering, and cross-modal retrieval, scoring semantic alignment between images and text through learned embeddings. It includes toolkits for fine-tuning pre-trained models on custom datasets

Sapiens 是一个高分辨率人体视觉模型，专为高精度、以人为中心的计算机视觉任务而设计。它是一套用于估计人体姿态、深度和表面几何形状的工具集。 该项目利用视觉 Transformer 主干网络通过共享编码器执行多项任务。这种架构能够同时预测骨骼结构、关节位置以及相机与人体对象之间的距离。 该模型的功能涵盖了人体部位分割（从背景中分离解剖区域）和表面法线预测（从 2D 图像中恢复 3D 几何细节）。这些任务由一个采用像素级回归和语义分割掩码的多任务学习框架提供支持。

This is an image segmentation framework and masking toolkit for constructing binary and multi-class neural network architectures. It serves as a deep learning encoder wrapper that integrates pre-trained convolutional neural network architectures into semantic segmentation models. The library enables the use of pre-trained backbones to isolate complex patterns and leverages transfer learning to accelerate training. It provides a collection of overlap-based loss functions and precision metrics specifically designed to evaluate and refine the accuracy of image masks. The toolkit covers the full

Roboflow Sports is a sports video analysis system that combines object detection and tracking with bird's-eye field visualization. Its core pipeline detects and tracks players, referees, and balls across video frames, then maps those tracked positions onto a radar-style overhead view of the playing field. The system goes beyond basic detection by localizing field boundaries and key landmarks such as pitch lines and corners, enabling spatial mapping of player positions relative to the field geometry. It classifies detected players by team affiliation through visual feature extraction and clust

DeiT 是一个专为图像分类设计的 PyTorch Vision Transformer 框架。它实现了一种基于 Transformer 的架构，使用自注意力层和位置感知序列建模而不是卷积滤波器，将图像处理为扁平化补丁序列。 该项目专注于通过知识蒸馏框架进行数据高效训练。该系统允许学生模型模仿高性能教师模型的软标签，以提高准确性和泛化能力，特别是在较小数据集上进行训练时。 该库涵盖了完整的开发生命周期，包括图像分类训练、交叉熵损失优化以及部署预训练权重进行推理。它还包括一个基准测试工具，用于评估模型在标准数据集上的性能和准确性。

This project is a research framework and toolkit designed for training large-scale vision transformers and multimodal language models. It provides a comprehensive suite for vision-language pretraining, enabling the development of models that map images and text into shared latent spaces. The framework is distinguished by its capabilities in high-fidelity image generation and multimodal research, utilizing normalizing flows and variational autoencoders to produce images from text prompts or class labels. It supports the development of both generative and contrastive models, allowing for a wide