Awesome GitHub RepositoriesVision Transformers

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This library provides a comprehensive collection of modular building blocks and research-backed architectures for implementing vision transformers within the PyTorch framework. It serves as a centralized repository for constructing, training, and analyzing attention-based models, offering a wide array of specialized variants designed for image classification and visual representation learning. The project distinguishes itself through a focus on architectural efficiency and flexibility, supporting diverse input formats including non-square images and volumetric data like video. It incorporates

This project is a comprehensive deep learning framework and educational platform designed for constructing, training, and evaluating neural network architectures. It provides a modular environment for building models through tensor operations and automatic differentiation, supporting a wide range of tasks from image classification and object detection to sequential data processing. Beyond its core technical capabilities, the project distinguishes itself by integrating professional career development resources directly into its learning ecosystem. It offers structured guidance, resume reviews,

Qwen3-VL is a multimodal vision-language model designed to process and reason across images, videos, and text. It functions as a computer vision framework capable of identifying objects, extracting structured data from documents, and interpreting spatial elements within visual media. The system operates as an automated user interface interaction agent, interpreting screen data to navigate software and mobile applications. By utilizing a unified transformer architecture, it performs complex visual reasoning to execute user-defined tasks without manual input. Beyond interface navigation, the m

LaTeX-OCR is a specialized optical character recognition system designed to identify and transcribe complex mathematical symbols and their spatial relationships from images. It functions as a machine learning engine that converts visual representations of equations into structured LaTeX code for use in technical documentation and academic typesetting. The project utilizes a hierarchical vision-based encoding and autoregressive sequence decoding architecture to process input images and generate mathematical notation token by token. Beyond its core recognition capabilities, the system provides

Swin-Transformer is a deep learning framework designed for training and deploying hierarchical vision transformer models. It serves as a research library and toolkit for computer vision tasks, providing the infrastructure to build models that replace standard convolution operations with sliding window self-attention mechanisms. By utilizing a multi-scale feature hierarchy, the framework enables the processing of visual data at varying resolutions and spatial scales. The project distinguishes itself through its implementation of shifted window partitioning, which facilitates global information

PaddleDetection is an object detection framework designed for the end-to-end development, training, and deployment of computer vision models. It provides a comprehensive library of modular neural network architectures and pipelines that support object detection, instance segmentation, and multi-object tracking tasks. The project distinguishes itself through a configuration-driven approach that decouples model components like backbones and heads, allowing for the flexible assembly of custom vision workflows. It incorporates advanced techniques such as anchor-free detection logic, joint detecti

This project is a research library and toolkit for deep learning computer vision, focused on implementing transformer and mixer-based architectures for image classification. It processes visual data by converting images into sequences of patches, allowing standard attention mechanisms to capture global dependencies without relying on traditional convolutional operations. The framework distinguishes itself through its support for multimodal embedding analysis, which maps images and text into a shared latent vector space. This capability enables zero-shot classification and cross-modal retrieva

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This is a PyTorch library and framework for self-supervised vision learning. It provides an implementation of masked autoencoders and vision transformers designed to learn image representations by reconstructing masked image patches from unlabeled data. The project features a distributed training pipeline that scales workloads across multiple GPU nodes. This infrastructure includes multi-node orchestration and gradient accumulation to manage large batch sizes and coordinate resource requests across clusters. The toolkit covers a complete workflow from self-supervised masked pre-training to d

This project is a PyTorch vision transformer framework designed for self-supervised learning. It implements a model that trains visual representations using a momentum teacher and self-distillation without the need for labeled data. The library functions as an image feature extractor and visual attention visualizer, allowing for the generation of high-dimensional vectors and the rendering of self-attention maps as heatmaps or videos to analyze model focus. It provides comprehensive tools for downstream vision evaluation, including linear probe classification, k-nearest neighbor categorizatio

Donut is an OCR-free document transformer and end-to-end document parser. It functions as a neural network that converts unstructured document images directly into structured data or text without the use of an external optical character recognition engine. The project includes a synthetic document generator to create artificial images and ground-truth labels for training. It employs a transformer model to perform visual question answering and document image classification based on visual layout and text. The system covers several document understanding capabilities, including structured info

x-transformers एक PyTorch लाइब्रेरी और रिसर्च टूलकिट है जिसे ट्रांसफॉर्मर आर्किटेक्चर बनाने के लिए बनाया गया है। यह प्रयोगात्मक ट्रांसफॉर्मर रिसर्च को लागू करने के लिए एक मॉड्यूलर फ्रेमवर्क प्रदान करता है, जिसमें उन्नत अटेंशन मैकेनिज्म, लॉन्ग-सीक्वेंस मॉडलिंग टूल्स और विज़न ट्रांसफॉर्मर्स के लिए एक फ्रेमवर्क शामिल है। यह प्रोजेक्ट मेमोरी-एफिशिएंट और हाई-परफॉरमेंस कंपोनेंट्स पर केंद्रित है, जैसे कि टाइल्ड कर्नेल के साथ Flash Attention और मल्टी-क्वेरी अटेंशन। यह कॉन्टेक्स्ट विंडोज़ का विस्तार करने के लिए विशेष तरीके भी लागू करता है, जिसमें सीक्वेंस रिकरेंस और रोटरी पोजीशनल एम्बेडिंग्स शामिल हैं। यह लाइब्रेरी ट्रेनिंग को स्थिर करने के लिए विभिन्न नॉर्मलाइजेशन स्कीम्स, गेटेड फीडफॉरवर्ड नेटवर्क्स और Macaron नेटवर्क्स जैसे कस्टम लेयर टोपोलॉजी सहित आर्किटेक्चरल क्षमताओं की एक विस्तृत श्रृंखला को कवर करती है।

MobileSAM is a lightweight image segmenter and promptable vision model designed for fast object isolation on resource-constrained hardware. It functions as an automatic image masking tool capable of detecting and isolating distinct objects across an entire image without manual input. The system enables prompt-based object masking using coordinate points or bounding boxes to generate precise masks. It also supports all-object image segmentation through object-aware prompt sampling to identify every distinct object in a scene. To facilitate mobile and edge deployment, the model is compatible w

BLIP is a vision-language model framework that combines contrastive, matching, and language modeling objectives to align images with text. Built on a multimodal encoder-decoder architecture, it supports distributed data-parallel training with cosine learning rate scheduling and sliding-window metric tracking for training stability. The framework provides capabilities for image captioning, visual question answering, and cross-modal retrieval, scoring semantic alignment between images and text through learned embeddings. It includes toolkits for fine-tuning pre-trained models on custom datasets

Sapiens is a high-resolution human vision model designed for high-precision, human-centric computer vision tasks. It functions as a suite of tools for estimating human pose, depth, and surface geometry. The project utilizes a vision transformer backbone to perform multiple tasks through a shared encoder. This architecture enables the simultaneous prediction of skeletal structures, joint locations, and the distance between a camera and a human subject. The model's capabilities cover human body part segmentation to isolate anatomical regions from backgrounds and surface normal prediction to re

This is an image segmentation framework and masking toolkit for constructing binary and multi-class neural network architectures. It serves as a deep learning encoder wrapper that integrates pre-trained convolutional neural network architectures into semantic segmentation models. The library enables the use of pre-trained backbones to isolate complex patterns and leverages transfer learning to accelerate training. It provides a collection of overlap-based loss functions and precision metrics specifically designed to evaluate and refine the accuracy of image masks. The toolkit covers the full

Roboflow Sports is a sports video analysis system that combines object detection and tracking with bird's-eye field visualization. Its core pipeline detects and tracks players, referees, and balls across video frames, then maps those tracked positions onto a radar-style overhead view of the playing field. The system goes beyond basic detection by localizing field boundaries and key landmarks such as pitch lines and corners, enabling spatial mapping of player positions relative to the field geometry. It classifies detected players by team affiliation through visual feature extraction and clust

DeiT एक PyTorch विज़न ट्रांसफार्मर फ्रेमवर्क है जिसे छवि वर्गीकरण के लिए डिज़ाइन किया गया है। यह एक ट्रांसफार्मर-आधारित आर्किटेक्चर को लागू करता है जो कनवल्शनल फ़िल्टर के बजाय सेल्फ-अटेंशन लेयर्स और स्थिति-जागरूक अनुक्रम मॉडलिंग का उपयोग करके छवियों को चपटे पैच (flattened patches) के अनुक्रम के रूप में संसाधित करता है। यह प्रोजेक्ट नॉलेज डिस्टिलेशन फ्रेमवर्क के माध्यम से डेटा-कुशल प्रशिक्षण पर केंद्रित है। यह सिस्टम एक छात्र मॉडल को उच्च-प्रदर्शन शिक्षक मॉडल के सॉफ्ट लेबल की नकल करने की अनुमति देता है ताकि सटीकता और सामान्यीकरण में सुधार हो सके, विशेष रूप से छोटे डेटासेट पर प्रशिक्षण देते समय। लाइब्रेरी छवि वर्गीकरण प्रशिक्षण, क्रॉस-एंट्रॉपी लॉस ऑप्टिमाइज़ेशन और इन्फरेंस के लिए प्रीट्रेन्ड वेट्स की तैनाती सहित पूर्ण विकास लाइफसाइकिल को कवर करती है। इसमें मानक डेटासेट के खिलाफ मॉडल प्रदर्शन और सटीकता का मूल्यांकन करने के लिए एक बेंचमार्किंग टूल भी शामिल है।

This project is a research framework and toolkit designed for training large-scale vision transformers and multimodal language models. It provides a comprehensive suite for vision-language pretraining, enabling the development of models that map images and text into shared latent spaces. The framework is distinguished by its capabilities in high-fidelity image generation and multimodal research, utilizing normalizing flows and variational autoencoders to produce images from text prompts or class labels. It supports the development of both generative and contrastive models, allowing for a wide