Awesome GitHub RepositoriesVision Transformers

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This library provides a comprehensive collection of modular building blocks and research-backed architectures for implementing vision transformers within the PyTorch framework. It serves as a centralized repository for constructing, training, and analyzing attention-based models, offering a wide array of specialized variants designed for image classification and visual representation learning. The project distinguishes itself through a focus on architectural efficiency and flexibility, supporting diverse input formats including non-square images and volumetric data like video. It incorporates

This project is a comprehensive deep learning framework and educational platform designed for constructing, training, and evaluating neural network architectures. It provides a modular environment for building models through tensor operations and automatic differentiation, supporting a wide range of tasks from image classification and object detection to sequential data processing. Beyond its core technical capabilities, the project distinguishes itself by integrating professional career development resources directly into its learning ecosystem. It offers structured guidance, resume reviews,

Qwen3-VL is a multimodal vision-language model designed to process and reason across images, videos, and text. It functions as a computer vision framework capable of identifying objects, extracting structured data from documents, and interpreting spatial elements within visual media. The system operates as an automated user interface interaction agent, interpreting screen data to navigate software and mobile applications. By utilizing a unified transformer architecture, it performs complex visual reasoning to execute user-defined tasks without manual input. Beyond interface navigation, the m

LaTeX-OCR is a specialized optical character recognition system designed to identify and transcribe complex mathematical symbols and their spatial relationships from images. It functions as a machine learning engine that converts visual representations of equations into structured LaTeX code for use in technical documentation and academic typesetting. The project utilizes a hierarchical vision-based encoding and autoregressive sequence decoding architecture to process input images and generate mathematical notation token by token. Beyond its core recognition capabilities, the system provides

Swin-Transformer is a deep learning framework designed for training and deploying hierarchical vision transformer models. It serves as a research library and toolkit for computer vision tasks, providing the infrastructure to build models that replace standard convolution operations with sliding window self-attention mechanisms. By utilizing a multi-scale feature hierarchy, the framework enables the processing of visual data at varying resolutions and spatial scales. The project distinguishes itself through its implementation of shifted window partitioning, which facilitates global information

PaddleDetection is an object detection framework designed for the end-to-end development, training, and deployment of computer vision models. It provides a comprehensive library of modular neural network architectures and pipelines that support object detection, instance segmentation, and multi-object tracking tasks. The project distinguishes itself through a configuration-driven approach that decouples model components like backbones and heads, allowing for the flexible assembly of custom vision workflows. It incorporates advanced techniques such as anchor-free detection logic, joint detecti

This project is a research library and toolkit for deep learning computer vision, focused on implementing transformer and mixer-based architectures for image classification. It processes visual data by converting images into sequences of patches, allowing standard attention mechanisms to capture global dependencies without relying on traditional convolutional operations. The framework distinguishes itself through its support for multimodal embedding analysis, which maps images and text into a shared latent vector space. This capability enables zero-shot classification and cross-modal retrieva

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This is a PyTorch library and framework for self-supervised vision learning. It provides an implementation of masked autoencoders and vision transformers designed to learn image representations by reconstructing masked image patches from unlabeled data. The project features a distributed training pipeline that scales workloads across multiple GPU nodes. This infrastructure includes multi-node orchestration and gradient accumulation to manage large batch sizes and coordinate resource requests across clusters. The toolkit covers a complete workflow from self-supervised masked pre-training to d

This project is a PyTorch vision transformer framework designed for self-supervised learning. It implements a model that trains visual representations using a momentum teacher and self-distillation without the need for labeled data. The library functions as an image feature extractor and visual attention visualizer, allowing for the generation of high-dimensional vectors and the rendering of self-attention maps as heatmaps or videos to analyze model focus. It provides comprehensive tools for downstream vision evaluation, including linear probe classification, k-nearest neighbor categorizatio

Donut is an OCR-free document transformer and end-to-end document parser. It functions as a neural network that converts unstructured document images directly into structured data or text without the use of an external optical character recognition engine. The project includes a synthetic document generator to create artificial images and ground-truth labels for training. It employs a transformer model to perform visual question answering and document image classification based on visual layout and text. The system covers several document understanding capabilities, including structured info

x-transformers هي مكتبة PyTorch ومجموعة أدوات بحثية لبناء معماريات المحولات (transformers). توفر إطار عمل معيارياً لتنفيذ أبحاث المحولات التجريبية، بما في ذلك مجموعة من آليات الانتباه المتقدمة، وأدوات نمذجة التسلسلات الطويلة، وإطار عمل لمحولات الرؤية. يتميز المشروع بتركيزه على المكونات عالية الأداء والموفرة للذاكرة، مثل Flash Attention مع النوى المبلطة وانتباه الاستعلام المتعدد. كما ينفذ طرقاً متخصصة لتوسيع نوافذ السياق، بما في ذلك تكرار التسلسل وتضمينات الموضع الدورانية. تغطي المكتبة مجموعة واسعة من القدرات المعمارية، بما في ذلك مخططات تطبيع متنوعة لتحقيق استقرار التدريب، وشبكات التغذية الأمامية المبوّبة، وطوبولوجيا الطبقات المخصصة مثل شبكات Macaron. تدعم كلاً من إنشاءات المشفر وفك التشفير، وتوفر أدوات لتوليد التسلسل التلقائي ومهام الرؤية واللغة مثل وصف الصور.

MobileSAM هو نموذج رؤية خفيف الوزن لتقسيم الصور وقابل للتوجيه، مصمم لعزل الكائنات بسرعة على الأجهزة ذات الموارد المحدودة. يعمل كأداة تلقائية لإخفاء الصور قادرة على اكتشاف وعزل كائنات متميزة عبر الصورة بأكملها دون تدخل يدوي. يُمكّن النظام من إخفاء الكائنات بناءً على توجيهات باستخدام نقاط إحداثيات أو مربعات إحاطة لتوليد أقنعة دقيقة. كما يدعم تقسيم الصور لجميع الكائنات من خلال أخذ عينات توجيهية واعية بالكائنات لتحديد كل كائن متميز في المشهد. لتسهيل النشر على الأجهزة المحمولة والحافة، يتوافق النموذج مع تصدير ONNX، مما يسمح لنموذج الرؤية بالعمل عبر بيئات تشغيل أجهزة متنوعة.

BLIP is a vision-language model framework that combines contrastive, matching, and language modeling objectives to align images with text. Built on a multimodal encoder-decoder architecture, it supports distributed data-parallel training with cosine learning rate scheduling and sliding-window metric tracking for training stability. The framework provides capabilities for image captioning, visual question answering, and cross-modal retrieval, scoring semantic alignment between images and text through learned embeddings. It includes toolkits for fine-tuning pre-trained models on custom datasets

Sapiens هو نموذج رؤية بشرية عالي الدقة مصمم لمهام رؤية الكمبيوتر التي تركز على الإنسان بدقة عالية. يعمل كمجموعة من الأدوات لتقدير وضعية الإنسان، والعمق، وهندسة السطح. يستخدم المشروع هيكل vision transformer لأداء مهام متعددة من خلال مشفر مشترك. تتيح هذه البنية التنبؤ المتزامن بالهياكل الهيكلية، ومواقع المفاصل، والمسافة بين الكاميرا والموضوع البشري. تغطي قدرات النموذج تجزئة أجزاء جسم الإنسان لعزل المناطق التشريحية عن الخلفيات والتنبؤ بوضع السطح لاستعادة التفاصيل الهندسية ثلاثية الأبعاد من الصور ثنائية الأبعاد. يتم دعم هذه المهام من خلال إطار عمل للتعلم متعدد المهام يستخدم الانحدار على مستوى البكسل وقناع التجزئة الدلالي.

This is an image segmentation framework and masking toolkit for constructing binary and multi-class neural network architectures. It serves as a deep learning encoder wrapper that integrates pre-trained convolutional neural network architectures into semantic segmentation models. The library enables the use of pre-trained backbones to isolate complex patterns and leverages transfer learning to accelerate training. It provides a collection of overlap-based loss functions and precision metrics specifically designed to evaluate and refine the accuracy of image masks. The toolkit covers the full

Roboflow Sports is a sports video analysis system that combines object detection and tracking with bird's-eye field visualization. Its core pipeline detects and tracks players, referees, and balls across video frames, then maps those tracked positions onto a radar-style overhead view of the playing field. The system goes beyond basic detection by localizing field boundaries and key landmarks such as pitch lines and corners, enabling spatial mapping of player positions relative to the field geometry. It classifies detected players by team affiliation through visual feature extraction and clust

DeiT هو إطار عمل محول رؤية (vision transformer) لـ PyTorch مصمم لتصنيف الصور. ينفذ معمارية قائمة على المحولات تعالج الصور كتسلسلات من الرقع المسطحة باستخدام طبقات الانتباه الذاتي ونمذجة التسلسل الواعية بالموقع بدلاً من المرشحات التلافيفية. يركز المشروع على التدريب الفعال للبيانات من خلال إطار عمل لتقطير المعرفة. يسمح هذا النظام لنموذج الطالب بتقليد التسميات اللينة لنموذج معلم عالي الأداء لتحسين الدقة والتعميم، خاصة عند التدريب على مجموعات بيانات أصغر. تغطي المكتبة دورة حياة التطوير الكاملة، بما في ذلك تدريب تصنيف الصور، وتحسين فقدان الإنتروبيا المتقاطعة، ونشر الأوزان المدربة مسبقاً للاستنتاج. كما تتضمن أداة قياس لتقييم أداء النموذج ودقته مقابل مجموعات البيانات القياسية.

This project is a research framework and toolkit designed for training large-scale vision transformers and multimodal language models. It provides a comprehensive suite for vision-language pretraining, enabling the development of models that map images and text into shared latent spaces. The framework is distinguished by its capabilities in high-fidelity image generation and multimodal research, utilizing normalizing flows and variational autoencoders to produce images from text prompts or class labels. It supports the development of both generative and contrastive models, allowing for a wide