Awesome GitHub RepositoriesEncoder-Decoder Architectures

Donut is an OCR-free document transformer and end-to-end document parser. It functions as a neural network that converts unstructured document images directly into structured data or text without the use of an external optical character recognition engine. The project includes a synthetic document generator to create artificial images and ground-truth labels for training. It employs a transformer model to perform visual question answering and document image classification based on visual layout and text. The system covers several document understanding capabilities, including structured info

BLIP is a vision-language model framework that combines contrastive, matching, and language modeling objectives to align images with text. Built on a multimodal encoder-decoder architecture, it supports distributed data-parallel training with cosine learning rate scheduling and sliding-window metric tracking for training stability. The framework provides capabilities for image captioning, visual question answering, and cross-modal retrieval, scoring semantic alignment between images and text through learned embeddings. It includes toolkits for fine-tuning pre-trained models on custom datasets

Sapiens 是一个高分辨率人体视觉模型，专为高精度、以人为中心的计算机视觉任务而设计。它是一套用于估计人体姿态、深度和表面几何形状的工具集。 该项目利用视觉 Transformer 主干网络通过共享编码器执行多项任务。这种架构能够同时预测骨骼结构、关节位置以及相机与人体对象之间的距离。 该模型的功能涵盖了人体部位分割（从背景中分离解剖区域）和表面法线预测（从 2D 图像中恢复 3D 几何细节）。这些任务由一个采用像素级回归和语义分割掩码的多任务学习框架提供支持。

This is an image segmentation framework and masking toolkit for constructing binary and multi-class neural network architectures. It serves as a deep learning encoder wrapper that integrates pre-trained convolutional neural network architectures into semantic segmentation models. The library enables the use of pre-trained backbones to isolate complex patterns and leverages transfer learning to accelerate training. It provides a collection of overlap-based loss functions and precision metrics specifically designed to evaluate and refine the accuracy of image masks. The toolkit covers the full

Roboflow Sports is a sports video analysis system that combines object detection and tracking with bird's-eye field visualization. Its core pipeline detects and tracks players, referees, and balls across video frames, then maps those tracked positions onto a radar-style overhead view of the playing field. The system goes beyond basic detection by localizing field boundaries and key landmarks such as pitch lines and corners, enabling spatial mapping of player positions relative to the field geometry. It classifies detected players by team affiliation through visual feature extraction and clust

This project is a research framework and toolkit designed for training large-scale vision transformers and multimodal language models. It provides a comprehensive suite for vision-language pretraining, enabling the development of models that map images and text into shared latent spaces. The framework is distinguished by its capabilities in high-fidelity image generation and multimodal research, utilizing normalizing flows and variational autoencoders to produce images from text prompts or class labels. It supports the development of both generative and contrastive models, allowing for a wide

SegFormer is a semantic segmentation framework and transformer-based model designed for pixel-level image classification. It provides a deep learning architecture that assigns class labels to pixels using a hierarchical transformer encoder and a multi-layer perceptron decoder. The framework utilizes a hierarchical transformer encoder to process multi-scale features through a pyramid of blocks and an all-MLP decoder to aggregate these features without complex attention mechanisms. It incorporates overlap patch embedding to preserve local continuity and sequential self-attention reduction to ma

manga-ocr is a Japanese OCR engine and text extraction tool designed to recognize vertical and horizontal Japanese text from manga images. It operates as a vision encoder-decoder model that converts visual text into digital characters. The project includes an OCR training pipeline and a synthetic data generator. These tools create artificial image-text pairs by overlaying diverse Japanese text fonts onto background images to refine recognition models. The system provides automation for extracting text by monitoring the system clipboard or directories. This allows for the conversion of manga