Projects sharing features with Mm Cot

AReaL is a system for agent orchestration, distributed model training, and parameter-efficient tuning. It provides a framework for developing multi-turn reasoning agents and training large models using reinforcement learning from human feedback. The project implements a toolkit for improving the visual reasoning and geometry problem solving capabilities of vision-language models. It utilizes a memory-efficient tuning system to optimize mathematical and reasoning models across different inference backends. The infrastructure supports large-scale training through tensor, pipeline, and expert p

This project is a comprehensive instructional resource and course for building neural networks using PyTorch. It covers the fundamental building blocks of deep learning, including tensor manipulation, automatic differentiation, and the construction of modular neural network components. The repository serves as a technical guide for several specialized domains. It provides implementation details for computer vision tasks such as image classification, object detection, and semantic segmentation, as well as natural language processing workflows involving transformers, recurrent networks, and gen

Prompt patterns is a framework for organizing AI-driven system design through structured prompt engineering and domain-driven development methodologies. It provides a library of standardized interaction strategies designed to improve the consistency, accuracy, and logical reasoning of large language model outputs. By applying these patterns, users can translate complex business scenarios into structured domain models and technical specifications. The project distinguishes itself by integrating domain-driven design principles directly into the prompting workflow. It utilizes techniques such as

R1-V is a toolset for the development of multimodal models, providing a low-cost training environment designed to optimize the reasoning and feedback loops of large vision-language models. It integrates a training framework, fine-tuning pipelines, and performance evaluation tools. The project features a reinforcement learning framework that improves visual reasoning and generalization by rewarding correct outputs based on visual verification. It also includes a supervised fine-tuning pipeline for customizing vision-language models to specific tasks using labeled datasets and configuration fil

Open CLIP is an open source framework for training and deploying Contrastive Language-Image Pre-training models. It serves as a vision-language training framework and multimodal embedding engine that maps images and text into a shared vector space for similarity searches and zero-shot classification. The project provides a toolkit for distributed training of contrastive models and includes an image-to-text generative model for producing natural language descriptions. It supports custom text encoder integration and utilizes teacher-student model distillation to transfer knowledge from large pr

Neuraltalk2 is a deep learning vision system designed for automatic image captioning. Built with PyTorch, it utilizes a hybrid architecture that combines a convolutional neural network encoder with a recurrent neural network decoder to generate textual descriptions from visual input. The project features a GPU-accelerated training pipeline capable of distributing workloads across multiple graphics processing units through multi-process distribution. It supports the generation of descriptions for both static image files and real-time video streams. The framework includes capabilities for enco

This project is a PyTorch person re-identification framework designed for training and evaluating models that identify individuals across different camera views. It provides a complete model training pipeline, a deep learning feature extractor for converting images into numeric vectors, and a suite of computer vision benchmarking tools to measure identity retrieval accuracy. The framework includes a specialized transfer learning toolkit that supports layer freezing, staged learning rate optimization, and differential learning rates for fine-tuning pretrained models. It distinguishes itself th

Kornia is a differentiable computer vision library and cross-framework tensor vision toolset. It implements vision operations as differentiable tensors to enable integration into deep learning pipelines and supports the transpilation of operations across PyTorch, TensorFlow, JAX, and NumPy. The project provides specialized toolsets for geometric vision and stereo depth, including algorithms for 3D scene reconstruction, camera calibration, and pose estimation. It further distinguishes itself as a differentiable image augmentation framework, applying random geometric and color transformations w

EasyR1 is a distributed model training system and reinforcement learning framework for large language and vision-language models. It functions as a multimodal trainer and an implementation of a Proximal Policy Optimization pipeline designed to refine the reasoning and perception capabilities of models that process both text and images. The system specializes in distributing reinforcement learning workloads across multiple compute nodes to manage high memory requirements. It optimizes hardware utilization through padding-free training and fine-tuning to fit large models onto available graphics

DeepDream is a deep learning image processor and convolutional neural network art generator designed to synthesize psychedelic imagery and visualize how neural networks interpret visual data. It functions as a tool for generating generative AI art by amplifying patterns recognized by a pre-trained model to produce dream-like effects. The project utilizes a TensorFlow image visualizer to explore how different layers of a neural network perceive images. This is achieved through algorithmic image manipulation and deep learning visualization techniques that transform standard photographs into sty

Neuraltalk is an automated image captioning system that generates natural language descriptions for images. It utilizes a deep learning model that integrates a pretrained convolutional neural network for visual feature extraction with a recurrent neural network decoder to produce text sequences. The project provides a full workflow for training and evaluating captioning models, including weight optimization via backpropagation and gradient descent. It includes tools for measuring caption accuracy by comparing generated text against reference descriptions. The system covers data preprocessing

LLaVA-NeXT is a multimodal large language model framework and training toolkit designed to process interleaved images and video sequences to generate text. It functions as a visual language model that combines vision encoders with language models to perform complex reasoning, question answering, and video understanding. The system is capable of analyzing high-resolution images and temporal video frames to describe events, summarize actions, and reason across multiple visual inputs. It supports the interpretation of documents and charts, spatial environment analysis, and the generation of desc

nanoVLM is a training framework and toolkit for small vision-language models. It provides a PyTorch-based environment for training and fine-tuning models to associate image inputs with textual descriptions and generate natural language answers. The project includes a cloud model versioning tool for saving and loading model weights to centralized repositories to synchronize assets across environments. It also features a dedicated evaluation suite for measuring the accuracy and reliability of vision-language models against standard task datasets. The framework covers GPU resource planning thro

AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end pipeline from data preprocessing to high-accuracy model training and validation. It functions as an automated model trainer for tabular, image, text, and time series data, as well as a tool for time series forecasting and foundation model finetuning. The project is distinguished by its ability to jointly process and fuse different data types, allowing for the construction of multimodal neural networks that integrate images, text, and structured tables. It supports zero-shot inferenc

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This is a PyTorch implementation of a neural style transfer system. It functions as a convolutional neural network image stylizer and artistic style blender designed to combine the content of one image with the artistic style of another. The system supports blending multiple style sources and adjusting the relative weights between content and style reconstruction. It includes capabilities for preserving the original color palette of the content image and adjusting style scales to determine which artistic patterns are transferred. The pipeline enables high-resolution image processing by distr

DeepSeek-VL is a multimodal large language model and image-to-text reasoning engine. It functions as a vision-language model and visual question answering system that integrates visual perception with linguistic reasoning to understand and describe images. The project enables multimodal image understanding and document image analysis, specifically processing screenshots of web pages and technical diagrams. It provides capabilities for visual conversational AI, allowing users to interact with visual data to extract insights and perform complex reasoning across different types of visual informa

This project is a pretrained model library for PyTorch, providing a collection of convolutional neural network architectures and weights. It serves as a computer vision model zoo for image classification and feature extraction, offering a framework for transfer learning where pretrained networks are adapted for custom image recognition tasks. The library focuses on transforming images into high-level numerical representations and calculating class probability scores. It includes utilities for downloading and initializing standard architectures such as ResNet, Inception, and Xception. Capabil

This project is a fine-tuning framework and training pipeline designed to optimize and adapt large language and vision models. It provides a specialized toolkit for parameter-efficient tuning and supervised learning, serving as both a trainer for multimodal models and a deployment tool for serving fine-tuned models via high-performance inference engines. The framework focuses on reducing memory and compute requirements by updating a small subset of model parameters. It supports a wide range of adaptation strategies, including vision-language model training to align text, image, video, and aud

Dimensionality reduction alo codes is a Python collection of implementations for classic data reduction algorithms, functioning as a machine learning feature extraction tool. It transforms complex high-dimensional datasets into lower-dimensional spaces through techniques that retain maximum variance, maximize class separability, and translate proximities between data points into spatial distances. The library includes capabilities for eigenvalue variance maximization, linear discriminant projection, nonlinear neighborhood preserving embedding, singular value matrix decomposition, and statist

Detectron2 is a PyTorch computer vision framework and visual recognition platform designed for training and deploying models for object detection, image segmentation, and visual recognition. It provides a research-oriented environment for training complex vision models with multi-GPU acceleration. The project includes a specialized object detection library for identifying and locating multiple objects via bounding boxes, as well as an image segmentation toolkit for creating pixel-level masks through instance, semantic, and panoptic segmentation. Additionally, it features a human pose estimati

This project is a 3D visual localization framework designed to determine a camera's exact position and orientation by matching 2D image features against a 3D reference model. It includes a structure-from-motion pipeline to reconstruct 3D scene geometry from unordered image sets, creating the necessary spatial maps for localization. The system employs a hierarchical coarse-to-fine localization approach. This process begins with a global-descriptor image retrieval system to identify candidate reference images from a large database and progresses through local feature matching to final 3D model

This project is a neural network image classifier and a set of tools for building and training convolutional neural networks to recognize and categorize images. It serves as a machine learning educational guide, providing a practical resource for learning neural network fundamentals through an onboarding process. The system includes a dedicated workflow for pretrained model fine-tuning, allowing existing network weights to be adapted to new image categories. This is supported by a transfer learning pipeline that replaces final classification layers and adjusts weights through targeted retrain

tracking.js is a browser computer vision library written in JavaScript for performing real-time image analysis and object tracking directly within a web browser. It functions as a real-time object tracker, a color tracking tool, and a face detection utility. The library enables the detection and monitoring of specific color ranges, human faces, and known visual patterns across consecutive video frames. It extracts visual features and descriptors from images to identify distinct landmarks for matching and tracking. The project covers broad computer vision capabilities, including the ability t

lmms-eval is a benchmarking system and performance analysis suite designed to measure the capabilities of large multimodal models. It provides a framework for evaluating models across text, image, audio, and video datasets, serving as a multimodal dataset orchestrator and benchmarking tool to quantify accuracy and efficiency. The project distinguishes itself through a unified multimodal message protocol that structures diverse media inputs for consistent model consumption. It features specialized benchmarking for audio, video, visual, document, and spatial reasoning, alongside tools for model

Otter is a framework and toolkit for the pretraining, fine-tuning, and evaluation of vision-language models. It provides a pipeline for training large language models to process high-resolution images and video frames, integrating visual encoders with textual token spaces. The system is designed for multi-visual input processing, allowing models to interpret multiple images or video sequences within a single prompt. It supports multi-round conversation management to maintain context across interactions for detailed scene comprehension and visual reasoning. The framework covers a full develop

This project provides a transformer-based object detection model that treats the task as a direct set prediction problem. It implements a vision system capable of predicting bounding boxes and class labels for objects within an image, as well as frameworks for instance and panoptic segmentation. The architecture utilizes a transformer encoder and decoder to perform end-to-end set prediction, employing a Hungarian matcher to assign predicted boxes to ground truth objects. It incorporates a convolutional backbone for feature extraction and a system of learnable object queries to probe image loc