Projects sharing features with LLaVA Plus Codebase

Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic speech recognition, and large-scale language model training. It provides a framework for processing and aligning diverse data sources, including text, audio, and video, to support tasks such as speech-to-text conversion and multimodal sequence learning. The project is distinguished by its distributed training capabilities, which utilize parameter sharding, mixed-precision training, and CPU offloading to handle models that exceed single-device memory. It also includes specializ

ALBERT

This project is a transformer-based language model and natural language processing toolkit designed to generate deep contextual representations of text. By utilizing a transformer-based encoder architecture, the system processes input sequences through stacked self-attention layers to capture the semantic meaning of tokens based on their surrounding sentence structure. The model distinguishes itself through bidirectional contextual processing, which analyzes text in both directions simultaneously, and masked language modeling, which trains the system by predicting hidden tokens within a seque

Scenic is a research framework designed for the development and training of deep learning models, with a specific focus on computer vision and multimodal transformer architectures. It provides a comprehensive toolkit for defining neural network structures, managing large-scale data pipelines, and executing training workflows across distributed hardware environments. The framework is built upon a functional programming paradigm that utilizes hardware-agnostic tensor abstractions and just-in-time compilation to maximize computational efficiency. By employing modular layer composition, it allows

Grounding DINO 1.5

This project provides a deep residual network framework and pre-trained PyTorch models designed for high-accuracy image recognition. It implements a neural network architecture that utilizes skip connections to enable the training of very deep models without gradient degradation. The system is designed for computer vision tasks, including image classification, object detection, and visual data segmentation. It includes weights trained on ImageNet to support transfer learning and the fine-tuning of models on custom image datasets. The architectural design focuses on residual learning blocks,

NeurIPS 2022 Spotlight VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

JARVIS is a system for large language model task orchestration, deployment management, and automation benchmarking. It utilizes a task orchestrator to decompose complex requests into actionable steps and coordinates various expert models to synthesize final responses. The project includes an AI model deployment manager to handle the local deployment of expert models across different hardware scales. It further provides an AI workflow API consisting of web endpoints used to trigger automated task workflows and retrieve results from model selection stages. The framework incorporates an automat

Official repo for MM-REACT