How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
ERNIE is a development toolkit for training, fine-tuning, and deploying large language models built on the PaddlePaddle deep learning platform. It provides a comprehensive suite of core components, including an inference server for vision and language models, a training and fine-tuning toolkit, and a framework for building retrieval-augmented generation systems using private knowledge bases. The project features multimodal AI models capable of reasoning across text, images, and video to perform complex visual understanding and information extraction. It distinguishes itself through specialize
Pythia is a multimodal research framework and distributed training system designed for building, training, and evaluating large models that combine visual and linguistic data. It provides a modular environment for developing vision-language models, focusing on the integration of image and text inputs into shared feature representations. The framework utilizes a modular architecture that decouples model building blocks into interchangeable components, allowing for flexible configuration of vision and language modules. It includes a benchmark suite for executing reference models against standar
Dolphin is a multimodal layout analyzer and image-to-structure converter that transforms photographed or digital document images into machine-readable structured data. It functions as an LLM document parser, utilizing vision-language models to simultaneously predict spatial layout and text content. The system is designed as a concurrent document processor, employing parallel document parsing to process multiple elements across distributed compute nodes. This high-throughput approach reduces the total time required to convert large volumes of images into structured formats. The project covers
DeepSeek-OCR is a vision processing framework designed to convert image-based text into machine-readable tokens for large language models. It functions as a document inference pipeline that encodes visual data into compact representations, enabling automated optical character recognition and document analysis workflows. The system distinguishes itself through a high-throughput architecture that utilizes hardware-accelerated batch inference to process large volumes of visual data. It incorporates dynamic resolution scaling to manage the balance between visual detail and token consumption, ensu
This repo contains code to run models from our paper Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks.
The main features of allenai/unified-io-inference are: Multimodal Models.
Projects with overlapping indexed features include: facebookresearch/pythia — Pythia is a multimodal research framework and distributed training system designed for building, training, and… paddlepaddle/ernie — ERNIE is a development toolkit for training, fine-tuning, and deploying large language models built on the… bytedance/dolphin — Dolphin is a multimodal layout analyzer and image-to-structure converter that transforms photographed or digital… facebookresearch/imagebind — ImageBind is a multi-modal embedding model and joint representation learner that maps images, text, audio, and other… deepseek-ai/deepseek-ocr — DeepSeek-OCR is a vision processing framework designed to convert image-based text into machine-readable tokens for… bradyfu/awesome-multimodal-large-language-models — :sparkles::sparkles:Latest Advances on Multimodal Large Language Models.