Open-source alternatives to Otter

Official code for VisProg (CVPR 2023 Best Paper!)

An implementation for ACL 2023 paper Learning In-context Learning for Named Entity Recognition

Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料

Emu Series: Generative Multimodal Models from BAAI

This is the github repository for the paper to be appeared at NAACL 2024 main conference: Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models.

MNBVC is a dataset pipeline and toolkit designed for the collection, cleaning, and normalization of massive text and code corpora used to train large language models. It provides specialized tools for harvesting source code, commit histories, and repository metadata from version control platforms, alongside a multilingual text corpus collector for gathering parallel text and academic papers. The project distinguishes itself through comprehensive capabilities for processing diverse document types, including a PDF-to-text converter that transforms complex layouts and formulas into structured JS

SciGraphQA: Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs

ICLR'24 Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

A VQA dataset that includes unanswerable questions TPAMI 2024.

MMICL, a state-of-the-art VLM with the in context learning ability from ICL, PKU

Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models

NeurIPS 2025 The official repository of "Inst-IT: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning"

Official repo for StableLLAVA

Zhao Zhang 3   Ziwei Liu &#x2709,1 1 S-Lab, Nanyang Technological University  2 Shanghai Jiao Tong University  3 SenseTime Research  4 Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China   Equal Contribution  † Project Lead …

Please save your dataset in data folder. Note that CoNLL2003 and WNUT2017 are open-source datasets, ACE2004 and ACE2005 are not free. We keep our CoNLL2003 and WNUT2017 train and test JSON files in data folder.

COYO-700M: Large-scale Image-Text Pair Dataset

Code, data, and results described in the paper "Mining experimental data from materials science literature with large language models: an evaluation study", https://www.tandfonline.com/doi/full/10.1080/27660400.2024.2356506

Codes for "Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models".

Data and code for NeurIPS 2022 Paper "Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering".

Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration

Code for ICCV 2023 Paper : “ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction”

This is the implementation of filter-then-rerank pipeline in Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!. EMNLP'2023 (Findings).

Updates | Datasets | Models | Environment | Running | Results | Website | Paper

JARVIS is a system for large language model task orchestration, deployment management, and automation benchmarking. It utilizes a task orchestrator to decompose complex requests into actionable steps and coordinates various expert models to synthesize final responses. The project includes an AI model deployment manager to handle the local deployment of expert models across different hardware scales. It further provides an AI workflow API consisting of web endpoints used to trigger automated task workflows and retrieve results from model selection stages. The framework incorporates an automat

Large Language-and-Vision Assistant for Biomedicine, built towards multimodal GPT-4 level capabilities.

Official repo for MM-REACT

This is the official repository for "CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors" (ACL 2023).