This repository provides an overview of all components used for the creation of BLOOMZ & mT0 and xP3 introduced in the paper Crosslingual Generalization through Multitask Finetuning. Link to 25min video on the paper by Samuel Albanie; Link to 4min video on the paper by Niklas Muennighoff.
The main features of bigscience-workshop/xmtf are: Instruction Tuning, Open Source Models.
Open-source alternatives to bigscience-workshop/xmtf include: facebookresearch/metaseq — Metaseq is a transformer sequence modeling toolkit designed for training, fine-tuning, and deploying… xorbitsai/inference — This project is a platform for the deployment of open source large language and multimodal models. It provides a… openlm-research/open_llama — Open Llama is an open source large language model and pre-trained transformer designed as a permissively licensed… tencentcloudadp/youtu-agent — Youtu Agent is an open-source framework for building, running, and evaluating autonomous agents powered by large… databrickslabs/dolly — Dolly is an instruction-tuned large language model designed to follow complex natural language directions. It operates… blinkdl/chatrwkv — ChatRWKV is an open-source frontend and GPU-accelerated inference engine designed for interacting with RWKV recurrent…
Metaseq is a transformer sequence modeling toolkit designed for training, fine-tuning, and deploying sequence-to-sequence models using open pre-trained weights. It provides a comprehensive framework for large language model training, including dedicated tools for sequence dataset processing and a standalone inference server for generating text via API requests. The project features specialized utilities for model quantization to reduce parameter precision to eight bits, which lowers memory usage and increases inference speed. It also includes a checkpoint conversion pipeline to transform mode
This project is a platform for the deployment of open source large language and multimodal models. It provides a unified interface to serve text, image, and speech models across local or cloud hardware. The system enables distributed AI inference by orchestrating model workloads across multiple nodes and devices. It includes a unified API adapter layer to standardize inputs and outputs, as well as tools for multimodal chat and structural image generation. The platform covers a broad capability surface including request batching for throughput optimization, dynamic model loading, and integrat
Open Llama is an open source large language model and pre-trained transformer designed as a permissively licensed alternative to proprietary weights. It serves as a base model reproduction of the Llama architecture, providing a set of weights for a decoder-only transformer. The project provides a transparently trained model based on the RedPajama dataset, supporting unrestricted commercial and research use. It includes systems for serving pre-trained weights in various sizes. The project covers natural language processing research and performance benchmarking through text quality evaluation
Youtu Agent is an open-source framework for building, running, and evaluating autonomous agents powered by large language models. It provides the core infrastructure for creating agents that follow reasoning loops, use toolkits, and coordinate with other agents to solve complex tasks, all managed through YAML-driven configuration files. The framework distinguishes itself through its support for multi-agent orchestration, where a planner agent decomposes tasks and coordinates specialized worker agents, and through its integration with the Model Context Protocol for connecting to external toolk