This repository provides an overview of all components used for the creation of BLOOMZ & mT0 and xP3 introduced in the paper Crosslingual Generalization through Multitask Finetuning. Link to 25min video on the paper by Samuel Albanie; Link to 4min video on the paper by Niklas Muennighoff.
Metaseq is a transformer sequence modeling toolkit designed for training, fine-tuning, and deploying sequence-to-sequence models using open pre-trained weights. It provides a comprehensive framework for large language model training, including dedicated tools for sequence dataset processing and a standalone inference server for generating text via API requests. The project features specialized utilities for model quantization to reduce parameter precision to eight bits, which lowers memory usage and increases inference speed. It also includes a checkpoint conversion pipeline to transform mode
Original Flan (2021) | The Flan Collection (2022) | Flan 2021 Citation | License
Toolkit for creating, sharing and using natural language prompts.
This repository provides an overview of all components from the paper OctoPack: Instruction Tuning Code Large Language Models. Link to 5-min video on the paper presented by Niklas Muennighoff.
The main features of bigcode-project/octopack are: Instruction Tuning.
Open-source alternatives to bigcode-project/octopack include: bigscience-workshop/xmtf — This repository provides an overview of all components used for the creation of BLOOMZ & mT0 and xP3 introduced in the… facebookresearch/metaseq — Metaseq is a transformer sequence modeling toolkit designed for training, fine-tuning, and deploying… google-research/flan — Original Flan (2021) | The Flan Collection (2022) | Flan 2021 Citation | License. google-research/text-to-text-transfer-transformer — This is a machine learning framework for treating diverse natural language processing tasks as a unified text-to-text… hkust-nlp/deita — 🤗 HF Repo 📄 Paper 📚 6K Data 📚 10K Data. bigscience-workshop/promptsource — Toolkit for creating, sharing and using natural language prompts.