Open-source alternatives to Octopack

This is a machine learning framework for treating diverse natural language processing tasks as a unified text-to-text problem. It provides a toolkit for pre-training and fine-tuning large-scale transformer models, utilizing a system where both inputs and outputs are formatted as raw text sequences. The framework is distinguished by its distributed training system, which uses mesh-based strategies to scale model weights and training batches across multiple TPU cores. It supports multi-task learning by combining diverse datasets into a single training stream using configurable mixture rates, al

🤗 HF Repo 📄 Paper 📚 6K Data 📚 10K Data

OpenChat is a framework for the training, fine-tuning, and deployment of large language models optimized for conversational and mathematical reasoning tasks. It provides a comprehensive lifecycle for these models, ranging from training pipelines and deployment stacks to a web-based chat interface. The project focuses on enabling high-performance model execution on consumer-grade hardware without the need for enterprise-grade accelerators. It includes a production-ready inference server that implements the OpenAI chat completion protocol and utilizes dynamic request batching to optimize hardwa

🎩 Models | 📚 Dataset | 🚀 Quick Start | 👀 Demo | 📝 Citation | 🙏 Acknowledgements

Towards Robust Grounded Language Modeling [DEMO](https://huggingface.co/spaces/luohy/SAIL-7B) | [WEB](https://openlsr.org/sail-7b)

New Release We released Adversarial training for both LM pre-training/finetuning and f-divergence.

Official Code Repository for the paper Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-intensive Tasks (NeurIPS 2023).

WizardLM is a large language model and instruction-tuning framework designed to execute sophisticated coding, mathematical, and conversational tasks. It functions as an AI system for mathematical reasoning and code generation, as well as a synthetic dataset generator used to train other language models. The project is distinguished by its evolutionary instruction tuning, which uses a method to rewrite simple instructions into complex tasks. This process expands training dataset difficulty and produces a high volume of open-domain tasks across various difficulty levels. The system covers capa

This repo introduces ExpertLLaMA, a solution to produce high-quality, elaborate, expert-like responses by augmenting vanilla instructions with specialized Expert Identity description. This repo contains: - Brief introduction on the method. - 52k Instruction-Following Expert Data generated by…

This repository contains the Unnatural Instructions dataset. Unnatural Instructions is a dataset of instructions automatically generated by a Large Language model. See full details in the paper: "Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor"

This repository contains the source code for reproducing the data curation of MUFFIN (Multi-faceted Instructions).

This is the repo for the Code Alpaca project, which aims to build and share an instruction-following LLaMA model for code generation. This repo is fully based on Stanford Alpaca ,and only changes the data used for training. Training approach is the same.

UltraChat is a collection of large-scale conversational datasets and instruction-tuning data designed for training and evaluating generative AI models. It provides structured JSON data consisting of complex, multi-round dialogue sequences intended to refine the performance of large language models in chat tasks. The project focuses on improving reasoning and response quality through a diverse set of interactions across multiple sectors. These datasets are used for supervised fine-tuning and instruction tuning workflows to improve how models follow complex directions and maintain context acros

Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements (ACL'24) Chinese Version: [知乎](https://zhuanlan.zhihu.com/p/720237237)

UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition Wenxuan Zhou, Sheng Zhang, Yu Gu, Muhao Chen, Hoifung Poon (*Equal Contribution)

Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts)

Self-instruct is a framework for generating synthetic instruction datasets and fine-tuning large language models to improve their instruction-following capabilities. It provides a pipeline for aligning pretrained models with human intentions through a supervised fine-tuning workflow. The system utilizes a synthetic data generator that uses a seed set of tasks to prompt a model to create new instructional data. It includes an instruction dataset curator to remove redundant or low-quality entries, maintaining dataset diversity through a filtered task pool. The framework covers the full alignme

Rethinking Negative Instances for Generative Named Entity Recognition

Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering