databrickslabs/dolly
10,795 stele·1,139 fork-uri·Python·Apache-2.0·10 vizualizăriwww.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html↗
Dolly
Dolly is an instruction-tuned large language model designed to follow complex natural language directions. It operates as a causal language model that predicts the next token in a sequence to generate coherent conversational responses and perform tasks such as brainstorming, classification, and question answering.
The project focuses on the development of models using open datasets suitable for commercial application. It enables the creation of instruction-following models by utilizing curated collections of human-generated instruction-response pairs.
The repository provides capabilities for language model fine-tuning and dataset management. This includes the ability to load instruction datasets and adapt base causal language models through supervised fine-tuning to improve their natural language instruction execution and response generation.
Features
- Model Adaptation Workflows - Adapting a base model using a custom instruction-response corpus to specialize it for particular natural language tasks.
- Instruction Tuning - Fine-tunes pre-trained causal language models using human-generated instruction-response pairs to follow complex directions.
- Instruction-Following Models - Provides a model designed to execute specific tasks such as brainstorming and question answering based on user prompts.
- Instruction-Response Mappings - Maps natural language inputs to structured outputs by learning the relationship between instructions and desired answers.
- Language Model Fine-Tuning - Enables the fine-tuning of base causal language models using custom instruction-response corpora.
- Instruction-Tuned Language Models - Ships an instruction-tuned large language model optimized for following complex natural language directions.
- Supervised Fine-Tuning - Uses supervised fine-tuning on labeled instruction datasets to align model outputs with human preferences.
- Causal Language Modeling - Implements a causal language model architecture that predicts the next token in a sequence to generate coherent text.
- Transformer Architectures - Utilizes a transformer-based architecture with self-attention mechanisms to process and generate sequences of text.
- Commercial LLM Development - Develops language models for business applications using datasets that are safe for commercial use.
- Commercially Viable LLM Datasets - Trained on curated collections of instruction datasets suitable for commercial application use.
- Natural Language Generation - Generates coherent natural language text responses to execute instructions, including question answering.
- Natural Language Processing - Processes diverse natural language prompts to execute complex tasks like classification and brainstorming.
- Natural Language - Automates complex text-based tasks including brainstorming, classification, and question answering.
- Foundation Models - Instruction-tuned language models trained on proprietary machine learning platforms.
- General Purpose Models - Instruction-tuned model designed for open commercial use.
- Language Models - An instruction-following model licensed for commercial use.
- Large Language Models - Instruction-tuned language models for enterprise use.
- Open Source Models - Offers an instruction-tuned model for enterprise platforms.
- Large Language Models (LLMs) - Listed in the “Large Language Models (LLMs)” section of the The Incredible Pytorch awesome list.
