EleutherAI/gpt-neox

Features

• Foundation Models - Parallelized autoregressive transformer implementation for GPU clusters.
• Large Language Models - Framework for training large-scale transformer models.
• Model Training Frameworks - Implementation of model-parallel autoregressive transformers on GPUs.
• Pre-trained Language Models - Large-scale open-source autoregressive language model.