Open Llama is an open source large language model and pre-trained transformer designed as a permissively licensed alternative to proprietary weights. It serves as a base model reproduction of the Llama architecture, providing a set of weights for a decoder-only transformer.
The project provides a transparently trained model based on the RedPajama dataset, supporting unrestricted commercial and research use. It includes systems for serving pre-trained weights in various sizes.
The project covers natural language processing research and performance benchmarking through text quality evaluation harnesses. It is designed for integration with deep learning frameworks and research pipelines.