# stanford-crfm/helm

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/stanford-crfm-helm).**

2,828 stars · 397 forks · Python · Apache-2.0

## Links

- GitHub: https://github.com/stanford-crfm/helm
- Homepage: https://crfm.stanford.edu/helm
- awesome-repositories: https://awesome-repositories.com/repository/stanford-crfm-helm.md

## Description

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

## Tags

### Part of an Awesome List

- [Evaluation Frameworks](https://awesome-repositories.com/f/awesome-lists/ai/evaluation-frameworks.md) — Holistic framework for increasing transparency in model evaluation.
- [Model Evaluation and Benchmarking](https://awesome-repositories.com/f/awesome-lists/ai/model-evaluation-and-benchmarking.md) — Holistic evaluation suite for language models and data.