How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
OpenCompass is an open-source framework for standardized benchmarking of large language models. It provides a configurable evaluation pipeline that supports both objective and subjective assessment, using a dual-engine architecture to handle closed-form answer comparison and open-ended response rating. The framework is designed as a modular platform where datasets, models, and metrics are composed through declarative YAML configuration files. The framework distinguishes itself through its extensible model integration layer, which supports custom models, HuggingFace models, and third-party API
Xiezhi (獬豸) is a comprehensive evaluation suite for Language Models (LMs). It consists of 249587 multi-choice questions spanning 516 diverse disciplines and four difficulty levels, as shown below. Please check our paper for more details, and our website will be open later on.
Official github repo for C-Eval, a Chinese evaluation suite for foundation models NeurIPS 2023
FlagEval, launched by BAAI in 2023, is a comprehensive large model evaluation system that encompasses over 800 open-source and closed-source models from around the globe. It features more than 40 capability dimensions, including reasoning, mathematical skills, and task-solving abilities, along…
The main features of flagopen/flageval are: Model Evaluation, Evaluation Benchmarks.
Open-source alternatives to flagopen/flageval include: sjtu-lit/ceval — Official github repo for C-Eval, a Chinese evaluation suite for foundation models [NeurIPS 2023]. thu-coai/safety-prompts — Chinese safety prompts for evaluating and improving the safety of LLMs. 中文安全prompts，用于评估和提升大模型的安全性。. mikegu721/xiezhibenchmark — Xiezhi (獬豸) is a comprehensive evaluation suite for Language Models (LMs). It consists of 249587 multi-choice… cluebenchmark/supercluelyb — SuperCLUE琅琊榜：中文通用大模型匿名对战评价基准. open-compass/opencompass — OpenCompass is an open-source framework for standardized benchmarking of large language models. It provides a… nvidia/isaac-gr00t.
