7 dépôts
Awesome GitHub RepositoriesEvaluation Benchmarks
Tools and frameworks for assessing the performance and safety of language models.
Explore 7 awesome GitHub repositories matching part of an awesome list · Evaluation Benchmarks. Refine with filters or upvote what's useful.
Awesome Evaluation Benchmarks GitHub Repositories
Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
- shishirpatil/gorilla
ShishirPatil/gorilla
12,908Voir sur GitHubGorilla is a foundational infrastructure framework for large language model function calling. It provides a system for training, evaluating, and executing the translation of natural language instructions into accurate API calls and executable code. The project integrates a structured API documentation index, a fine-tuning pipeline for model adaptation, and a secure sandboxed action runtime for executing model-generated commands. The framework distinguishes itself through a specialized evaluation benchmark suite that measures the accuracy, cost, and latency of function calls. It includes tools
Evaluates the ability of models to generate correct function calls within real-world multi-turn agentic workflows.
Python - open-compass/opencompass
open-compass/opencompass
6,678Voir sur GitHubOpenCompass is an open-source framework for standardized benchmarking of large language models. It provides a configurable evaluation pipeline that supports both objective and subjective assessment, using a dual-engine architecture to handle closed-form answer comparison and open-ended response rating. The framework is designed as a modular platform where datasets, models, and metrics are composed through declarative YAML configuration files. The framework distinguishes itself through its extensible model integration layer, which supports custom models, HuggingFace models, and third-party API
Comprehensive evaluation platform for language and multimodal models.
Pythonbenchmarkchatgptevaluation - sjtu-lit/ceval
SJTU-LIT/ceval
1,854Voir sur GitHubOfficial github repo for C-Eval, a Chinese evaluation suite for foundation models NeurIPS 2023
Comprehensive benchmark suite for evaluating Chinese language models.
Python - thu-coai/safety-prompts
thu-coai/Safety-Prompts
1,176Voir sur GitHubChinese safety prompts for evaluating and improving the safety of LLMs. 中文安全prompts，用于评估和提升大模型的安全性。
Framework for evaluating safety and ethical alignment in models.
attack-defensechatgptchinese-language - cluebenchmark/supercluelyb
CLUEbenchmark/SuperCLUElyb
SuperCLUE琅琊榜：中文通用大模型匿名对战评价基准
Crowdsourced leaderboard for evaluating conversational model performance.
- mikegu721/xiezhibenchmark
MikeGu721/XiezhiBenchmark
Xiezhi (獬豸) is a comprehensive evaluation suite for Language Models (LMs). It consists of 249587 multi-choice questions spanning 516 diverse disciplines and four difficulty levels, as shown below. Please check our paper for more details, and our website will be open later on.
Academic benchmark for evaluating models across diverse disciplines.
Python - flagopen/flageval
FlagOpen/FlagEval
FlagEval, launched by BAAI in 2023, is a comprehensive large model evaluation system that encompasses over 800 open-source and closed-source models from around the globe. It features more than 40 capability dimensions, including reasoning, mathematical skills, and task-solving abilities, along…
Multi-dimensional evaluation toolkit for foundational AI models.
