1 repository
Awesome GitHub RepositoriesResult Storage Integrations
Pushes evaluation results to the Hugging Face Hub, S3, or local directories for sharing and persistence.
Distinct from Hugging Face: Distinct from Hugging Face: focuses on storing evaluation results on the Hub, not model conversion or authentication.
Explore 1 awesome GitHub repository matching devops & infrastructure · Result Storage Integrations. Refine with filters or upvote what's useful.
Awesome Result Storage Integrations GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- huggingface/lighteval
huggingface/lighteval
2,453Vezi pe GitHubLighteval is an open-source framework for running standardized benchmarks and custom evaluation tasks against language models. It provides a system for defining new evaluation tasks with custom prompts, metrics, and scoring in YAML configuration files, and integrates with the Hugging Face Hub for storing and comparing results. The framework supports evaluating models across multiple inference backends, including transformers, vllm, and custom APIs, through a unified generation and log-probability interface. It includes a pluggable metric registry for built-in and custom scoring, a prediction
Stores evaluation results on the Hugging Face Hub for sharing and comparing model performance.
Pythonevaluationevaluation-frameworkevaluation-metrics
