1 Repo
Awesome GitHub RepositoriesModel Interface Protocols
Uniform communication protocols ensuring different language models can be swapped and tested interchangeably.
Distinct from Model Testing: Focuses on model interface standardization for testing rather than general model evaluation methods.
Explore 1 awesome GitHub repository matching testing & quality assurance · Model Interface Protocols. Refine with filters or upvote what's useful.
Awesome Model Interface Protocols GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- openai/evals
openai/evals
18,702Auf GitHub ansehenEvals is a framework designed for automating, managing, and executing repeatable benchmarking suites to analyze the quality and performance of language models. It provides a platform for running standardized tests to measure model accuracy and track behavioral changes over time. The system distinguishes itself through a modular architecture that uses a standardized adapter layer to normalize inputs and outputs, allowing different models to be swapped and tested interchangeably. It supports the creation of custom benchmarks using proprietary data, enabling quality assurance on sensitive tasks
Implements a uniform communication protocol to ensure different language models can be swapped and tested interchangeably.
Python
