1 个仓库
Awesome GitHub RepositoriesModel Interface Protocols
Uniform communication protocols ensuring different language models can be swapped and tested interchangeably.
Distinct from Model Testing: Focuses on model interface standardization for testing rather than general model evaluation methods.
Explore 1 awesome GitHub repository matching testing & quality assurance · Model Interface Protocols. Refine with filters or upvote what's useful.
Awesome Model Interface Protocols GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- openai/evals
openai/evals
18,702在 GitHub 上查看Evals is a framework designed for automating, managing, and executing repeatable benchmarking suites to analyze the quality and performance of language models. It provides a platform for running standardized tests to measure model accuracy and track behavioral changes over time. The system distinguishes itself through a modular architecture that uses a standardized adapter layer to normalize inputs and outputs, allowing different models to be swapped and tested interchangeably. It supports the creation of custom benchmarks using proprietary data, enabling quality assurance on sensitive tasks
Implements a uniform communication protocol to ensure different language models can be swapped and tested interchangeably.
Python
