1 repo
Awesome GitHub RepositoriesLLM Evaluation Harnesses
Tools for measuring the quality of model outputs using custom metrics and automated judges.
Distinguishing note: Focuses on programmatic evaluation of LLM pipelines, distinct from standard unit testing.
Explore 1 awesome GitHub repository matching testing & quality assurance · LLM Evaluation Harnesses. Refine with filters or upvote what's useful.
Awesome LLM Evaluation Harnesses GitHub Repositories
Find the best repos with AI.We'll search the best matching repositories with AI.
- stanfordnlp/dspy
stanfordnlp/dspy
32,291View on GitHubDSPy is a declarative programming framework designed for building complex language model applications. It treats model interactions as modular, composable programs, allowing developers to define task logic through typed class schemas rather than relying on manually written prompts. By organizing workflows into hierarchical, reusable Python objects, the framework enables the construction of sophisticated AI systems that manage state and execution flow independently. The framework distinguishes itself through an automated optimization engine that iteratively refines prompt instructions and few-
Measures output quality using custom metrics and model-based judges to ensure consistent behavior across pipelines.
Python
