1 repository
Awesome GitHub RepositoriesAPI Generation Benchmarks
Standardized datasets and metrics for evaluating the correctness of API call generation in agentic workflows.
Distinct from Evaluation Benchmarks: Specifically evaluates the syntax and logic of generated API calls rather than general model safety or reasoning.
Explore 1 awesome GitHub repository matching part of an awesome list · API Generation Benchmarks. Refine with filters or upvote what's useful.
Awesome API Generation Benchmarks GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- shishirpatil/gorilla
ShishirPatil/gorilla
12,908Vezi pe GitHubGorilla is a foundational infrastructure framework for large language model function calling. It provides a system for training, evaluating, and executing the translation of natural language instructions into accurate API calls and executable code. The project integrates a structured API documentation index, a fine-tuning pipeline for model adaptation, and a secure sandboxed action runtime for executing model-generated commands. The framework distinguishes itself through a specialized evaluation benchmark suite that measures the accuracy, cost, and latency of function calls. It includes tools
Evaluates the ability of models to generate correct function calls within real-world multi-turn agentic workflows.
Python
