1 Repo
Awesome GitHub RepositoriesAPI Generation Benchmarks
Standardized datasets and metrics for evaluating the correctness of API call generation in agentic workflows.
Distinct from Evaluation Benchmarks: Specifically evaluates the syntax and logic of generated API calls rather than general model safety or reasoning.
Explore 1 awesome GitHub repository matching part of an awesome list · API Generation Benchmarks. Refine with filters or upvote what's useful.
Awesome API Generation Benchmarks GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- shishirpatil/gorilla
ShishirPatil/gorilla
12,908Auf GitHub ansehenGorilla is a foundational infrastructure framework for large language model function calling. It provides a system for training, evaluating, and executing the translation of natural language instructions into accurate API calls and executable code. The project integrates a structured API documentation index, a fine-tuning pipeline for model adaptation, and a secure sandboxed action runtime for executing model-generated commands. The framework distinguishes itself through a specialized evaluation benchmark suite that measures the accuracy, cost, and latency of function calls. It includes tools
Evaluates the ability of models to generate correct function calls within real-world multi-turn agentic workflows.
Python
