How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
NeurIPS 2024 D&B GTA: A Benchmark for General Tool Agents & arXiv 2026 GTA-2
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
MCPMark is a comprehensive, stress-testing MCP benchmark designed to evaluate model and agent capabilities in real-world MCP use.
The main features of eval-sys/mcpmark are: Tool and API Benchmarks.
Projects with overlapping indexed features include: modelscope/mcpbench — The evaluation benchmark on MCP servers. open-compass/gta — [NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2. thudm/agentbench — A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24).