How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
MLAgentBench is a suite of end-to-end Machine Learning (ML) experimentation tasks for benchmarking AI agents, where the agent aims to take a given dataset and a machine learning task description and autonomously develop or improve an ML model. Paper: https://arxiv.org/abs/2310.03302
rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline that runs the same agent code for both evaluation and training, automatically capturing traces for gradient computation. The framework supports distributed reinforcement learning across multiple GPUs and nodes using pluggable backends, and executes agents in isolated sandboxes—either locally or in the cloud—for safe and scalable rollout collection. It trains agents built with LangGraph, SmolAgents, OpenAI Agents SDK, or custom frameworks without requiring core logic changes. T
FARA is a visual computer-use agent model that controls a browser by predicting screen coordinates for clicking, typing, and scrolling, without relying on DOM or accessibility trees. It is designed to automate multi-step web tasks such as searching, form filling, booking, and shopping by reasoning over visual state and decomposing tasks into sequential actions. The model uses a compact 7-billion-parameter decoder-only transformer that can run on consumer GPUs for low-latency on-device inference, or be deployed as a managed endpoint on Azure Foundry for cloud-based inference without local infr
MLGym A New Framework and Benchmark for Advancing AI Research Agents
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
The main features of thudm/agentbench are: Agent Benchmarks, Evaluation And Benchmarks, General Agent Benchmarks.
Open-source alternatives to thudm/agentbench include: snap-stanford/mlagentbench — MLAgentBench is a suite of end-to-end Machine Learning (ML) experimentation tasks for benchmarking AI agents, where… rllm-org/rllm — rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline… microsoft/fara — FARA is a visual computer-use agent model that controls a browser by predicting screen coordinates for clicking,… facebookresearch/mlgym — MLGym A New Framework and Benchmark for Advancing AI Research Agents. masworks/x-mas — [2025/05/23] See our preprint paper in ArXiv. gersteinlab/ml-bench — ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code…