rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline that runs the same agent code for both evaluation and training, automatically capturing traces for gradient computation. The framework supports distributed reinforcement learning across multiple GPUs and nodes using pluggable backends, and executes agents in isolated sandboxes—either locally or in the cloud—for safe and scalable rollout collection. It trains agents built with LangGraph, SmolAgents, OpenAI Agents SDK, or custom frameworks without requiring core logic changes. T
Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024
BioCoder is a challenging bioinformatics code generation benchmark for examining the capabilities of state-of-the-art large language models (LLMs).
FullStack Bench: Evaluating LLMs as Full Stack Coders
MultiPL-E is a system for translating unit test-driven neural code generation benchmarks to new languages. We have used MultiPL-E to translate two popular Python benchmarks (HumanEval and MBPP) to 18 other programming languages.
Las características principales de nuprl/multipl-e son: Benchmarks and Datasets.
Las alternativas de código abierto para nuprl/multipl-e incluyen: rllm-org/rllm — rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline… evalplus/evalplus — Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024. gersteinlab/biocoder — BioCoder is a challenging bioinformatics code generation benchmark for examining the capabilities of state-of-the-art… google-research/google-research — This repository serves as a comprehensive research platform and toolkit for advancing machine learning, quantum… hendrycks/apps — This is the repository for Measuring Coding Challenge Competence With APPS by Dan Hendrycks\, Steven Basart\, Saurav… bytedance/fullstackbench — FullStack Bench: Evaluating LLMs as Full Stack Coders.