rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline that runs the same agent code for both evaluation and training, automatically capturing traces for gradient computation. The framework supports distributed reinforcement learning across multiple GPUs and nodes using pluggable backends, and executes agents in isolated sandboxes—either locally or in the cloud—for safe and scalable rollout collection. It trains agents built with LangGraph, SmolAgents, OpenAI Agents SDK, or custom frameworks without requiring core logic changes. T
Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024
BioCoder is a challenging bioinformatics code generation benchmark for examining the capabilities of state-of-the-art large language models (LLMs).
FullStack Bench: Evaluating LLMs as Full Stack Coders
MultiPL-E is a system for translating unit test-driven neural code generation benchmarks to new languages. We have used MultiPL-E to translate two popular Python benchmarks (HumanEval and MBPP) to 18 other programming languages.
Die Hauptfunktionen von nuprl/multipl-e sind: Benchmarks and Datasets.
Open-Source-Alternativen zu nuprl/multipl-e sind unter anderem: rllm-org/rllm — rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline… evalplus/evalplus — Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024. gersteinlab/biocoder — BioCoder is a challenging bioinformatics code generation benchmark for examining the capabilities of state-of-the-art… google-research/google-research — This repository serves as a comprehensive research platform and toolkit for advancing machine learning, quantum… hendrycks/apps — This is the repository for Measuring Coding Challenge Competence With APPS by Dan Hendrycks\, Steven Basart\, Saurav… bytedance/fullstackbench — FullStack Bench: Evaluating LLMs as Full Stack Coders.