rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline that runs the same agent code for both evaluation and training, automatically capturing traces for gradient computation. The framework supports distributed reinforcement learning across multiple GPUs and nodes using pluggable backends, and executes agents in isolated sandboxes—either locally or in the cloud—for safe and scalable rollout collection. It trains agents built with LangGraph, SmolAgents, OpenAI Agents SDK, or custom frameworks without requiring core logic changes. T
Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024
BioCoder is a challenging bioinformatics code generation benchmark for examining the capabilities of state-of-the-art large language models (LLMs).
FullStack Bench: Evaluating LLMs as Full Stack Coders
This is the repository for Measuring Coding Challenge Competence With APPS by Dan Hendrycks\, Steven Basart\, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt.
Principalele funcționalități ale hendrycks/apps sunt: Benchmarks and Datasets.
Alternativele open-source pentru hendrycks/apps includ: rllm-org/rllm — rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline… evalplus/evalplus — Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024. gersteinlab/biocoder — BioCoder is a challenging bioinformatics code generation benchmark for examining the capabilities of state-of-the-art… google-research/google-research — This repository serves as a comprehensive research platform and toolkit for advancing machine learning, quantum… leolty/repobench — ✨ RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems - ICLR 2024. bytedance/fullstackbench — FullStack Bench: Evaluating LLMs as Full Stack Coders.