1 रिपॉजिटरी
Frameworks for evaluating the ability of AI models to resolve software bugs by generating functional code patches.
Distinct from Geometric Problem Solving: Candidates are focused on human interview techniques or specific mathematical algorithms, not AI-driven software engineering evaluation.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Code-Based Problem Solving Assessments. Refine with filters or upvote what's useful.
SWE-bench is a software engineering benchmark and evaluation framework designed to measure the ability of large language models to resolve real-world GitHub issues. It provides datasets and evaluation suites to verify whether model-generated code patches correctly fix software bugs. The project includes a multimodal benchmark for testing visual language models on issues involving graphical interfaces. It utilizes a collection of pre-processed repository issues and gold-standard patches to train and test AI coding agents. The framework provides infrastructure for containerized patch verificat
A process to assess how language models resolve software issues by generating code patches based on a provided codebase.