# fchollet/arc-agi

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/fchollet-arc-agi).**

_How this analysis was created: the description and tags below were written by an AI model that read this project's README and public documentation pages; stars, license and language come straight from the GitHub API. The model does not read the source code._

4,787 stars · 716 forks · JavaScript · Apache-2.0

## Links

- GitHub: https://github.com/fchollet/ARC-AGI
- awesome-repositories: https://awesome-repositories.com/repository/fchollet-arc-agi.md

## Topics

`artificial-intelligence` `intelligence-testing` `program-synthesis` `psychometrics`

## Description

This project is a standardized set of abstraction and reasoning problems designed for benchmarking the ability of artificial intelligence models to learn new rules. It functions as a fluid intelligence test and a reasoning benchmark, utilizing a collection of grid-based puzzles and a program synthesis dataset to evaluate how agents generate algorithms from examples.

The project focuses on measuring general fluid intelligence and the capacity for zero-shot generalization, testing whether a system can apply learned logic to unseen problems without relying on task-specific training. It provides a framework for abstract reasoning research, specifically evaluating program synthesis and the ability of models to discover latent patterns for novel grid transformations.

The system incorporates a grid-based domain representation and a combinatorial search space of geometric and topological operations. It includes a human-in-the-loop interface that allows for the manual construction of output grids to define ground truth for validation and benchmarking.

## Tags

### Artificial Intelligence & ML

- [Fluid Intelligence Benchmarks](https://awesome-repositories.com/f/artificial-intelligence-ml/fluid-intelligence-benchmarks.md) — Serves as a standardized benchmark to measure general fluid intelligence and program synthesis in AI models. ([source](https://github.com/fchollet/arc-agi#readme))
- [Geometric Knowledge Priors](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-networks/structural-priors/geometric-knowledge-priors.md) — Utilizes a core set of innate geometric biases such as symmetry and containment to guide the program search.
- [Example-Based Synthesis](https://awesome-repositories.com/f/artificial-intelligence-ml/example-based-synthesis.md) — Derives general transformation rules by learning patterns from a small set of input-output examples.
- [Fluid Intelligence Tests](https://awesome-repositories.com/f/artificial-intelligence-ml/fluid-intelligence-tests.md) — Features a series of logic puzzles requiring the discovery of latent patterns to solve grid transformations.
- [LLM Benchmarking](https://awesome-repositories.com/f/artificial-intelligence-ml/large-language-models/llm-benchmarking.md) — Provides a dataset of grid-based tasks to measure general fluid intelligence and reasoning in LLMs.
- [AI Evaluation Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-evaluation-analysis/ai-evaluation-frameworks.md) — Provides a standardized set of abstraction and reasoning problems to assess the quality of AI reasoning.
- [Problem Domain Representations](https://awesome-repositories.com/f/artificial-intelligence-ml/problem-domain-representations.md) — Represents reasoning tasks as multi-dimensional arrays of integers mapping to colors and spatial coordinates.
- [Zero-Shot Generalization](https://awesome-repositories.com/f/artificial-intelligence-ml/zero-shot-generalization-tuning/zero-shot-generalization.md) — Tests the ability of AI systems to solve novel reasoning tasks without training on the specific problem distribution.
- [Human-in-the-Loop Interfaces](https://awesome-repositories.com/f/artificial-intelligence-ml/human-in-the-loop-interfaces.md) — Provides an interface for manual grid manipulation to define ground truth and validate reasoning tasks.

### Part of an Awesome List

- [Abstract Reasoning Research](https://awesome-repositories.com/f/awesome-lists/ai/abstract-reasoning-research.md) — Provides a framework for researching how machines perform conceptual abstraction and general reasoning.
- [Program Synthesis](https://awesome-repositories.com/f/awesome-lists/ai/program-synthesis.md) — Tests the capacity of models to generate executable code that solves abstract logic problems from few examples.
- [Datasets](https://awesome-repositories.com/f/awesome-lists/ai/program-synthesis/datasets.md) — Includes a collection of visual reasoning puzzles designed to test an agent's capacity to generate algorithms.
- [Standardized Benchmarks](https://awesome-repositories.com/f/awesome-lists/learning/evaluation-benchmarks/standardized-benchmarks.md) — Uses a standardized set of grid-based puzzles to benchmark an AI system's fluid intelligence.

### Testing & Quality Assurance

- [Agent Generalization Testing](https://awesome-repositories.com/f/testing-quality-assurance/agent-generalization-testing.md) — Evaluates whether AI can apply learned logic to unseen problems without task-specific training.

### Education & Learning Resources

- [Combinatorial Search Spaces](https://awesome-repositories.com/f/education-learning-resources/search-algorithms/search-space-definition/combinatorial-search-spaces.md) — Implements a search space of primitive geometric and topological operations to discover valid transformation programs.