karpathy/autoresearch

Features

• Autonomous Research Agents - Functions as an autonomous agent that iteratively rewrites source code and evaluates metrics to improve model performance.
• Automated Machine Learning - Provides an autonomous framework for architecture search and hyperparameter optimization using AI agents.
• Coding Agents - Employs AI agents to programmatically read and edit training source code to optimize model configurations.
• Model Evaluation Metrics - Uses validation bits per byte as a specialized metric to compare architectural changes independently of vocabulary size.
• Large Language Model Optimization - Uses AI agents to iteratively modify training code and architectures to optimize the performance of language models.
• Training and Evaluation Pipelines - Provides automated workflows for executing model training, iteration, and validation on a single GPU.
• Automated Code Refinement Loops - Iteratively modifies training code and evaluates metrics to automatically improve model performance.
• Model Performance Optimization - Adjusts training code and model architecture to minimize validation bits per byte and maximize performance.
• Model Training Optimizers - Automatically modifies training code and evaluates results to optimize model accuracy and convergence.
• Neural Architecture Search - Implements an automated process to test and discover optimal model configurations to minimize validation loss.
• Performance Metrics - Calculates and tracks performance indicators like validation bits per byte to measure the impact of architectural changes.
• Architecture Search Frameworks - Provides a framework that uses AI agents to iteratively modify training code and architectures to optimize performance.
• Iterative Feedback Loops - Implements an automated system that refines model code through iterative cycles of modification and performance evaluation.
• Automated Architecture Search - Programmatically rewrites training and architecture source code to autonomously discover optimal model configurations.
• Close Buttons - Iteratively modifies training code and evaluates metrics to automatically improve model performance based on target objectives.
• Experiment Tracking - Logs performance metrics and memory usage across all trials to maintain a detailed history of the research process.
• Model Optimization - Improves language model quality by tracking validation bits per byte across different architectural iterations.
• Training Efficiency - Evaluates training efficiency using vocabulary-size-independent metrics to compare architectural configurations fairly.
• Execution Time Limits - Limits each experimental training run to a specific time window to evaluate efficiency across different configurations.
• Version Control and Management - Manages experimental iterations using version control branches to isolate changes and track improvements.
• Branch-Based Isolation - Utilizes Git branches to isolate experimental architectural changes and track successful model training iterations.
• Training Workflow Orchestrators - Manages the execution, versioning, and monitoring of small language model experiments on a single GPU.
• Agent Frameworks - Self-improving agent framework for hypothesis generation and experimentation.
• Research Automation - Automated research exploration tool.

Star history