awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
karpathy avatar

karpathy/makemore

0
View on GitHub↗
3,672 stars·917 forks·Python·mit·17 views

Makemore

makemore is a character-level language model and text generation engine. It serves as an educational implementation of the architecture and training processes used in large language models, built as a neural network using the PyTorch framework.

The system demonstrates sequence prediction by learning the probability distributions of characters within a dataset to generate novel text strings. It implements this through a progression of techniques, including n-gram probability modeling and the use of automatic differentiation for weight optimization.

The project covers the full machine learning lifecycle for sequence prediction, from processing text corpora during model training to producing output via probabilistic sampling. This includes the application of softmax normalization and multinomial sampling to convert model outputs into predictable character sequences.

Features

  • Character-Level Models - Trains a model to predict the next character in a sequence to generate text mimicking a specific dataset.
  • Generative Model Sampling - Generates novel text by sampling from the learned probability distributions of a trained model.
  • N-Gram Co-occurrence Models - Implements n-gram modeling to calculate the likelihood of subsequent characters based on preceding sequences.
  • Gradient Descent Algorithms - Optimizes model weights via gradient descent and automatic differentiation to minimize cross-entropy loss.
  • Neural Network Implementations - Builds a character-level neural network using the PyTorch framework for sequence prediction.
  • Autoregressive Text Generation - Creates new character sequences by autoregressively predicting subsequent characters based on training patterns.
  • Text Generation Engines - Provides an engine that generates novel character strings based on learned probability distributions.
  • Text Model Training - Processes text corpora to build predictive models by learning character-level probability distributions.
  • LLM Education - Provides a step-by-step educational implementation of the architecture and training processes used in LLMs.
  • Bigram Frequency Tables - Implements a lookup table to track character pairs and determine the statistical probability of transitions.
  • Multinomial Samplers - Generates text by drawing characters from a probability distribution using multinomial sampling.
  • Softmax Normalization - Converts raw model outputs into normalized probability distributions using the softmax function.
  • Maximum Likelihood Estimators - Uses maximum likelihood estimation to calculate model parameters by optimizing based on observed training data.

Star history

Star history chart for karpathy/makemoreStar history chart for karpathy/makemore

How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Open-source alternatives to Makemore

Similar open-source projects, ranked by how many features they share with Makemore.
  • hunkim/deeplearningzerotoallhunkim avatar

    hunkim/DeepLearningZeroToAll

    4,494View on GitHub↗

    DeepLearningZeroToAll is a comprehensive educational resource and implementation collection focused on deep learning and machine learning. It provides a structured learning path using TensorFlow to move from foundational linear models to complex neural network architectures. The project is distinguished by its practical implementations of various network types, including multilayer perceptrons for logic problems, convolutional neural networks for spatial data and image recognition, and recurrent neural networks using LSTM cells for time-series forecasting and character sequence prediction. It

    Jupyter Notebookkeraslabmxnet
    View on GitHub↗4,494
  • d2l-ai/d2l-end2l-ai avatar

    d2l-ai/d2l-en

    29,001View on GitHub↗

    This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

    Pythonbookcomputer-visiondata-science
    View on GitHub↗29,001
  • spro/practical-pytorchspro avatar

    spro/practical-pytorch

    4,546View on GitHub↗

    Practical PyTorch is a collection of deep learning tutorials and guides focused on implementing recurrent neural networks. The project provides practical code for building sequence models and sequence-to-sequence architectures using the PyTorch framework. The repository covers the implementation of models for neural machine translation, character-level text generation, and text classification. It includes examples for transforming input sequences into output sequences for machine translation and synthesizing new text. The project also extends to sequence data prediction and time series analy

    Jupyter Notebook
    View on GitHub↗4,546
  • karpathy/char-rnnkarpathy avatar

    karpathy/char-rnn

    12,066View on GitHub↗

    This project is a character-level language modeling system that uses recurrent neural networks to predict and generate text one character at a time. It implements LSTM and GRU architectures to learn sequential patterns and probability distributions from text corpora. The system includes mechanisms for text generation sampling, allowing users to produce new sequences from trained models. It features temperature-based stochasticity to control the randomness and diversity of the generated output. The implementation covers the full model lifecycle, including training, state persistence through c

    Lua
    View on GitHub↗12,066
See all 30 alternatives to Makemore→

Frequently asked questions

What does karpathy/makemore do?

makemore is a character-level language model and text generation engine. It serves as an educational implementation of the architecture and training processes used in large language models, built as a neural network using the PyTorch framework.

What are the main features of karpathy/makemore?

The main features of karpathy/makemore are: Character-Level Models, Generative Model Sampling, N-Gram Co-occurrence Models, Gradient Descent Algorithms, Neural Network Implementations, Autoregressive Text Generation, Text Generation Engines, Text Model Training.

What are some open-source alternatives to karpathy/makemore?

Open-source alternatives to karpathy/makemore include: hunkim/deeplearningzerotoall — DeepLearningZeroToAll is a comprehensive educational resource and implementation collection focused on deep learning… d2l-ai/d2l-en — This project is an educational platform and research toolkit designed to teach deep learning through a combination of… spro/practical-pytorch — Practical PyTorch is a collection of deep learning tutorials and guides focused on implementing recurrent neural… karpathy/char-rnn — This project is a character-level language modeling system that uses recurrent neural networks to predict and generate… nndl/llm-beginner — This project is a collection of educational resources and technical guides focused on the development and… cs231n/cs231n.github.io — This project is a static educational website and comprehensive curriculum focused on computer vision and deep…