Awesome GitHub RepositoriesLanguage Model Architectures

This project is a collection of educational examples and code for implementing deep learning architectures using the PyTorch framework. It serves as a tutorial and implementation guide for building various neural network architectures for machine learning tasks. The project provides practical implementations for computer vision, including image classification and neural style transfer, as well as natural language processing examples for building sequence models and language predictors. It also covers generative models using adversarial and variational networks to synthesize or transform visua

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This repository serves as an educational resource for learning the foundational architectures of natural language processing through concise code implementations. It provides a structured collection of deep learning models designed to process and understand human language, focusing on the core mechanics of neural network sequence modeling and text analysis. The project distinguishes itself by offering direct, hands-on implementations of complex architectures, including Transformers, attention mechanisms, and word embedding generation. By utilizing tensor-based computational graphs and gradien

SentencePiece is a text segmentation engine and tokenization library designed for machine learning workflows. It provides a comprehensive toolkit for transforming raw text into subword units or numerical identifiers, enabling consistent data representation for neural network training and inference. The library supports the training of segmentation models from raw text, allowing for the creation of custom vocabularies tailored to specific domain requirements. The project distinguishes itself through its byte-level encoding and fallback mechanisms, which ensure that every input can be represent

ChatRWKV is an open-source frontend and GPU-accelerated inference engine designed for interacting with RWKV recurrent neural network language models. It provides a self-hosted web chat interface and a specialized client for generating human-like text using a linear-complexity architecture. The project utilizes a GPU-accelerated backend that employs custom CUDA kernels and dynamic model format conversion to increase processing speed and reduce memory overhead. It manages conversation history through state-based context management, updating a fixed-size hidden state to maintain a constant memor

This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene

This is a PyTorch implementation of reinforcement learning from human feedback designed to align large language models with human values and preferences. It provides a framework for the PaLM architecture and incorporates parameter-efficient fine-tuning to adapt models while minimizing the number of updated weights. The system enables the development of reward models that act as scoring mechanisms built from human preference data. These models evaluate generative outputs to guide the alignment process. The workflow covers policy optimization using a clipped objective, reward modeling based on

This project is a collection of TensorFlow 2.x machine learning tutorials and practical code examples. It serves as a deep learning implementation guide for constructing diverse neural network architectures, including convolutional, recurrent, and generative networks. The repository provides templates and examples for several specialized domains, including computer vision for image classification and object detection, natural language processing for text generation and language understanding, and generative AI for synthesizing data using adversarial networks and autoencoders. It also includes

Lit-llama es un framework de implementación basado en PyTorch para el modelo de lenguaje LLaMA, que proporciona un sistema para pre-entrenamiento, ajuste fino e inferencia de alto rendimiento. Incluye un pipeline de pre-entrenamiento para crear modelos de lenguaje fundamentales desde cero y herramientas para ejecutar pesos pre-entrenados para generar texto natural y predecir secuencias. El proyecto proporciona toolkits especializados para el ajuste fino eficiente en parámetros utilizando adaptación de bajo rango (LoRA) y adaptadores ligeros. También incluye una librería de cuantización que reduce las huellas de memoria del modelo a través de precisión de cuatro y ocho bits para permitir la ejecución en hardware con recursos limitados. El framework incorpora un diseño de transformador simplificado y emplea flash attention para optimizar la memoria y la velocidad. Además, gestiona datasets a gran escala a través de formatos de datos en streaming para evitar cargar corpora completos en la memoria del sistema.

AlphaGeometry es un sistema de IA neuro-simbólico diseñado como un solucionador de geometría automatizado y demostrador de teoremas. Integra un modelo de lenguaje de gran tamaño con un motor de prueba formal para derivar conclusiones geométricas a partir de premisas dadas y generar pruebas formales paso a paso. El sistema se distingue por combinar un motor de deducción simbólica con construcciones auxiliares impulsadas por redes neuronales. Esta arquitectura utiliza un modelo de lenguaje para proponer puntos o líneas adicionales para cerrar brechas en una prueba, mientras que una base de datos deductiva y un sistema de razonamiento algebraico verifican la corrección matemática de cada paso frente a axiomas estrictos. El proyecto cubre la verificación matemática formal y el razonamiento geométrico simbólico a través de una búsqueda de pruebas iterativa y un bucle híbrido simbólico-neuronal.

This project is an LLM research orchestrator and autonomous AI agent framework designed to automate the scientific lifecycle. It functions as an end-to-end research pipeline and model training toolkit, managing everything from initial literature reviews and hypothesis testing to the final drafting of academic papers. The system is distinguished by its ability to convert unstructured academic PDFs into machine-executable knowledge layers, allowing agents to reproduce and extend research findings. It employs a two-loop orchestration architecture and a specialized research engineering skill libr

This is a structured deep learning curriculum for programmers, delivered as a collection of Jupyter notebooks. It teaches the fundamentals of training neural networks for computer vision, natural language processing, tabular data analysis, and collaborative filtering using PyTorch and the fastai library. The course is designed to be hands-on, guiding learners from building a training loop from scratch to fine-tuning pretrained models for a variety of practical tasks. The curriculum distinguishes itself by covering the full lifecycle of a deep learning project, from data preparation and augmen

This project is a technical resource and implementation guide for building transformer-based language model architectures and training pipelines from scratch. It focuses on the design of models capable of natural language processing, including the integration of pretrained weights and the creation of foundational model frameworks. The project specifically emphasizes logical reasoning and mathematical problem solving. It provides a framework for optimizing these capabilities through reinforcement learning and the use of automated verifiers to evaluate and reward correct reasoning paths. The r

Supertonic is an on-device neural text-to-speech engine that runs entirely locally without cloud dependencies or GPU acceleration. It converts written text into natural-sounding speech across 31 languages with automatic language detection and a fallback model for unsupported locales. The engine provides expressive speech control through inline prosody tags that dynamically adjust pitch, rate, and tone during synthesis. It supports voice cloning from a short reference audio clip by extracting a speaker embedding vector, and offers a selection of pre-built voices tuned for different use cases.