Awesome GitHub RepositoriesDistributed Computing

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This project is a machine learning array framework and tensor computation library designed for high-performance numerical computing. It provides a comprehensive suite of tools for constructing and training neural networks, featuring an automatic differentiation engine that facilitates gradient-based optimization and complex mathematical modeling. The library distinguishes itself through a unified memory architecture that allows data to be shared across CPU and GPU devices without explicit copies, significantly reducing data movement overhead. Its execution model relies on a lazy evaluation en

Meshroom is a node-based photogrammetry software designed to transform collections of two-dimensional images into three-dimensional models and scene geometry. It provides a visual interface for constructing and managing modular data pipelines, allowing users to automate complex computer vision tasks such as feature extraction, depth map estimation, and mesh generation. The software distinguishes itself through a distributed computational framework that dispatches resource-intensive tasks across local hardware or remote render farms. By utilizing a directed acyclic graph execution model, it en

Modin is a distributed dataframe library and parallel data processing engine designed to handle large datasets that exceed system memory. It functions as a distributed computing framework that parallelizes data manipulation tasks across multiple CPU cores or clusters to increase throughput and avoid memory errors. The project mirrors the Pandas API, allowing for the distribution of data workflows without changing core code logic. It utilizes a pluggable backend interface, which enables users to switch between different distributed execution engines to optimize performance based on available h

This project is a structured learning curriculum and technical reference for mastering deep learning with TensorFlow. It provides a comprehensive guide for building, training, and deploying neural networks, combining theoretical fundamentals with practical implementation examples. The repository distinguishes itself by covering the end-to-end machine learning workflow, from low-level tensor mathematics and linear algebra to the creation of complex model architectures. It includes specific guidance on developing data pipelines for diverse data types, such as images, text, and time-series seque

PowerInfer is a high-performance local large language model inference engine and sparse inference framework. It provides a runtime for executing models on consumer-grade hardware, utilizing a GPU acceleration backend to optimize tensor operations for graphics processors. The system distinguishes itself through a sparse inference framework that increases generation speed by skipping computations based on activation sparsity in model weights. It includes a GGUF model converter for transforming weights and metadata into a unified binary format, as well as an OpenAI API compatible server for inte

Featuretools is a Python data science library and automated feature engineering framework designed to create predictive features from multiple related datasets. It automates the data preparation and transformation steps required for machine learning models through deep feature synthesis. The library enables the automatic generation of comprehensive feature tables by applying recursive transformations to relational data. It supports the transformation of unstructured text into structured numeric features and allows users to define custom primitives to extend the synthesis process with specific

Este proyecto es un framework de transformers basado en JAX y un entrenador de modelos de lenguaje a gran escala diseñado para construir y entrenar modelos distribuidos en aceleradores de hardware TPU. Proporciona un sistema para el preentrenamiento y ajuste fino (fine-tuning) de modelos autorregresivos dividiendo pesos y cálculos a través de una malla de dispositivos para reducir la sobrecarga de memoria y aumentar la velocidad de procesamiento. El framework incluye un orquestador de cómputo TPU para aprovisionar recursos y automatizar la instalación de dependencias en nodos distribuidos remotos. También cuenta con un convertidor de pesos de modelos capaz de transformar y redistribuir (resharding) checkpoints entre diferentes configuraciones de hardware y precisiones numéricas. El proyecto cubre capacidades más amplias, incluyendo la gestión de checkpoints fragmentados para almacenamiento en la nube, carga de datos basada en flujos con restauración de estado y generación de texto basada en núcleos para la inferencia de modelos. Además, admite aceleración de hardware compilada con XLA para clusters de TPU y GPU, y proporciona herramientas para benchmarking de rendimiento frente a tareas de lenguaje estandarizadas.

Este proyecto es un framework de alineación y un conjunto de pipelines para entrenar modelos de lenguaje mediante fine-tuning supervisado y optimización de preferencias. Proporciona herramientas para ejecutar entrenamiento distribuido a gran escala a través de múltiples GPU y nodos de cómputo, junto con un sistema para medir la utilidad del modelo y la calidad del diálogo mediante benchmarks de turno único y turnos múltiples. El framework incluye herramientas especializadas para la optimización directa de preferencias (DPO) con el fin de refinar el comportamiento del modelo utilizando datos emparejados sin necesidad de un modelo de recompensa independiente. También admite la alineación mediante IA constitucional y el entrenamiento de modelos de recompensa para clasificar y puntuar respuestas basadas en criterios de preferencia. El proyecto cubre capacidades más amplias para la mezcla y combinación de datasets, fine-tuning eficiente en parámetros mediante adaptación de bajo rango (LoRA) y optimización mediante muestreo de rechazo. Gestiona el ciclo de vida del entrenamiento a través de recetas basadas en configuración y proporciona sistemas para transmitir métricas de rendimiento en tiempo real a paneles de control externos.