Awesome GitHub RepositoriesDataset Transformations

Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f

This project is a comprehensive research platform designed for the end-to-end lifecycle of robotic learning. It provides a modular framework for training neural network policies—specifically through imitation and reinforcement learning—and deploying them onto physical robotic hardware. By offering a unified interface for hardware abstraction, the platform decouples high-level control logic from the specific sensors and actuators of diverse robotic systems. The framework distinguishes itself through a standardized approach to data and policy management. It utilizes a consistent schema for reco

Vega is a reactive visualization engine that translates structured specifications into interactive, browser-based graphical representations. It functions as a declarative grammar for data visualization, allowing users to define complex charts and maps through a JSON-based configuration format rather than imperative code. The system operates on a dataflow-based reactive graph that automatically propagates updates through the visualization whenever input data or user interactions change. By integrating a modular transformation pipeline, the engine handles data filtering, sorting, and aggregatio

Data-Juicer is an open-source framework for cleaning, filtering, deduplicating, and transforming multimodal datasets to prepare them for training large language and vision models. It functions as a distributed data pipeline engine that runs processing jobs across Ray clusters, handling billions of samples with automatic operator fusion and adaptive parallelism. The framework provides a library of operators that leverage large language models for semantic extraction, filtering, and data synthesis within processing pipelines. The project distinguishes itself through a YAML-based data recipe sys

Flashlight es una biblioteca de aprendizaje automático y de tensores independiente en C++ utilizada para construir y entrenar redes neuronales. Funciona como un framework integral de redes neuronales y motor de diferenciación automática, proporcionando las herramientas para construir grafos de computación y calcular gradientes mediante retropropagación. El proyecto sirve como framework de entrenamiento distribuido, utilizando operaciones all-reduce para sincronizar gradientes y parámetros a través de múltiples nodos de cómputo y dispositivos. Se distingue por una integración profunda de manipulación de tensores de alto rendimiento, interoperabilidad nativa de memoria de dispositivo y un sistema para sincronizar pesos a través de trabajadores distribuidos para acelerar el entrenamiento de modelos a gran escala. El framework cubre una amplia gama de capacidades de aprendizaje profundo, incluyendo composición modular de capas para diseñar arquitecturas complejas como bloques residuales y celdas recurrentes. Proporciona utilidades extensas de gestión de datos para ingesta y prefetching, junto con sistemas de serialización para persistir estados de modelos. Además, incluye una suite de herramientas de monitorización y observabilidad para rastrear métricas de entrenamiento y medir errores de secuencia. La biblioteca está implementada en C++.

SparkInternals es una referencia técnica y guía de arquitectura que detalla el diseño interno y la implementación del motor de computación distribuida Apache Spark. Sirve como un estudio de análisis de motores de big data, centrándose en cómo el sistema gestiona la ejecución en clúster y la interacción entre nodos driver, ejecutores y workers. El proyecto proporciona un desglose detallado de cómo los planes lógicos se convierten en etapas de ejecución física. Analiza específicamente la mecánica de las operaciones de shuffle de datos, la gestión de memoria y la coordinación de la programación de trabajos distribuidos. La documentación cubre una amplia gama de capacidades de computación distribuida, incluyendo la planificación de ejecución de consultas, la gestión de dependencias de datos y estrategias de caché en memoria. También examina la distribución de tareas, la ejecución paralela y los procesos utilizados para la recuperación ante fallos y la persistencia de datos.

This is an interactive notebook-based course that teaches machine learning from Python fundamentals through deep learning and natural language processing. It uses real datasets and multiple frameworks within a structured, hands-on curriculum that combines concise explanations with executable code cells, built-in datasets, and embedded exercise checkpoints. Learning progresses through data preparation and exploration, classical machine learning workflows, computer vision with convolutional neural networks, and natural language processing with deep learning, all delivered as a cohesive progressi

This is a structured deep learning curriculum for programmers, delivered as a collection of Jupyter notebooks. It teaches the fundamentals of training neural networks for computer vision, natural language processing, tabular data analysis, and collaborative filtering using PyTorch and the fastai library. The course is designed to be hands-on, guiding learners from building a training loop from scratch to fine-tuning pretrained models for a variety of practical tasks. The curriculum distinguishes itself by covering the full lifecycle of a deep learning project, from data preparation and augmen