Awesome GitHub RepositoriesMulti-Source CSV Loading

TensorFlow.js is a JavaScript machine learning library used for training and deploying models in web browsers and server-side environments. It functions as a browser-based model trainer, a WebAssembly inference engine, and a WebGPU accelerated tensor library for low-level linear algebra. The project also includes a model converter to transform Python-based models into optimized formats for JavaScript execution. The library distinguishes itself through a pluggable backend architecture that allows mathematical operations to be executed via CPU, WebGL, or WebGPU. It supports the conversion of Py

Perspective is a columnar data analytics engine and high-performance visualization component powered by WebAssembly. It provides a system for analyzing and visualizing large or streaming datasets through interactive data grids and charts, utilizing a compiled binary to achieve near-native performance within the browser. The project distinguishes itself through a WebSocket-based data streaming interface and deep Apache Arrow integration, which minimize memory overhead when synchronizing tables between servers and clients. It acts as a remote query proxy capable of translating visualization con

Apache DataFusion is an extensible, columnar SQL query engine that runs embedded within a host application without requiring a separate server process. It processes data in columnar batches using Apache Arrow for memory-efficient analytics, and can scale analytic workloads across multiple nodes for parallel execution. The engine supports both SQL and DataFrame queries through a modular, streaming architecture that allows custom operators, data sources, functions, and optimizer rules. The engine distinguishes itself through its modular extension framework, which enables building custom query e

AlaSQL is a JavaScript SQL database engine that allows for the filtering, grouping, and joining of in-memory object arrays and JSON data. It functions as an in-memory SQL database and client-side data processor, enabling the execution of SQL statements against JavaScript arrays and external data sources in both browser and server environments. The project serves as a universal data query tool capable of performing relational joins across diverse sources, such as merging Google Spreadsheets, SQLite files, and remote APIs into a single result set. It also acts as an IndexedDB SQL wrapper, allow

Feast is an open-source feature store for machine learning that provides a central platform for defining, storing, and serving features across both training and inference workflows. It operates as a declarative system where feature definitions are written as code in Python files, synchronized to a central registry, and made available for low-latency online retrieval or point-in-time correct historical joins for training datasets. The project abstracts storage behind a pluggable architecture, allowing offline and online backends to be swapped without changing retrieval logic, and coordinates ma

Data-Juicer is an open-source framework for cleaning, filtering, deduplicating, and transforming multimodal datasets to prepare them for training large language and vision models. It functions as a distributed data pipeline engine that runs processing jobs across Ray clusters, handling billions of samples with automatic operator fusion and adaptive parallelism. The framework provides a library of operators that leverage large language models for semantic extraction, filtering, and data synthesis within processing pipelines. The project distinguishes itself through a YAML-based data recipe sys

This repository is the official documentation for TensorFlow, a machine learning framework. It provides comprehensive guides, tutorials, and API references for building, training, and deploying machine learning models. The documentation covers the full lifecycle of machine learning projects, from constructing data pipelines and building neural networks with high-level APIs to customizing training loops and deploying trained models in production, on edge devices, or in browsers. The documentation includes step-by-step tutorials for a range of tasks, including reinforcement learning, ranking mo

pgloader is a command-line tool that automates the migration of data and schema from various source databases and file formats into PostgreSQL. It combines schema discovery, parallel data pipelines, and type casting into a single, declarative workflow, using PostgreSQL's COPY protocol for high-throughput bulk loading. The tool distinguishes itself by compiling a dedicated command language into concurrent reader-writer pipelines that handle schema introspection, data transformation, and error-resilient batch processing. It supports migrating entire databases from MySQL, MS SQL, SQLite, and Pos

PlotJuggler is an interactive time series visualization tool that loads, streams, and renders large datasets using hardware-accelerated OpenGL graphics. It functions as a multi-format data loader, supporting file formats such as CSV, ULog, and ROS bags, and also serves as a live data stream viewer that subscribes to real-time sources via MQTT, WebSockets, ZeroMQ, and UDP. The tool distinguishes itself through a plugin-based extensibility platform that allows users to add custom data sources, file formats, and processing capabilities. It includes a Lua scripting engine for creating custom data

River es un framework de Python para machine learning online, diseñado para entrenar y evaluar modelos en datos de streaming. Permite el aprendizaje incremental actualizando los parámetros del modelo una observación a la vez, eliminando la necesidad de almacenar datasets de entrenamiento completos en memoria. La librería se distingue por un sistema dedicado de detección de concept drift que monitorea cambios en las distribuciones de datos para disparar la adaptación del modelo. También proporciona un framework de validación progresiva que simula el despliegue en tiempo real probando modelos en muestras antes de usarlos para el entrenamiento. El sistema cubre un amplio rango de capacidades de streaming, incluyendo ingeniería de características en tiempo real, pronóstico de series temporales y detección de anomalías online. Soporta aprendizaje no supervisado mediante clustering incremental y árboles de decisión, así como agregación de ensamblajes y políticas de bandidos para la selección de modelos. El proyecto incluye utilidades para la ingesta de datos de streaming desde fuentes como archivos CSV y APIs, así como herramientas para calcular estadísticas en ejecución y sketches de datos eficientes en memoria.

Anomalib is a PyTorch-based library for visual anomaly detection, offering a modular framework, a comprehensive model zoo, and a benchmarking suite designed for industrial defect detection. It provides a wide range of algorithms—including generative, discriminative, teacher-student, and vision-language approaches—that support unsupervised, few-shot, and zero-shot settings. The library enables deployment through model export to ONNX and OpenVINO for edge devices, and includes a no-code web application for training and inference. It also features a command-line interface for orchestrating multi

NVIDIA DALI is a GPU-accelerated data loading and preprocessing library designed for deep learning workflows. It constructs high-performance data pipelines that offload decoding, augmentation, and normalization to the GPU, eliminating CPU bottlenecks in training and inference. The library reads data from multiple storage formats and streams it directly into GPU memory, with support for multi-GPU execution to scale throughput across large-scale workloads. DALI distinguishes itself by enabling data pipelines to be built once and executed across multiple deep learning frameworks without code cha

Data on COVID-19 (coronavirus) cases, deaths, hospitalizations, tests • All countries • Updated daily by Our World in Data

Esta es una biblioteca de visualización de gramática de gráficos utilizada para construir gráficos mapeando datos tabulares a marcas visuales. Funciona como una herramienta de visualización de datos SVG y una API de análisis exploratorio de datos, permitiendo a los usuarios renderizar visualizaciones complejas y mapas geográficos. La biblioteca cuenta con un renderizador de mapas GeoJSON que proyecta coordenadas esféricas en un espacio de píxeles bidimensional y una interfaz de visualización de Apache Arrow para el procesamiento de datos de alta eficiencia. Su superficie de capacidades cubre la transformación de datos mediante binning y agrupación, codificación visual mediante inferencia automática de escala y aplicación de esquemas de color, y la generación de múltiples pequeños (small multiples). Admite la renderización de formas geométricas en vistas en capas y la exportación de imágenes estáticas en entornos de servidor.

Este proyecto es un índice de datos de búsqueda de código abierto y una colección de datos históricos de tendencias de búsqueda proporcionados como un archivo público de tendencias. Sirve como un dataset abierto para analizar patrones y eventos globales a través de archivos descargables. El repositorio proporciona un índice agregado de datasets de búsqueda y medios anonimizados y normalizados. Estos recursos están diseñados para el análisis académico y profesional, permitiendo el estudio de tendencias longitudinales a través de diferentes regiones y marcos temporales. Los datos soportan el análisis de tendencias de búsqueda global, análisis de patrones de mercado e investigación de interés público. Permite la adquisición de datos abiertos para el estudio del interés del consumidor, cambios sociales y comportamiento de búsqueda.

ExcelDataReader es una librería de C# utilizada para extraer datos y metadatos de hojas de cálculo de Microsoft Excel y archivos CSV. Funciona como un analizador de libros de trabajo que convierte el contenido de la hoja de cálculo en conjuntos de datos estructurados para acceso e iteración programática. El proyecto incluye un extractor de metadatos especializado para recuperar detalles a nivel de celda, como formatos de número, estilos, alturas de fila, anchos de columna y rangos de celdas combinadas. También proporciona un procesador de streams para analizar archivos CSV de texto plano con codificación personalizable y detección de separadores. La librería soporta el estándar OpenXML para archivos de hoja de cálculo modernos y utiliza análisis basado en stream e iteración de filas basada en cursor para recorrer los libros de trabajo. Estas capacidades permiten la conversión de libros de trabajo de múltiples hojas en tablas de datos relacionales.

docetl is an AI-powered document ETL tool and map-reduce orchestrator designed to transform large collections of unstructured documents into structured, queryable tables using language models. It provides a declarative pipeline framework for extracting, cleaning, and transforming data from sources such as PDFs and text files into predefined schemas. The project distinguishes itself through a semantic data integration suite that enables joining datasets and resolving duplicate entities based on embedding-based similarity. It includes an interactive prompt playground for developing and optimizi

mcp-context-forge is a Model Context Protocol federation gateway that unifies diverse AI tool servers and APIs into a single consistent interface for discovery and execution. It acts as a centralized proxy that aggregates multiple servers and APIs, allowing AI agents to access and invoke a unified set of tools, prompts, and resources. The project distinguishes itself through a multi-protocol translation bridge that converts communication between standard I/O, SSE, gRPC, and REST to enable interoperability between disparate tool servers. It includes a comprehensive LLM evaluation framework for