9 个仓库
Techniques for processing data in batches to improve computational efficiency.
Distinguishing note: Focuses on batch-oriented processing rather than row-level iteration.
Explore 9 awesome GitHub repositories matching data & databases · Vectorized Data Processing. Refine with filters or upvote what's useful.
Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f
Processes datasets in vectorized batches to achieve higher performance compared to row-by-row operations.
This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex
Groups processed features and labels into minibatches to facilitate efficient training and testing loops.
This project is a linear algebra tutorial and educational resource focused on the mathematical foundations of machine learning. It serves as a technical guide and instructional material for understanding how matrix calculations and linear operations power predictive algorithms. The resource emphasizes the transition from basic arithmetic to the implementation of predictive models. It focuses on linear algebra visualization to demonstrate how matrix operations translate into the geometric transformations used in data science. The material covers the implementation of machine learning logic th
Demonstrates processing multiple data samples simultaneously using vectorized matrix operations to increase throughput.
This repository provides a complete framework for training generative adversarial networks (GANs) that produce high-resolution photorealistic images, up to 1024 by 1024 pixels. The core technique is progressive layer growth, where both the generator and discriminator networks start training at low resolution and gradually add new layers to model finer details, enabling stable synthesis of large images. The framework includes a high-resolution image generator, an image quality metric evaluator, a latent space interpolation tool for creating smooth transition videos, and a multi-resolution datas
Implements minibatch standard deviation to help the discriminator detect mode collapse during training.
dplyr 是一个 R 语言数据处理库,为转换表格数据框提供了语法。它充当内存数据框处理器和关系数据代数工具,使用一组一致的动词来过滤、选择和汇总数据。 该项目包含一个 SQL 翻译引擎,可将高级数据处理表达式转换为优化后的查询。这允许用户直接在远程关系数据库和云存储上执行转换,而无需将数据拉取到本地。 该库涵盖了广泛的表格操作,包括列变异、行子集化和关系数据连接。它还提供了分组数据分析功能,允许对数据集进行分区以进行独立的聚合和汇总。
Applies functions across entire columns simultaneously to maximize computational efficiency within the R memory model.
ArrayFire 是一个硬件无关的计算框架和 JIT 编译张量引擎,专为高性能数值计算而设计。它作为一个 GPU 数值计算库和并行信号处理工具包,抽象了硬件后端,允许同一代码库在各种 GPU 架构和 CPU 上执行。 该项目以其使用表达式编译来融合操作并最小化内存开销的 JIT 引擎而脱颖而出。它采用延迟执行图来优化计算链,并提供互操作性原语以与 CUDA 和 OpenCL 等外部计算平台共享数据和执行上下文。 该库涵盖了广泛的功能,包括并行线性代数、数字信号处理和加速计算机视觉。它提供了用于机器学习实现、金融建模模拟以及求解物理系统模拟偏微分方程的工具。其张量管理系统处理多维数组分配、切片和主机-设备数据传输。
Executes operations across N-dimensional arrays by tiling data and parallelizing loop iterations on hardware.
ZLinq is a zero-allocation LINQ library and memory-efficient collection toolkit for C#. It provides a high-performance replacement for standard query operations by using value-type enumerators and pooled memory to eliminate heap allocations and reduce garbage collection overhead. The library features a C# source generator that automatically routes standard query method calls to these zero-allocation implementations. It further accelerates data processing through a SIMD accelerated data library, using hardware vectorization for numeric aggregations and bulk operations on primitive arrays and s
Processes array and span elements using hardware vector widths via lambda expressions for high-performance iteration.
USearch is a high-performance vector similarity search engine and approximate nearest neighbor index designed for dense embeddings. It functions as a low-level vector database core and high-dimensional vector indexer, providing the primitives necessary to store and retrieve vectors across massive datasets. The engine distinguishes itself through hardware-level SIMD acceleration for distance kernels and a proximity-graph indexing system that enables fast retrieval across billions of vectors. It supports multi-precision vector quantization to balance memory usage and accuracy, and utilizes memo
Processes multiple query vectors simultaneously using flattened arrays to maximize throughput for bulk similarity searches.
This project is a multi-purpose toolkit comprising a static site generator, a predictive modeling tool, and a sports analytics dashboard. It functions as a content syndication engine that converts source files into static HTML and machine-readable XML streams for blogs and professional portfolios. The system features a data processing engine designed for sports performance analytics, using linear and logistic regression to estimate season win totals and calculate win probabilities. It includes a time-series visualization framework that renders these performance trends using high-contrast them
Processes large datasets using vectorization and row-by-row application to increase computation speed.