8 个仓库
Abstractions that transform sequential data structures into parallel streams for multi-threaded processing.
Distinct from Parallel Data Transformation: Distinct from Parallel Data Transformation: focuses on the iterator-based interface for data partitioning rather than general transformation logic.
Explore 8 awesome GitHub repositories matching data & databases · Parallel Iterators. Refine with filters or upvote what's useful.
Genesis World is an embodied AI simulation platform designed for training robotic agents through physics-based interactions. It centers on a multi-physics simulation engine that integrates rigid body, particle, and finite element method dynamics, supported by a parallel simulation kernel compiler that translates Python functions into optimized GPU and CPU kernels. The platform features a photorealistic robot renderer that utilizes path-tracing and Gaussian Splatting to generate synthetic training data. It includes a domain randomization framework to vary lighting and physical parameters acros
Accelerates data-parallel computations by distributing top-level simulation loop iterations across hardware threads.
Codon is an LLVM-based Python compiler and statically typed implementation that translates source code into optimized machine instructions. It functions as a high-performance numerical backend and a GPU computing framework designed to remove runtime overhead. The project implements a compiled alternative to NumPy, translating array logic directly into machine code. It differentiates itself by generating specialized hardware kernels for graphics processors and utilizing static type inference to enable aggressive machine-code optimization. The system provides capabilities for parallel workload
Distributes loop iterations across physical hardware threads to maximize CPU throughput.
Rayon is a data parallelism library for Rust that provides a framework for converting sequential computations into parallel operations. It enables the transformation of standard data structures and loops into parallel iterators, allowing workloads to be distributed across multiple processor cores. By utilizing a work-stealing scheduler, the library dynamically balances tasks to maximize throughput and minimize execution time. The library distinguishes itself through its focus on safe, scoped task synchronization, which ensures that all spawned operations complete before a scope exits to preve
Transforms sequential data structures into parallel iterators to partition work across multiple processor cores.
oneAPI Threading Building Blocks (oneTBB)
Provides parallelfor and parallelreduce algorithms that split data ranges into chunks for concurrent processing.
Osmedeus is a security workflow orchestration engine that coordinates AI agents, shell commands, and scanning tools through declarative YAML pipelines. It functions as a distributed security scanner, a declarative workflow automator, and an AI agent framework for security, enabling automated multi-step security analysis with conditional branching, parallel execution, and distributed workers. The engine distinguishes itself through a hybrid runner model that executes workflow steps on the local host, inside Docker containers, or over SSH to remote machines, selected per step or module. It supp
Processes lists of items in parallel within workflow steps using a foreach executor.
ArrayFire 是一个硬件无关的计算框架和 JIT 编译张量引擎,专为高性能数值计算而设计。它作为一个 GPU 数值计算库和并行信号处理工具包,抽象了硬件后端,允许同一代码库在各种 GPU 架构和 CPU 上执行。 该项目以其使用表达式编译来融合操作并最小化内存开销的 JIT 引擎而脱颖而出。它采用延迟执行图来优化计算链,并提供互操作性原语以与 CUDA 和 OpenCL 等外部计算平台共享数据和执行上下文。 该库涵盖了广泛的功能,包括并行线性代数、数字信号处理和加速计算机视觉。它提供了用于机器学习实现、金融建模模拟以及求解物理系统模拟偏微分方程的工具。其张量管理系统处理多维数组分配、切片和主机-设备数据传输。
Distributes high-level loop iterations across physical hardware threads to maximize GPU and CPU throughput.
Async 是一个 Swift 库,为 Grand Central Dispatch 提供包装器,以简化异步任务和队列的管理。它作为一个任务调度器,用于代码块的执行、计时和编排。 该项目包含一个任务链式处理器,用于在不同线程间序列化操作而无需嵌套回调结构,以及一个将迭代分布在多个线程上的并发循环包装器。它还具有基于令牌的取消系统,用于在挂起操作运行前从执行队列中跟踪并移除它们。 该库通过协调异步组和同步多个任务以在集体完成后触发操作,涵盖了并发管理功能。
Runs multiple blocks of code simultaneously by wrapping concurrent iteration functions to reduce total processing time.
ndarray 是一个 Rust 的多维数组库,用作线性代数框架和科学计算工具。它提供了创建和操作 n 维数组的核心基础设施,既充当并行数组处理器,也充当数值数据分析工具包。 该库的独特之处在于提供高效的切片和内存视图,允许在不复制的情况下共享数据。它利用优化的后端数学库进行高速矩阵乘法,并将繁重的数学迭代分布到多个 CPU 线程上以加速处理。 该项目涵盖了广泛的数学运算,包括逐元素算术、基于轴的数据聚合和点积计算。它还包括用于数组操作的全面实用程序,如重塑、展平、堆叠和坐标网格生成,以及对随机数组生成和序列化的支持。
Distributes element-wise operations and axis reductions across multiple CPU cores using parallel iterators.