3 个仓库
Awesome GitHub RepositoriesConcurrent Data Processing
Simultaneous processing of large datasets across multiple workers to improve computational speed.
Distinct from Concurrent Stream Processing: General-purpose concurrent processing of item collections, not limited to streams or network packets.
Explore 3 awesome GitHub repositories matching data & databases · Concurrent Data Processing. Refine with filters or upvote what's useful.
Awesome Concurrent Data Processing GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- grosser/parallel
grosser/parallel
4,261在 GitHub 上查看Parallel is a Ruby library and multi-process execution framework designed to accelerate CPU-intensive operations. It functions as a parallel job orchestrator and concurrent task runner that enables the execution of code across multiple processes or threads. The project distinguishes itself through secure inter-process communication, utilizing signed data serialization to prevent the injection of forged payloads between parent and child processes. It further differentiates its worker management by assigning unique identifiers to individual processes to prevent collisions when accessing shared
Processes large collections of items simultaneously using multiple workers to speed up heavy computational workloads.
Ruby - more-itertools/more-itertools
more-itertools/more-itertools
4,074在 GitHub 上查看more-itertools 是 Python itertools 模块的扩展库。它作为一个用于操作可迭代对象的工具包，提供了广泛的数据转换、组合生成和迭代器状态管理例程。 该库以高级状态管理和复杂的序列生成为特色。它提供了查看未来元素、在序列内搜索，以及从可能包含重复元素的集合中生成唯一排列、组合和集合划分的功能。 其更广泛的功能涵盖了数据处理任务，如递归展平、分组、填充和数据流重塑。它还包括用于流合并、局部邻域分析的窗口化以及线程安全的迭代同步的工具。 该项目还提供了用于数值序列处理的专门例程，包括矩阵乘法、离散线性卷积和傅里叶变换。
Supports concurrent processing of multiple data collections to improve throughput for large datasets.
Python - gildas-lormeau/zip.js
gildas-lormeau/zip.js
3,859在 GitHub 上查看zip.js is a JavaScript library designed for creating, reading, and extracting ZIP archives directly within a web browser. It provides a comprehensive toolkit for managing compressed files and encrypted data storage entirely on the client side, eliminating the need for server-side backends or external dependencies. The library distinguishes itself by utilizing the Web Streams API and multi-core processing to handle large datasets efficiently. By offloading heavy compression and decompression tasks to background worker threads, it ensures that the browser interface remains responsive during int
Utilizes background worker pools and asynchronous operations to execute multiple read and write tasks simultaneously, maintaining application responsiveness during heavy processing.
JavaScriptaes-encryptionbrowsercompression-stream
