4 repository-uri
Awesome GitHub RepositoriesConcurrent Data Processing
Simultaneous processing of large datasets across multiple workers to improve computational speed.
Distinct from Concurrent Stream Processing: General-purpose concurrent processing of item collections, not limited to streams or network packets.
Explore 4 awesome GitHub repositories matching data & databases · Concurrent Data Processing. Refine with filters or upvote what's useful.
Awesome Concurrent Data Processing GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- grosser/parallel
grosser/parallel
4,261Vezi pe GitHubParallel is a Ruby library and multi-process execution framework designed to accelerate CPU-intensive operations. It functions as a parallel job orchestrator and concurrent task runner that enables the execution of code across multiple processes or threads. The project distinguishes itself through secure inter-process communication, utilizing signed data serialization to prevent the injection of forged payloads between parent and child processes. It further differentiates its worker management by assigning unique identifiers to individual processes to prevent collisions when accessing shared
Processes large collections of items simultaneously using multiple workers to speed up heavy computational workloads.
Ruby - more-itertools/more-itertools
more-itertools/more-itertools
4,074Vezi pe GitHubmore-itertools is an extension library for the Python itertools module. It serves as a toolkit for manipulating iterables, providing a wide range of routines for data transformation, combinatorial generation, and iterator state management. The library distinguishes itself through advanced state management and complex sequence generation. It provides capabilities for peeking at future elements, seeking within sequences, and producing unique permutations, combinations, and set partitions from collections that may contain duplicate elements. Its broader capability surface covers data processing
Supports concurrent processing of multiple data collections to improve throughput for large datasets.
Python - gildas-lormeau/zip.js
gildas-lormeau/zip.js
3,859Vezi pe GitHubzip.js is a JavaScript library designed for creating, reading, and extracting ZIP archives directly within a web browser. It provides a comprehensive toolkit for managing compressed files and encrypted data storage entirely on the client side, eliminating the need for server-side backends or external dependencies. The library distinguishes itself by utilizing the Web Streams API and multi-core processing to handle large datasets efficiently. By offloading heavy compression and decompression tasks to background worker threads, it ensures that the browser interface remains responsive during int
Utilizes background worker pools and asynchronous operations to execute multiple read and write tasks simultaneously, maintaining application responsiveness during heavy processing.
JavaScriptaes-encryptionbrowsercompression-stream - camelot-dev/camelot
camelot-dev/camelot
3,764Vezi pe GitHubCamelot is a Python library and processing engine designed to extract tabular data from PDF documents. It converts unstructured tables into machine-readable formats such as CSV, JSON, and Excel. The project provides specialized toolsets for different document types, using line detection for ruled tables and whitespace analysis for borderless tables. It includes an optical character recognition system to recover structured data from image-based scanned PDFs that lack a digital text layer. The library handles complex document layouts, including encrypted files, rotated pages, and tables that s
Implements parallel execution across multiple CPU cores to increase the speed of PDF page parsing.
Python
