3 repositorios
Awesome GitHub RepositoriesConcurrent Data Processing
Simultaneous processing of large datasets across multiple workers to improve computational speed.
Distinct from Concurrent Stream Processing: General-purpose concurrent processing of item collections, not limited to streams or network packets.
Explore 3 awesome GitHub repositories matching data & databases · Concurrent Data Processing. Refine with filters or upvote what's useful.
Awesome Concurrent Data Processing GitHub Repositories
Encuentra los mejores repositorios con IA.Buscaremos los repositorios que mejor coincidan usando IA.
- grosser/parallel
grosser/parallel
4,261Ver en GitHubParallel is a Ruby library and multi-process execution framework designed to accelerate CPU-intensive operations. It functions as a parallel job orchestrator and concurrent task runner that enables the execution of code across multiple processes or threads. The project distinguishes itself through secure inter-process communication, utilizing signed data serialization to prevent the injection of forged payloads between parent and child processes. It further differentiates its worker management by assigning unique identifiers to individual processes to prevent collisions when accessing shared
Processes large collections of items simultaneously using multiple workers to speed up heavy computational workloads.
Ruby - more-itertools/more-itertools
more-itertools/more-itertools
4,074Ver en GitHubmore-itertools es una biblioteca de extensión para el módulo itertools de Python. Sirve como un toolkit para manipular iterables, proporcionando una amplia gama de rutinas para transformación de datos, generación combinatoria y gestión de estado de iteradores. La biblioteca se distingue por la gestión de estado avanzada y la generación de secuencias complejas. Proporciona capacidades para observar elementos futuros, buscar dentro de secuencias y producir permutaciones, combinaciones y particiones de conjuntos únicas a partir de colecciones que pueden contener elementos duplicados. Su superficie de capacidades más amplia cubre tareas de procesamiento de datos como aplanamiento recursivo, agrupación, relleno y reformateo de flujos de datos. También incluye utilidades para fusión de flujos, ventanas para análisis de vecindad local y sincronización de iteración segura para hilos. El proyecto proporciona además rutinas especializadas para el procesamiento de secuencias numéricas, incluyendo multiplicación de matrices, convolución lineal discreta y transformadas de Fourier.
Supports concurrent processing of multiple data collections to improve throughput for large datasets.
Python - gildas-lormeau/zip.js
gildas-lormeau/zip.js
3,859Ver en GitHubzip.js is a JavaScript library designed for creating, reading, and extracting ZIP archives directly within a web browser. It provides a comprehensive toolkit for managing compressed files and encrypted data storage entirely on the client side, eliminating the need for server-side backends or external dependencies. The library distinguishes itself by utilizing the Web Streams API and multi-core processing to handle large datasets efficiently. By offloading heavy compression and decompression tasks to background worker threads, it ensures that the browser interface remains responsive during int
Utilizes background worker pools and asynchronous operations to execute multiple read and write tasks simultaneously, maintaining application responsiveness during heavy processing.
JavaScriptaes-encryptionbrowsercompression-stream
