2 dépôts
Processing multiple data elements across pipeline stages simultaneously to maximize throughput.
Distinct from Per-Element Effect Execution: None of the candidates cover general concurrent element processing in data streams without being tied to images or hardware optimization.
Explore 2 awesome GitHub repositories matching data & databases · Concurrent Stream Processing. Refine with filters or upvote what's useful.
Streem is a stream-based programming language and data pipeline orchestrator. It provides a domain-specific language for defining concurrent data flows, allowing users to link data sources to destinations through a sequence of operations that transform and filter individual stream elements. The system uses a custom script syntax to define data-flow connections and pipeline definitions. This allows for the orchestration of concurrent data processing where multiple pipeline stages execute simultaneously to move data elements through the system. The platform covers functional data transformatio
Executes multiple pipeline stages simultaneously to move data elements through the system with higher throughput.
more-itertools is a Python iterable utility library providing advanced functions for manipulating, filtering, and transforming data sequences. It serves as a data stream processing toolkit and a set of utilities for iterator state management, extending the capabilities of the standard Python itertools module. The library includes a combinatorial math toolkit for generating permutations, combinations, and powersets, alongside routines for number theory calculations and matrix operations. It also provides tools for stream state management, allowing users to peek at upcoming elements or seek wit
Provides capabilities to synchronize and distribute data across multiple simultaneous consumers to prevent race conditions.