15 Repos
Interfaces for streaming large datasets in chunks to prevent memory exhaustion during database traversal.
Distinct from Collection Iterators: Distinct from general Collection Iterators: focuses on asynchronous, chunked streaming for database records.
Explore 15 awesome GitHub repositories matching data & databases · Asynchronous Iterators. Refine with filters or upvote what's useful.
Async is a JavaScript asynchronous flow library designed to manage the execution and coordination of asynchronous tasks in Node.js and the browser. It provides functional utilities to wrap, process, and orchestrate complex asynchronous workflows. The library distinguishes itself through a comprehensive task orchestrator that handles dependency graphs to resolve circular references and manages concurrent task queues. It includes a unification bridge that allows callback-style and promise-based functions to operate within the same execution interface. The project covers several primary capabil
Applies asynchronous functions to each item in an array or object using parallel or series execution.
core-js is a comprehensive compatibility layer and standard library polyfill that implements ECMAScript proposals and stable language features across diverse JavaScript runtimes. It serves as a runtime environment shim to ensure consistent execution of global objects, iteration protocols, and standard library methods in older browsers or non-browser environments. The project is distinguished by its delivery models, offering both prototype-based global polyfilling and a pure-module implementation. This allows for the integration of modern functionality without modifying global prototypes to pr
Provides helper functions to sequence, chunk, or zip data streams for iterators.
This project is a comprehensive JavaScript programming tutorial and language reference. It serves as a web development education resource providing instruction on modern language fundamentals, object-oriented design, and advanced asynchronous programming patterns. The resource functions as both a frontend development guide and a technical reference. It covers core language features such as closures, prototypes, promises, and typed arrays, while providing practical lessons on managing browser data and handling network requests. The content spans several key capability areas, including browser
Teaches how to process values that arrive on-demand using asynchronous iteration and loops.
node-redis is a Node.js client and database driver for interacting with Redis key-value stores. It functions as a connection manager and data iterator, allowing applications to execute database commands and manage atomic transactions on a remote server using asynchronous JavaScript. The project provides specialized capabilities for handling large datasets through an asynchronous interface and supports connection pooling to isolate blocking operations. It includes support for client-side caching and token-based authentication via Entra ID. The driver covers a broad range of operational areas,
Provides asynchronous iterators to stream large datasets from Redis without risking memory overflow.
This project is a database driver for Node.js applications designed to interface with Redis. It provides structured access to data stores, enabling the execution of commands, management of data structures, and the implementation of atomic transaction processing. The client distinguishes itself through native support for the binary-safe serialization protocol and a promise-based command pipeline that groups operations to minimize latency. It includes a dedicated manager for distributed environments that handles node discovery and request routing, alongside an event-driven messaging system that
Streams large datasets in chunks to prevent memory exhaustion and avoid blocking the database server.
This project is a comprehensive programming course and educational curriculum designed to transition developers from basic scripting to advanced software development. It provides structured guides and technical exercises focusing on language internals, professional software architecture, and sophisticated programming techniques. The curriculum distinguishes itself through a deep focus on language internals, analyzing object behavior and memory efficiency to improve execution speed. It provides specialized instruction on metaprogramming using decorators and dynamic attributes, as well as async
Instructs on the management of streaming data and concurrent execution using asynchronous iterators and generators.
pysheeet ist eine technische Referenzbibliothek, die eine kuratierte Sammlung von Code-Snippets und Implementierungsmustern für fortgeschrittene Python-Entwicklung, Systemintegration und High-Performance-Computing bereitstellt. Sie dient als umfassender Leitfaden für die Implementierung von Low-Level-Netzwerkprogrammierung, nativen C-Erweiterungen sowie asynchroner und nebenläufiger Programmierung. Das Projekt bietet spezialisierte Frameworks für die Entwicklung und Bereitstellung von Large Language Models, einschließlich Werkzeugen für verteilte GPU-Inferenz und High-Performance-Serving. Es enthält zudem detaillierte Muster für die Orchestrierung von High-Performance-Computing-Clustern, die GPU-Ressourcenzuweisung und Multi-Node-Workload-Management abdecken. Die Bibliothek deckt ein breites Spektrum an Funktionen ab, einschließlich sicherer Netzwerkkommunikation und Kryptografie, Object-Relational-Mapping und Datenbankverwaltung sowie die Implementierung komplexer Datenstrukturen und Algorithmen. Sie bietet zudem Utilities für Speicherverwaltung, native Interoperabilität via Foreign-Function-Interfaces und systemnahe OS-Integration.
Demonstrates the use of asynchronous comprehensions to filter and transform values from async generators.
collect.js is a dependency-free JavaScript library that provides a fluent, chainable interface for manipulating arrays and objects. It mirrors the Laravel Collection API, offering a consistent set of methods for data transformation across JavaScript and Laravel backend environments. The library stores collection data as plain arrays internally and supports fluent method chaining, where each method returns a new collection instance. The library distinguishes itself by closely replicating the Laravel Collection API in JavaScript, mapping each PHP method to an equivalent JavaScript implementatio
Executes a callback for each item in a collection, enabling side effects without modifying the original data.
Lazy.js is a JavaScript library that implements a lazy evaluation model for processing collections and data streams. It defers all computation until iteration begins, building chains of transformations that execute only when values are consumed, avoiding intermediate arrays and buffering. The library wraps data sources into a uniform sequence interface, enabling operations like map and filter to be chained together without materializing intermediate results. The library extends lazy processing beyond simple collections to handle asynchronous data sources, DOM events, strings, and Node.js stre
Iterate over sequence elements at timed intervals without blocking the main thread.
Toolz is a Python library that implements functional programming utilities for iterable transformation, dictionary manipulation, function composition, and lazy evaluation. It provides a set of pure functions designed to work with Python's built-in data structures, enabling concise and composable data processing workflows. What distinguishes toolz is its support for curried partial application, allowing functions to be incrementally applied and reused. It includes dictionary-centric operations that handle nested structures, and offers iterable chain transformers that combine mapping, filtering
Groups, deduplicates, interposes, and transforms sequences for processing iterable data.
Radash is a TypeScript functional utility library designed for data transformation, validation, and workflow management. It provides a collection of typed helper functions to manipulate arrays, objects, and strings. The project features an asynchronous workflow toolkit for managing promises, retries, and concurrency limits, alongside a suite of boolean validators for verifying primitive types, objects, and promises. It also implements functional programming patterns such as function composition, partial application, and error-first response patterns. The library covers a broad surface of cap
Offers functional utilities for executing asynchronous callbacks over each item in a collection.
more-itertools ist eine Erweiterungsbibliothek für das Python-Modul itertools. Sie dient als Toolkit zur Manipulation von Iterables und bietet eine breite Palette an Routinen für Datentransformation, kombinatorische Generierung und Iterator-Zustandsverwaltung. Die Bibliothek zeichnet sich durch fortgeschrittene Zustandsverwaltung und komplexe Sequenzgenerierung aus. Sie bietet Funktionen zum Einsehen zukünftiger Elemente, zum Suchen innerhalb von Sequenzen und zum Erzeugen einzigartiger Permutationen, Kombinationen und Mengenpartitionen aus Sammlungen, die doppelte Elemente enthalten können. Der Funktionsumfang deckt Datenverarbeitungsaufgaben wie rekursives Flattening, Gruppierung, Padding und Umformung von Datenströmen ab. Sie enthält zudem Dienstprogramme für Stream-Merging, Windowing für lokale Nachbarschaftsanalysen und thread-sichere Iterationssynchronisation. Das Projekt bietet zudem spezialisierte Routinen für die Verarbeitung numerischer Sequenzen, einschließlich Matrixmultiplikation, diskreter linearer Faltung und Fourier-Transformationen.
Provides utilities to map, filter, group, and combine iterables for data processing.
more-itertools ist eine Python-Utility-Bibliothek für Iterables, die erweiterte Funktionen zur Manipulation, Filterung und Transformation von Datensequenzen bereitstellt. Sie dient als Toolkit für die Verarbeitung von Datenströmen und als Sammlung von Hilfsmitteln für das Management von Iterator-Zuständen, womit sie die Möglichkeiten des Standard-Moduls itertools erweitert. Die Bibliothek enthält ein kombinatorisches Mathe-Toolkit zur Erzeugung von Permutationen, Kombinationen und Potenzmengen sowie Routinen für zahlentheoretische Berechnungen und Matrixoperationen. Zudem bietet sie Werkzeuge für das Stream-State-Management, mit denen Benutzer einen Blick auf kommende Elemente werfen oder innerhalb einer Sequenz navigieren können, um die Datenverarbeitung zu steuern. Weitere Funktionen umfassen Routinen für das Chunking, Interleaving und Flattening komplexer Sequenzen. Das Toolkit enthält außerdem Funktionen zur Analyse von Iterable-Eigenschaften und zur Synchronisierung paralleler Datenströme.
Provides a comprehensive set of functions to map, filter, group, and combine iterables with a functional style.
pulldown-cmark is a pull-parsing library that transforms Markdown text into a stream of events based on the CommonMark specification. It functions as an event-based processor that represents document structure as an iterator of events rather than a concrete syntax tree, serving as both a parser and a renderer to convert Markdown into HTML strings. The library is designed for memory efficiency by processing text as a stream to minimize resource usage. It supports programmatic document transformation, allowing users to map or filter the event stream before final rendering. The project includes
Provides functional iterator combinators to map or filter the stream of parsed Markdown events.
Taco is a sparse tensor algebra compiler that translates high-level tensor index expressions into optimized machine code. It functions as a numerical code generator, producing specialized C kernels designed to execute complex multidimensional array operations efficiently on both CPU and GPU hardware. The project distinguishes itself by allowing users to define custom tensor storage layouts by composing dimension-level formats, such as dense or compressed structures, to match the specific sparsity patterns of their datasets. By analyzing the mathematical structure of tensor operations at compi
Optimizes data access patterns by automatically reordering loops and applying tiling strategies to match the underlying hardware memory hierarchy.