15 dépôts
Interfaces for streaming large datasets in chunks to prevent memory exhaustion during database traversal.
Distinct from Collection Iterators: Distinct from general Collection Iterators: focuses on asynchronous, chunked streaming for database records.
Explore 15 awesome GitHub repositories matching data & databases · Asynchronous Iterators. Refine with filters or upvote what's useful.
Async is a JavaScript asynchronous flow library designed to manage the execution and coordination of asynchronous tasks in Node.js and the browser. It provides functional utilities to wrap, process, and orchestrate complex asynchronous workflows. The library distinguishes itself through a comprehensive task orchestrator that handles dependency graphs to resolve circular references and manages concurrent task queues. It includes a unification bridge that allows callback-style and promise-based functions to operate within the same execution interface. The project covers several primary capabil
Applies asynchronous functions to each item in an array or object using parallel or series execution.
core-js is a comprehensive compatibility layer and standard library polyfill that implements ECMAScript proposals and stable language features across diverse JavaScript runtimes. It serves as a runtime environment shim to ensure consistent execution of global objects, iteration protocols, and standard library methods in older browsers or non-browser environments. The project is distinguished by its delivery models, offering both prototype-based global polyfilling and a pure-module implementation. This allows for the integration of modern functionality without modifying global prototypes to pr
Provides helper functions to sequence, chunk, or zip data streams for iterators.
This project is a comprehensive JavaScript programming tutorial and language reference. It serves as a web development education resource providing instruction on modern language fundamentals, object-oriented design, and advanced asynchronous programming patterns. The resource functions as both a frontend development guide and a technical reference. It covers core language features such as closures, prototypes, promises, and typed arrays, while providing practical lessons on managing browser data and handling network requests. The content spans several key capability areas, including browser
Teaches how to process values that arrive on-demand using asynchronous iteration and loops.
node-redis is a Node.js client and database driver for interacting with Redis key-value stores. It functions as a connection manager and data iterator, allowing applications to execute database commands and manage atomic transactions on a remote server using asynchronous JavaScript. The project provides specialized capabilities for handling large datasets through an asynchronous interface and supports connection pooling to isolate blocking operations. It includes support for client-side caching and token-based authentication via Entra ID. The driver covers a broad range of operational areas,
Provides asynchronous iterators to stream large datasets from Redis without risking memory overflow.
This project is a database driver for Node.js applications designed to interface with Redis. It provides structured access to data stores, enabling the execution of commands, management of data structures, and the implementation of atomic transaction processing. The client distinguishes itself through native support for the binary-safe serialization protocol and a promise-based command pipeline that groups operations to minimize latency. It includes a dedicated manager for distributed environments that handles node discovery and request routing, alongside an event-driven messaging system that
Streams large datasets in chunks to prevent memory exhaustion and avoid blocking the database server.
This project is a comprehensive programming course and educational curriculum designed to transition developers from basic scripting to advanced software development. It provides structured guides and technical exercises focusing on language internals, professional software architecture, and sophisticated programming techniques. The curriculum distinguishes itself through a deep focus on language internals, analyzing object behavior and memory efficiency to improve execution speed. It provides specialized instruction on metaprogramming using decorators and dynamic attributes, as well as async
Instructs on the management of streaming data and concurrent execution using asynchronous iterators and generators.
pysheeet est une bibliothèque de référence technique fournissant une collection organisée d'extraits de code et de modèles d'implémentation pour le développement Python avancé, l'intégration système et le calcul haute performance. Il sert de guide complet pour implémenter la programmation réseau de bas niveau, les extensions C natives, et la programmation asynchrone et concurrente. Le projet fournit des frameworks spécialisés pour le développement et le déploiement de grands modèles de langage, y compris des outils pour l'inférence GPU distribuée et le service haute performance. Il inclut également des modèles détaillés pour l'orchestration de clusters de calcul haute performance, couvrant l'allocation des ressources GPU et la gestion des charges de travail multi-nœuds. La bibliothèque couvre une large surface de capacités, y compris la communication réseau sécurisée et la cryptographie, l'ORM et la gestion de base de données, et l'implémentation de structures de données et d'algorithmes complexes. Elle fournit également des utilitaires pour la gestion de la mémoire, l'interopérabilité native via des interfaces de fonctions étrangères (FFI) et l'intégration au niveau du système d'exploitation.
Demonstrates the use of asynchronous comprehensions to filter and transform values from async generators.
collect.js is a dependency-free JavaScript library that provides a fluent, chainable interface for manipulating arrays and objects. It mirrors the Laravel Collection API, offering a consistent set of methods for data transformation across JavaScript and Laravel backend environments. The library stores collection data as plain arrays internally and supports fluent method chaining, where each method returns a new collection instance. The library distinguishes itself by closely replicating the Laravel Collection API in JavaScript, mapping each PHP method to an equivalent JavaScript implementatio
Executes a callback for each item in a collection, enabling side effects without modifying the original data.
Lazy.js is a JavaScript library that implements a lazy evaluation model for processing collections and data streams. It defers all computation until iteration begins, building chains of transformations that execute only when values are consumed, avoiding intermediate arrays and buffering. The library wraps data sources into a uniform sequence interface, enabling operations like map and filter to be chained together without materializing intermediate results. The library extends lazy processing beyond simple collections to handle asynchronous data sources, DOM events, strings, and Node.js stre
Iterate over sequence elements at timed intervals without blocking the main thread.
Toolz is a Python library that implements functional programming utilities for iterable transformation, dictionary manipulation, function composition, and lazy evaluation. It provides a set of pure functions designed to work with Python's built-in data structures, enabling concise and composable data processing workflows. What distinguishes toolz is its support for curried partial application, allowing functions to be incrementally applied and reused. It includes dictionary-centric operations that handle nested structures, and offers iterable chain transformers that combine mapping, filtering
Groups, deduplicates, interposes, and transforms sequences for processing iterable data.
Radash est une bibliothèque d'utilitaires fonctionnels TypeScript conçue pour la transformation de données, la validation et la gestion de workflow. Elle fournit une collection de fonctions d'aide typées pour manipuler les tableaux, les objets et les chaînes de caractères. Le projet propose une boîte à outils de workflow asynchrone pour gérer les promesses, les tentatives et les limites de concurrence, aux côtés d'une suite de validateurs booléens pour vérifier les types primitifs, les objets et les promesses. Il implémente également des patterns de programmation fonctionnelle tels que la composition de fonctions, l'application partielle et les patterns de réponse error-first. La bibliothèque couvre une large surface de capacités, incluant la gestion d'état d'objet avec clonage profond et fusion, des manipulations de tableaux complexes et le formatage de chaînes. Elle fournit en outre des outils pour l'optimisation de l'exécution, tels que le rate limiting et la mise en cache des résultats.
Offers functional utilities for executing asynchronous callbacks over each item in a collection.
more-itertools est une bibliothèque d'extension pour le module itertools de Python. Elle sert de boîte à outils pour manipuler les itérables, fournissant un large éventail de routines pour la transformation de données, la génération combinatoire et la gestion de l'état des itérateurs. La bibliothèque se distingue par une gestion d'état avancée et une génération de séquences complexes. Elle fournit des capacités pour jeter un coup d'œil aux éléments futurs, chercher au sein des séquences et produire des permutations, combinaisons et partitions d'ensembles uniques à partir de collections pouvant contenir des éléments en double. Sa surface de capacités plus large couvre des tâches de traitement de données telles que l'aplatissement récursif, le regroupement, le remplissage et le remodelage des flux de données. Elle inclut également des utilitaires pour la fusion de flux, le fenêtrage pour l'analyse de voisinage local et la synchronisation d'itération thread-safe. Le projet fournit en outre des routines spécialisées pour le traitement de séquences numériques, incluant la multiplication de matrices, la convolution linéaire discrète et les transformées de Fourier.
Provides utilities to map, filter, group, and combine iterables for data processing.
more-itertools is a Python iterable utility library providing advanced functions for manipulating, filtering, and transforming data sequences. It serves as a data stream processing toolkit and a set of utilities for iterator state management, extending the capabilities of the standard Python itertools module. The library includes a combinatorial math toolkit for generating permutations, combinations, and powersets, alongside routines for number theory calculations and matrix operations. It also provides tools for stream state management, allowing users to peek at upcoming elements or seek wit
Provides a comprehensive set of functions to map, filter, group, and combine iterables with a functional style.
pulldown-cmark is a pull-parsing library that transforms Markdown text into a stream of events based on the CommonMark specification. It functions as an event-based processor that represents document structure as an iterator of events rather than a concrete syntax tree, serving as both a parser and a renderer to convert Markdown into HTML strings. The library is designed for memory efficiency by processing text as a stream to minimize resource usage. It supports programmatic document transformation, allowing users to map or filter the event stream before final rendering. The project includes
Provides functional iterator combinators to map or filter the stream of parsed Markdown events.
Taco is a sparse tensor algebra compiler that translates high-level tensor index expressions into optimized machine code. It functions as a numerical code generator, producing specialized C kernels designed to execute complex multidimensional array operations efficiently on both CPU and GPU hardware. The project distinguishes itself by allowing users to define custom tensor storage layouts by composing dimension-level formats, such as dense or compressed structures, to match the specific sparsity patterns of their datasets. By analyzing the mathematical structure of tensor operations at compi
Optimizes data access patterns by automatically reordering loops and applying tiling strategies to match the underlying hardware memory hierarchy.