1 dépôt
Awesome GitHub RepositoriesBatch-Based Stream Processing
Processing large datasets in small chunks to minimize memory usage during transfer.
Distinct from Event-Based Stream Processing: Distinct from Event-Based Stream Processing: focuses on fixed-size chunking for memory efficiency rather than event-driven triggers.
Explore 1 awesome GitHub repository matching data & databases · Batch-Based Stream Processing. Refine with filters or upvote what's useful.
Awesome Batch-Based Stream Processing GitHub Repositories
Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
- bruin-data/ingestr
bruin-data/ingestr
3,714Voir sur GitHubingestr is a command-line tool for copying and syncing data between different database engines and third-party platforms without writing custom code. It functions as an ETL pipeline utility that extracts data from diverse sources and loads it into destinations. The tool features a schema-agnostic data loader that maps source fields to destination columns dynamically, removing the need for predefined static table definitions. It also operates as an incremental data synchronizer, updating destination tables by appending new records or merging changes to maintain current datasets. The system pr
Processes large datasets in small chunks to maintain a low memory footprint during data transfer.
Go
