1 repositorio
Partitioning SQL queries to download data across simultaneous threads for increased ingestion speed.
Distinct from Data Parallelism Frameworks: Distinct from Data Parallelism Frameworks: focuses specifically on the partitioning and parallel downloading of SQL query results.
Explore 1 awesome GitHub repository matching devops & infrastructure · Parallel SQL Loading. Refine with filters or upvote what's useful.
Connector-X is a high-performance SQL data extraction library and bridge for transferring relational database records into memory-efficient data structures. It functions as a parallel database connector and federated query engine capable of executing and joining queries across multiple remote database connections to aggregate data locally. The project distinguishes itself through a zero-copy approach to data loading, which transfers SQL query results into memory structures without duplicating data. It maximizes throughput by partitioning SQL queries into threads, employing parallel columnar a
Increases data throughput by splitting SQL queries into partitions and downloading them via multiple simultaneous threads.