Open-source alternatives to Prql

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e

Crystal is a statically typed, compiled programming language designed for high performance and memory safety. It leverages an LLVM-based compiler to translate source code into optimized machine-executable binaries, while its type-inference-based static analysis enforces strict safety rules during the build process. The language distinguishes itself through a fiber-based concurrent runtime that manages lightweight execution units for asynchronous input and output without blocking the main process. It also features a powerful compile-time macro system that allows for the inspection and transfor

Arktype is a TypeScript runtime validation library and schema orchestrator. It synchronizes TypeScript types with runtime data validation, allowing users to define type-safe schemas that ensure unknown data adheres to specific structures during application execution. The project distinguishes itself by using set-theory type analysis to determine intersections and subtype compatibility, alongside JIT-compiled validation functions for optimized performance. It supports advanced type modeling through branded type constraints, recursive alias resolution, and the ability to generate runtime valida

sqlglot is a SQL parser and transpiler that represents queries as abstract syntax trees to enable structural analysis, modification, and semantic transformation. It functions as a dialect translator and query optimizer, converting SQL code between different database engines and simplifying syntax trees through rule-based normalization. The project provides a framework for defining custom SQL dialects by overriding tokenizers, parsers, and generators. It includes a lineage analyzer to track data flow from source tables through complex queries to identify the origin of specific columns. Additi

Knex is a multi-dialect database client that provides a programmatic SQL query builder, a connection pool manager, and a versioned schema migration tool. It enables programmatic database interaction across multiple SQL engines, including PostgreSQL, MySQL, SQLite3, SQL Server, CockroachDB, and Oracle. The project distinguishes itself through a fluent interface for constructing complex SQL statements and a dedicated framework for database seeding. It utilizes specialized dialects to translate generic query representations into database-specific syntax while maintaining a consistent API across

Squirrel is a Go database library and SQL query builder that provides a programmatic interface for constructing and executing SQL statements. It enables the creation of queries using a fluent interface to avoid manual string concatenation. The library functions as a SQL dialect generator, producing queries formatted for specific database engines by adjusting placeholders and syntax to match target requirements. This allows for the generation of SQL compatible with multiple different database environments. Beyond query construction, the project covers the execution of generated statements aga

Bun is a type-safe object relational mapper for Go that prioritizes SQL-first query construction and result mapping. It functions as a programmable SQL query builder, a database connection manager, and a tool for mapping database tables to Go structs. The project distinguishes itself through a multi-dialect SQL support system, allowing a single codebase to interact with different database engines via a consistent interface. It includes a built-in database observability tool for query interception, distributed tracing, and logging, as well as a schema migration tool for versioning structural c

Medoo is a lightweight PHP database abstraction layer and SQL query builder. It serves as a minimal database wrapper that provides a unified API to execute queries across multiple SQL engines, translating PHP method calls into specific database dialects. The project enables multi-dialect SQL interfacing, allowing a single codebase to interact with various backends including MySQL, MariaDB, PostgreSQL, SQLite, MSSQL, Oracle, and Sybase. It uses prepared statement parameterization to prevent injection attacks while maintaining a programmatic approach to constructing complex SQL statements. The

r4ds is a data science curriculum and educational resource designed for mastering the R programming language. It provides a structured learning path for the end-to-end process of importing, tidying, transforming, and visualizing data. The project emphasizes a reproducible data science guide and a comprehensive curriculum for data wrangling. It includes specialized tutorials on the grammar of graphics for layered data visualization and technical publications created with Quarto that blend executable code with narrative prose. The material covers a broad range of analytical capabilities, inclu

Xan is a command-line tool and data transformation engine for processing CSV, TSV, and JSONL datasets. It functions as a processor for compressed files, enabling random access and seeking within gzipped and Zstd files, and serves as a converter for specialized bioinformatics data formats. The tool handles large datasets without requiring full memory loads by utilizing stream-based processing. It provides capabilities for merging, sorting, and deduplicating massive files, as well as converting data between various tabular formats. The project covers a broad range of data wrangling and analysi

Kysely is a TypeScript SQL query builder that provides a type-safe interface for constructing and executing database queries. It functions as a database layer that ensures schema compliance and prevents runtime errors by using a fluent interface and a programmable way to build complex SQL statements. The project features a type-safe database layer capable of inferring return types and aliases from SQL selections and joins. It also includes a SQL migration manager to track and apply schema changes across different environments to keep database versions synchronized. The toolkit covers relatio

This project is a PostgreSQL client library and SQL query builder for JavaScript and TypeScript. It provides a low-level database driver and connection manager to handle database sessions, along with a logical replication client for monitoring real-time changes. The library distinguishes itself with a high-performance bulk data streamer that utilizes the database copy command for importing and exporting large datasets. It also implements a logical replication protocol to facilitate real-time database synchronization through change subscriptions and channel-based notifications. The toolset co

CodeIgniter is a PHP web framework built on the Model-View-Controller pattern, designed for building full-stack web applications. It provides a lightweight toolkit with minimal configuration, organizing application logic into controllers, models, and views for clean separation of concerns. The framework includes a fluent query builder for constructing SQL statements programmatically, PSR-4 autoloading with namespace mapping, and a service-based dependency injection container for managing shared class instances. The framework distinguishes itself through its comprehensive set of built-in tools

csvkit is a composable Unix-style command-line toolkit for converting, filtering, and analyzing CSV files directly from the terminal. It provides a suite of focused single-purpose commands that can be combined via pipes to build complex data processing workflows, with a modular architecture that includes a column-type inference engine for automatically detecting data types and a streaming-pipeline design for efficient handling of tabular data. The toolkit distinguishes itself through its SQL-engine abstraction layer, which allows users to run SQL queries directly against CSV files without req

Swift Evolution serves as the central governance and design platform for the Swift programming language. It provides a structured, collaborative framework for tracking, discussing, and managing the formal proposals and technical goals that define the language's syntax, semantics, and core features. By maintaining a comprehensive collection of design documentation, the project ensures the long-term stability and consistency of the language as it matures. The repository acts as the primary hub for the language's evolution, coordinating community feedback and technical decisions through a transp

This project is a comprehensive, community-maintained knowledge base and toolkit designed for competitive programming. It serves as a centralized repository for algorithmic theory, data structures, and mathematical techniques, providing a structured reference for informatics and collegiate programming competitions. The project distinguishes itself by integrating educational content with a robust suite of automation utilities. It provides a complete workflow for competitive programming, including tools for automated test case generation, solution verification, and direct interaction with onlin

Knex is a programmatic SQL query builder for Node.js applications. It provides a unified interface for generating database statements and managing structural versioning through a schema migration tool. The project features a multi-dialect database layer that translates programmatic queries into specific syntax for relational databases including PostgreSQL, MySQL, SQLite3, and MSSQL. This allows for writing database logic that remains compatible across different systems. The system includes infrastructure for managing reusable database connection pools and handling relational transactions to

FerretDB is an open-source database emulator and protocol translator that mimics a MongoDB environment to support existing drivers and client tools on a relational backend. It functions as a stateless database proxy that converts binary wire protocol messages into SQL statements, allowing a relational engine to handle document-oriented requests. The project serves as a migration tool for moving applications from MongoDB to PostgreSQL without rewriting queries or changing client drivers. It achieves this by using PostgreSQL as a document store, storing and querying BSON documents through a tra

This project is a command-line processor designed for the parsing, filtering, and transformation of structured data streams. It functions as a declarative programming environment that treats data as immutable streams, allowing users to perform complex structural modifications through the composition of small, reusable functions. By utilizing a recursive tree traversal engine, the system enables the navigation, inspection, and modification of deeply nested hierarchical data structures. The engine distinguishes itself through a stream-oriented architecture that processes input records one by on

Data-Juicer is an open-source framework for cleaning, filtering, deduplicating, and transforming multimodal datasets to prepare them for training large language and vision models. It functions as a distributed data pipeline engine that runs processing jobs across Ray clusters, handling billions of samples with automatic operator fusion and adaptive parallelism. The framework provides a library of operators that leverage large language models for semantic extraction, filtering, and data synthesis within processing pipelines. The project distinguishes itself through a YAML-based data recipe sys

This project is a SQL database abstraction layer that provides a consistent object-oriented interface for interacting with multiple relational database systems. It includes a driver wrapper to standardize connections and result sets, a fluent query builder for constructing portable SQL statements, and a type mapper for converting database-specific data types into native application types and vice versa. The library enables programmatic schema management through a schema manager that can introspect database metadata, model structures as objects, and generate the SQL required to migrate between

linq2db is a type-safe object-relational mapper that translates LINQ expressions into optimized SQL queries for multiple database providers. It functions as a database mapper that links classes to tables and includes a SQL query builder and a command-line schema tool for generating data classes from existing databases. The project provides high-performance bulk data processing for inserting and loading large volumes of records via batch or binary copy methods. It also supports advanced SQL operations, including window functions, common table expressions for recursive hierarchical querying, an

Fluvio is a distributed event streaming platform and cloud-native streaming engine designed for collecting, persisting, and replicating real-time data streams across a distributed cluster. It functions as a real-time data pipeline for building stateful workflows that ingest, enrich, and export data between external sources and sinks. The platform is distinguished by its use of WebAssembly to execute compiled modules for in-line data transformations and filtering. This allows for the execution of custom business logic to reshape information in motion without requiring a restart of the cluster.

PostgREST is a standalone server process that automatically transforms a relational database schema into a fully functional RESTful API. By querying system catalogs at startup, it maps tables, views, and stored procedures into standardized web endpoints, allowing developers to build backend services by focusing exclusively on database design. The service functions as a declarative engine that translates HTTP requests and query parameters directly into native SQL operations. It maintains stateless request processing and integrates connection pooling to manage high-frequency interactions effici

APIJSON is a no-code database API generator and object-relational mapping layer that automatically transforms relational database schemas into functional HTTP endpoints. It provides a system for generating CRUD operations and technical documentation without the need to write manual backend code. The project is distinguished by its ability to allow clients to define custom JSON response structures and field aliases directly within the request. It features dynamic query translation that converts structured JSON parameters into optimized SQL for complex relational joins, subqueries, and data agg

sqlx is a set of extensions for the Go database/sql package that reduces boilerplate code by automating the mapping of database query results directly into structs and slices. It provides a wrapper around standard database types to eliminate manual row scanning and repetitive error handling. The project distinguishes itself through named parameter binding and query placeholder rebinding, which translate generic markers into driver-specific symbols. It also enables dynamic SQL execution by allowing the application to read and execute SQL statements directly from the filesystem. The library co

Sequel is a relational database toolkit for Ruby that provides object-relational mapping, a fluent SQL query builder, and schema migration capabilities. It maps database tables to Ruby classes with support for associations, validations, lifecycle hooks, and eager loading, offering a comprehensive ORM layer for building data-centric applications. Sequel distinguishes itself through a plugin-based extension architecture that allows composable customization of models, databases, and datasets without relying on deep inheritance hierarchies. It includes a thread-safe connection pool with support f