pudo/dataset
Dataset
该项目是一个 SQL 数据访问层和模式生成器，允许通过将表视为简单数据结构来读取和写入关系数据库中的记录。它作为一个自动模式生成器，根据传入数据的结构即时创建数据库表和列。
该工具提供了一个高性能批量加载器，使用分组原子事务导入大数据集以确保数据一致性。它还包括一个记录 Upsert 机制，根据唯一标识符确定是更新现有行还是插入新行。
该系统涵盖动态模式管理，包括隐式列解析和表配置。它还提供了一个基于集合的查询接口，用于检索记录或提取唯一值，而无需编写手动查询。
Features
- Dynamic Column Resolutions - Detects new fields in incoming data and modifies the database schema to accommodate them without manual migrations.
- SQL Data Access Layers - Read and write records by treating tables as simple data structures instead of writing manual queries.
- Dynamic Schema Evolution - Automatically creating and updating database tables and columns on the fly based on the structure of incoming data.
- SQL Translation Layers - Provides a translation layer that allows records to be read and written by treating database tables as simple data structures.
- Schema-on-Write Generators - Create tables and columns automatically when writing data to a destination that does not yet exist.
- Bulk Data Loading - Efficiently importing large sets of records into a database using bulk loading and transaction support.
- Collection Querying - Wraps SQL SELECT statements in simplified methods to retrieve full tables or unique column values.
- Data Abstraction Layers - Maps database rows to simple data structures to allow record manipulation without writing manual SQL queries.
- Upsert Operations - Implements a mechanism to update existing rows or insert new ones based on the presence of unique identifiers.
- On-the-Fly Table Creations - Automatically creates database tables and columns based on the structure of the data being inserted at runtime.
- Dynamic Column Provisioning - Automatically generates and modifies database tables and columns on the fly based on the structure of the incoming data.
- Dynamic Column Resolution - A tool that creates SQL tables and columns on the fly based on the structure of the data being written.
- Upsert Operations - Update existing rows or insert new ones based on whether a matching record exists in the table.
- Bulk Loaders - Groups multiple data insertions into a single atomic transaction to increase write throughput and ensure data consistency.
- Transactional Updates - Groups multiple data insertions into a single transaction to increase write performance and ensure atomic updates.
- Database Transaction Wrappers - Wraps multiple database operations into atomic transactions to ensure data consistency and increase write performance during bulk loading.
- Upsert Operations - Checks for existing records using unique identifiers to determine whether to update a row or insert a new one.
- Database Clients - Data handling for SQL stores.
- Database ORMs - Listed in the “Database ORMs” section of the Awesome Python awesome list.
- Database Tools - JSON-like interface for working with SQL databases.
- Object Relational Mappers - Stores Python dicts in relational databases.
- ORM Frameworks - Stores dictionaries in databases.
