20 个仓库
Tools and methods for mapping application objects to database records and managing data ingestion.
Distinguishing note: Focuses on the programmatic interface for record creation rather than general database management.
Explore 20 awesome GitHub repositories matching data & databases · Data Insertion Interfaces. Refine with filters or upvote what's useful.
SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer
Maps application objects to database records while supporting auto-generated and custom identifiers for data integrity.
Dapper is a lightweight object-relational mapper for .NET that functions as a high-performance data access library. It operates by extending standard database connection interfaces, allowing developers to execute raw SQL queries while automating the mapping of database results to strongly-typed objects. The library distinguishes itself through its use of runtime code generation, which creates high-performance instructions to map database rows to object properties with minimal overhead. It provides flexible data retrieval options, supporting both memory-buffered loading for speed and row-by-ro
Saves parent and child entities in a single operation by automatically managing relationships.
DataX is a distributed data integration framework and plugin-based ETL tool designed for synchronizing large datasets between heterogeneous sources and destinations. It functions as a JDBC data migration engine and offline synchronization tool, enabling the movement of data between relational databases, NoSQL stores, and object storage. The system utilizes a plugin-based connector architecture that decouples reader and writer logic, allowing it to map and transform data types across different storage engines using a standardized internal representation. This design supports heterogeneous data
Implements a programmatic interface for inserting records into target systems via standard JDBC statements.
This project is an educational framework designed to teach the fundamentals of building core distributed systems and web services from scratch in Go. It provides a collection of modular implementations that demonstrate how to construct essential infrastructure components, including web servers, remote procedure call systems, distributed caches, and database abstraction layers. The framework distinguishes itself by focusing on the internal mechanics of these systems rather than providing a high-level abstraction for production use. It covers the implementation of complex architectural patterns
The framework maps object fields to database columns and executes batch insertion commands to persist data into the underlying storage.
This project is a type-safe database toolkit and object-relational mapper designed to enforce structural consistency between database schemas and application code. By leveraging compile-time validation, it ensures that SQL queries and data structures remain synchronized, preventing common errors before the application executes. It provides a comprehensive framework for relational data modeling, allowing developers to define table associations and map database results directly into strongly-typed language objects. The library distinguishes itself through its focus on compile-time safety and au
Supports efficient batch insertion of multiple records to reduce query overhead.
This is a Laravel Excel integration library used for importing and exporting spreadsheet files within the Laravel framework. It functions as both a spreadsheet data exporter for converting database queries and collections into files and a spreadsheet data importer for mapping workbook data to database models. The library includes an HTML to spreadsheet converter that transforms HTML table templates into downloadable documents to preserve specific layout designs. It supports large dataset exports via background queuing and processes bulk data imports through chunked reading and batch inserts.
Implements programmatic interfaces to execute multiple database insert statements in batches during spreadsheet imports.
EmDash is an open-source content management system built on Astro that combines a visual admin panel with a plugin-driven architecture and server-side rendering. It provides a complete content management system with structured content modeling, a rich text editor using Portable Text format, and a TypeScript API for type-safe content queries. The system supports authentication through passkeys, OAuth 2.1, and external providers, with role-based access control and fine-grained permission scopes. What distinguishes EmDash is its plugin development framework, which supports both native plugins ru
Reads, writes, or deletes multiple records in a single call to reduce round trips and improve throughput.
Epoxy is an Android library for building complex RecyclerView screens using a model-driven approach. It generates RecyclerView adapter models at compile time from annotated custom views, data binding layouts, or view holders, eliminating the manual boilerplate typically associated with view holders and adapters. The library provides a diffing engine that automatically compares model lists and applies minimal updates with animations for insertions, removals, and moves. The library distinguishes itself through its controller-based model building, where a controller class with a buildModels meth
Compares old and new model lists to determine minimal changes and applies smooth animations.
A lightweight client for managing MariaDB, MySQL, SQL Server, PostgreSQL, SQLite, Interbase and Firebird, written in Delphi and Lazarus/FreePascal
Reads multiple files in a batch into BLOB columns with macros for neighbor fields in a binary-safe manner.
pg is a PostgreSQL object-relational mapper (ORM) for Go that maps Go structs to database tables and provides a fluent query builder for constructing SQL statements programmatically. At its core, it automatically generates CREATE TABLE statements from Go struct definitions using struct tags and naming conventions, and builds queries through method chaining with placeholder-based parameter binding to prevent SQL injection. The library distinguishes itself through relation-aware join generation that automatically constructs JOIN clauses for has-one, has-many, many-to-many, and polymorphic assoc
Combines multiple row insertions, updates, or deletions into single SQL statements to reduce round trips.
Inngest is a durable execution framework and event-driven automation engine designed to orchestrate background workflows. It enables developers to build resilient, stateful processes by memoizing function steps, ensuring that long-running tasks can automatically resume from the last successful operation after failures, timeouts, or infrastructure restarts. The platform distinguishes itself through its event-driven architecture, which uses a schema-validated bus to trigger functions and coordinate complex, multi-step logic. It employs an onion-model middleware approach for cross-cutting concer
Aggregates multiple individual database events into a single execution to improve efficiency when performing bulk operations.
Superduper 是一个 AI 代理开发套件和 LLM 应用程序框架,旨在构建自主代理和数据驱动的应用程序。它作为一个 RAG 编排平台和向量搜索基础设施,协调 AI 模型与数据库存储,以使用持久化数据状态执行多步计算和操作。 该项目通过提供数据库集成的机器学习流水线脱颖而出,该流水线直接在 SQL 和 NoSQL 数据库中托管的数据上执行训练和推理任务。它允许在私有硬件上部署自托管 AI 基础设施,从而实现对推理和数据的完全控制。 该框架涵盖了广泛的功能面,包括用于各种数据库后端的统一存储 API、自动化模式映射以及用于语义搜索的向量索引同步。它进一步提供了用于 AI 工作流执行、事件驱动模型触发以及将应用程序逻辑打包为可移植、可重用模板的工具。 该系统通过插件式抽象层支持与多种机器学习框架和托管 API 的集成。
Automatically analyzes data types during insertion to construct database tables and map storage types.
Querydsl 是一个用于构建类型安全查询的框架。它使用流畅的 API 和基于注解的代码生成从领域模型创建镜像类,从而实现数据库查询的编译时验证,并消除了手动字符串拼接的需要。 该项目提供了一种统一的查询语法,可转换为多个后端的特定方言,包括 SQL、MongoDB、Lucene 和 JDO。它支持高级查询功能,如公用表表达式(CTE)、窗口函数、地理空间操作和复杂的嵌套子查询。 除了数据检索外,该框架还涵盖了用于批量更新和删除的类型安全 DML 执行,以及将结果映射到 Java Bean、构造函数或元组。它包括对内存集合查询的支持,并与 Spring 框架集成以进行事务管理和连接处理。
Bundles multiple consecutive insert, update, or delete operations into a single database round trip.
Blitzar 是一个可验证的 SQL 证明引擎和加密库,专为可验证的 SQL 计算而设计。它支持在链下执行数据库查询,同时生成零知识证明,以证明结果的正确性,从而实现链上验证。 该项目以其 GPU 加速的证明加速器脱颖而出,该加速器将繁重的加密工作负载卸载到图形处理器,减少了简洁证明生成所需的时间。它为 C++ 和 Rust 应用提供了高性能加密原语,专注于椭圆曲线运算和多标量乘法。 该系统涵盖了广泛的数据管理和安全面,包括将区块链索引与链下数据集结合到防篡改关系表中的无信任数据集成。它利用 BFT 共识和阈值签名来维护状态完整性,以及用于基于法定人数的数据同步和通过智能合约回调进行验证结果交付的机制。 该代码库提供了 C++ 和 Rust 的原生绑定,以公开其加密工具集和证明计算库。
Ensures every data insertion into relational tables is verified using signatures or consensus mechanisms.
Bun 是一个 Go 语言的类型安全对象关系映射器(ORM),优先考虑 SQL 优先的查询构建和结果映射。它作为一个可编程 SQL 查询构建器、数据库连接管理器以及将数据库表映射到 Go 结构体的工具。 该项目通过多方言 SQL 支持系统脱颖而出,允许单个代码库通过一致的接口与不同的数据库引擎交互。它包括一个用于查询拦截、分布式跟踪和日志记录的内置数据库可观测性工具,以及一个用于版本化结构变更的模式迁移工具。 该库涵盖了广泛的数据操作,包括批量处理、Upsert、软删除以及多态关联等关系数据的管理。它提供使用公用表表达式(CTE)和窗口函数进行高级 SQL 分析的功能,以及原子事务管理和连接池。 模式管理通过用于应用版本化迁移脚本的命令行界面支持。
Synchronizes data by performing combined insert, update, or delete operations in a single SQL statement.
TablePro is a cross-platform database management client designed for browsing, querying, and administering both SQL and NoSQL databases. It functions as a unified workspace that integrates a code-centric SQL editor with schema visualization tools, allowing developers to manage complex data models and execute queries across diverse database engines. The application distinguishes itself through an agentic AI integration layer that connects language models directly to database tools, enabling automated query generation, optimization, and error fixing with configurable approval gates. It features
Updates, inserts, or deletes records directly within the data grid using standard database operations applied to the underlying storage.
Orioledb 是一个专为 PostgreSQL 设计的云原生存储引擎,旨在替换默认存储层,以提高现代硬件上的垂直扩展性和性能。它作为一个索引组织表存储,直接在主索引内组织表行,以加速数据检索。 该引擎利用撤销日志存储系统来管理数据版本控制,这消除了手动清理(vacuuming)的需要并防止了表膨胀。它还通过块级和页级数据压缩进一步减少了磁盘占用。 该项目提供了高级索引管理和自动化数据库维护功能。它包括通过行级日志记录实现高可用性恢复的功能,以及用于分析空间利用率和验证表完整性的工具。
Implements a batch page insertion path to reduce lock contention and increase write throughput.
EFCore.BulkExtensions 是一个用于在 Entity Framework Core 生态系统中执行高性能批量插入、更新和删除操作的库。它充当数据库批量处理工具包,并作为原生 SQL Bulk Copy 的包装器,以实现跨多个数据库提供商的更快数据摄取和同步。 该库为关系数据同步提供了专门的功能,允许用户通过批量更新插入 (upsert) 和条件同步将数据库表与本地实体列表对齐。它还支持关系数据图插入,这使得在维护外键关系的同时,能够跨多个表添加大量父子记录。 该工具涵盖了广泛的批量数据操作,包括常规 CRUD 处理、表截断和直接表对表数据复制。它还包括用于服务器端批量更新和高性能数据库迁移的实用程序,无需将实体加载到应用程序内存中即可移动大量数据。
Executes multiple row modifications or deletions in a single round trip to the database using filtered queries.
linq2db is a type-safe object-relational mapper that translates LINQ expressions into optimized SQL queries for multiple database providers. It functions as a database mapper that links classes to tables and includes a SQL query builder and a command-line schema tool for generating data classes from existing databases. The project provides high-performance bulk data processing for inserting and loading large volumes of records via batch or binary copy methods. It also supports advanced SQL operations, including window functions, common table expressions for recursive hierarchical querying, an
Inserts large collections of records using efficient batching or binary copy interfaces.
Drift is a type-safe SQL persistence library and relational mapper that provides a structured way to map database tables to classes and execute SQL queries with build-time validation. It functions as a type-safe query builder and a wrapper for SQLite and PostgreSQL, eliminating manual result set parsing by binding query outputs to native objects. The project distinguishes itself through a build-time code generation system that produces type-safe APIs and validates raw SQL statements against database versions before execution. It features reactive query streaming, which transforms SQL queries
Groups multiple insert statements into a single operation to improve write performance.