19 रिपॉजिटरी
Tools and methods for mapping application objects to database records and managing data ingestion.
Distinguishing note: Focuses on the programmatic interface for record creation rather than general database management.
Explore 19 awesome GitHub repositories matching data & databases · Data Insertion Interfaces. Refine with filters or upvote what's useful.
SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer
Maps application objects to database records while supporting auto-generated and custom identifiers for data integrity.
Dapper is a lightweight object-relational mapper for .NET that functions as a high-performance data access library. It operates by extending standard database connection interfaces, allowing developers to execute raw SQL queries while automating the mapping of database results to strongly-typed objects. The library distinguishes itself through its use of runtime code generation, which creates high-performance instructions to map database rows to object properties with minimal overhead. It provides flexible data retrieval options, supporting both memory-buffered loading for speed and row-by-ro
Saves parent and child entities in a single operation by automatically managing relationships.
DataX is a distributed data integration framework and plugin-based ETL tool designed for synchronizing large datasets between heterogeneous sources and destinations. It functions as a JDBC data migration engine and offline synchronization tool, enabling the movement of data between relational databases, NoSQL stores, and object storage. The system utilizes a plugin-based connector architecture that decouples reader and writer logic, allowing it to map and transform data types across different storage engines using a standardized internal representation. This design supports heterogeneous data
Implements a programmatic interface for inserting records into target systems via standard JDBC statements.
This project is an educational framework designed to teach the fundamentals of building core distributed systems and web services from scratch in Go. It provides a collection of modular implementations that demonstrate how to construct essential infrastructure components, including web servers, remote procedure call systems, distributed caches, and database abstraction layers. The framework distinguishes itself by focusing on the internal mechanics of these systems rather than providing a high-level abstraction for production use. It covers the implementation of complex architectural patterns
The framework maps object fields to database columns and executes batch insertion commands to persist data into the underlying storage.
This project is a type-safe database toolkit and object-relational mapper designed to enforce structural consistency between database schemas and application code. By leveraging compile-time validation, it ensures that SQL queries and data structures remain synchronized, preventing common errors before the application executes. It provides a comprehensive framework for relational data modeling, allowing developers to define table associations and map database results directly into strongly-typed language objects. The library distinguishes itself through its focus on compile-time safety and au
Supports efficient batch insertion of multiple records to reduce query overhead.
This is a Laravel Excel integration library used for importing and exporting spreadsheet files within the Laravel framework. It functions as both a spreadsheet data exporter for converting database queries and collections into files and a spreadsheet data importer for mapping workbook data to database models. The library includes an HTML to spreadsheet converter that transforms HTML table templates into downloadable documents to preserve specific layout designs. It supports large dataset exports via background queuing and processes bulk data imports through chunked reading and batch inserts.
Implements programmatic interfaces to execute multiple database insert statements in batches during spreadsheet imports.
EmDash is an open-source content management system built on Astro that combines a visual admin panel with a plugin-driven architecture and server-side rendering. It provides a complete content management system with structured content modeling, a rich text editor using Portable Text format, and a TypeScript API for type-safe content queries. The system supports authentication through passkeys, OAuth 2.1, and external providers, with role-based access control and fine-grained permission scopes. What distinguishes EmDash is its plugin development framework, which supports both native plugins ru
Reads, writes, or deletes multiple records in a single call to reduce round trips and improve throughput.
Epoxy is an Android library for building complex RecyclerView screens using a model-driven approach. It generates RecyclerView adapter models at compile time from annotated custom views, data binding layouts, or view holders, eliminating the manual boilerplate typically associated with view holders and adapters. The library provides a diffing engine that automatically compares model lists and applies minimal updates with animations for insertions, removals, and moves. The library distinguishes itself through its controller-based model building, where a controller class with a buildModels meth
Compares old and new model lists to determine minimal changes and applies smooth animations.
A lightweight client for managing MariaDB, MySQL, SQL Server, PostgreSQL, SQLite, Interbase and Firebird, written in Delphi and Lazarus/FreePascal
Reads multiple files in a batch into BLOB columns with macros for neighbor fields in a binary-safe manner.
pg is a PostgreSQL object-relational mapper (ORM) for Go that maps Go structs to database tables and provides a fluent query builder for constructing SQL statements programmatically. At its core, it automatically generates CREATE TABLE statements from Go struct definitions using struct tags and naming conventions, and builds queries through method chaining with placeholder-based parameter binding to prevent SQL injection. The library distinguishes itself through relation-aware join generation that automatically constructs JOIN clauses for has-one, has-many, many-to-many, and polymorphic assoc
Combines multiple row insertions, updates, or deletions into single SQL statements to reduce round trips.
Inngest is a durable execution framework and event-driven automation engine designed to orchestrate background workflows. It enables developers to build resilient, stateful processes by memoizing function steps, ensuring that long-running tasks can automatically resume from the last successful operation after failures, timeouts, or infrastructure restarts. The platform distinguishes itself through its event-driven architecture, which uses a schema-validated bus to trigger functions and coordinate complex, multi-step logic. It employs an onion-model middleware approach for cross-cutting concer
Aggregates multiple individual database events into a single execution to improve efficiency when performing bulk operations.
Superduper is an AI agent development kit and LLM application framework designed to build autonomous agents and data-driven applications. It functions as a RAG orchestration platform and vector search infrastructure, coordinating AI models with database storage to perform multi-step computations and actions using persisted data states. The project distinguishes itself by providing a database-integrated machine learning pipeline that executes training and inference tasks directly on data hosted within SQL and NoSQL databases. It allows for the deployment of self-hosted AI infrastructure on pri
Automatically analyzes data types during insertion to construct database tables and map storage types.
Querydsl is a framework for the construction of type-safe queries. It uses a fluent API and annotation-based code generation to create mirror classes from domain models, enabling compile-time validation of database queries and removing the need for manual string concatenation. The project provides a unified query syntax that translates into specific dialects for multiple backends, including SQL, MongoDB, Lucene, and JDO. It supports advanced query capabilities such as common table expressions, window functions, geospatial operations, and complex nested subqueries. Beyond data retrieval, the
Bundles multiple consecutive insert, update, or delete operations into a single database round trip.
Blitzar एक वेरिफिएबल SQL प्रूफ इंजन और क्रिप्टोग्राफिक लाइब्रेरी है जिसे वेरिफिएबल SQL कंप्यूटेशन के लिए डिज़ाइन किया गया है। यह ऑन-चेन सत्यापन के लिए परिणामों की शुद्धता को प्रमाणित करने वाले ज़ीरो-नॉलेज प्रूफ जेनरेट करते समय डेटाबेस क्वेरी को ऑफ-चेन निष्पादित करने में सक्षम बनाता है। यह प्रोजेक्ट एक GPU-त्वरित प्रूफ एक्सीलरेटर के माध्यम से खुद को अलग करता है जो भारी क्रिप्टोग्राफिक वर्कलोड को ग्राफिक्स प्रोसेसर पर ऑफलोड करता है, जिससे संक्षिप्त प्रूफ जनरेशन के लिए आवश्यक समय कम हो जाता है। यह C++ और Rust एप्लिकेशन के लिए उच्च-प्रदर्शन क्रिप्टोग्राफिक प्रिमिटिव प्रदान करता है, जो एलिप्टिक कर्व ऑपरेशंस और मल्टी-स्केलर मल्टीप्लिकेशन पर केंद्रित है। यह सिस्टम डेटा प्रबंधन और सुरक्षा की एक विस्तृत सतह को कवर करता है, जिसमें ट्रस्टलेस डेटा एकीकरण शामिल है जो ब्लॉकचेन इंडेक्सिंग को ऑफ-चेन डेटासेट के साथ छेड़छाड़-रोधी रिलेशनल टेबल में जोड़ता है। यह स्टेट अखंडता बनाए रखने के लिए BFT कंसेंसस और थ्रेशोल्ड सिग्नेचर का उपयोग करता है, साथ ही स्मार्ट कॉन्ट्रैक्ट कॉलबैक के माध्यम से कोरम-आधारित डेटा सिंक्रोनाइज़ेशन और सत्यापित परिणाम वितरण के लिए तंत्र का उपयोग करता है। कोडबेस अपने क्रिप्टोग्राफिक टूलसेट्स और प्रूफ कंप्यूटेशन लाइब्रेरी को उजागर करने के लिए C++ और Rust के लिए नेटिव बाइंडिंग प्रदान करता है।
Ensures every data insertion into relational tables is verified using signatures or consensus mechanisms.
Bun, Go के लिए एक टाइप-सेफ ऑब्जेक्ट रिलेशनल मैपर है जो SQL-फर्स्ट क्वेरी निर्माण और परिणाम मैपिंग को प्राथमिकता देता है। यह एक प्रोग्रामेबल SQL क्वेरी बिल्डर, एक डेटाबेस कनेक्शन मैनेजर, और डेटाबेस टेबल्स को Go स्ट्रक्ट्स (structs) से मैप करने के लिए एक टूल के रूप में कार्य करता है। यह प्रोजेक्ट मल्टी-डायलेक्ट SQL सपोर्ट सिस्टम के माध्यम से खुद को अलग करता है, जिससे एक सिंगल कोडबेस को एक सुसंगत इंटरफेस के माध्यम से विभिन्न डेटाबेस इंजनों के साथ इंटरैक्ट करने की अनुमति मिलती है। इसमें क्वेरी इंटरसेप्शन, डिस्ट्रीब्यूटेड ट्रेसिंग और लॉगिंग के लिए एक इन-बिल्ट डेटाबेस ऑब्जर्वेबिलिटी टूल, साथ ही संरचनात्मक परिवर्तनों के वर्ज़निंग के लिए एक स्कीमा माइग्रेशन टूल शामिल है। यह लाइब्रेरी बल्क प्रोसेसिंग, अपसर्ट्स, सॉफ्ट डिलीट्स, और पॉलीमॉर्फिक एसोसिएशन जैसे रिलेशनल डेटा के मैनेजमेंट सहित डेटा ऑपरेशंस की एक विस्तृत श्रृंखला को कवर करती है। यह एटॉमिक ट्रांजेक्शन मैनेजमेंट और कनेक्शन पूलिंग के साथ-साथ कॉमन टेबल एक्सप्रेशंस और विंडो फंक्शन्स का उपयोग करके उन्नत SQL एनालिटिक्स के लिए क्षमताएं प्रदान करती है। स्कीमा मैनेजमेंट को वर्ज़न्ड माइग्रेशन स्क्रिप्ट्स को लागू करने के लिए एक कमांड-लाइन इंटरफेस के माध्यम से सपोर्ट किया जाता है।
Synchronizes data by performing combined insert, update, or delete operations in a single SQL statement.
Orioledb PostgreSQL के लिए एक क्लाउड-नेटिव स्टोरेज इंजन है जिसे आधुनिक हार्डवेयर पर वर्टिकल स्केलेबिलिटी और प्रदर्शन में सुधार करने के लिए डिफ़ॉल्ट स्टोरेज लेयर को बदलने के लिए डिज़ाइन किया गया है। यह एक इंडेक्स-ऑर्गनाइज्ड टेबल स्टोर के रूप में कार्य करता है, जो डेटा रिट्रीवल को तेज करने के लिए टेबल पंक्तियों को सीधे प्राथमिक इंडेक्स के भीतर व्यवस्थित करता है। इंजन डेटा वर्शनिंग को प्रबंधित करने के लिए एक अनडू लॉग स्टोरेज सिस्टम का उपयोग करता है, जो मैन्युअल वैक्यूमिंग की आवश्यकता को समाप्त करता है और टेबल ब्लोट को रोकता है। यह ब्लॉक-लेवल और पेज-लेवल डेटा कम्प्रेशन के माध्यम से डिस्क फुटप्रिंट को और कम करता है। यह प्रोजेक्ट उन्नत इंडेक्स प्रबंधन और स्वचालित डेटाबेस रखरखाव के लिए क्षमताएं प्रदान करता है। इसमें रो-लेवल लॉगिंग के माध्यम से उच्च उपलब्धता रिकवरी के लिए सुविधाएं शामिल हैं, साथ ही स्थान उपयोग का विश्लेषण करने और टेबल अखंडता को सत्यापित करने के लिए उपकरण भी शामिल हैं।
Increases write throughput by using a batch page insertion path to reduce lock contention.
EFCore.BulkExtensions is a library for executing high-performance batch insert, update, and delete operations within the Entity Framework Core ecosystem. It functions as a database batch processing toolkit and a wrapper for native SQL Bulk Copy to enable faster data ingestion and synchronization across multiple database providers. The library provides specialized capabilities for relational data synchronization, allowing users to align database tables with local entity lists through bulk upserts and conditional synchronization. It also supports relational data graph insertions, which enable t
Compares local and database datasets using primary keys to automatically determine which records require synchronization.
linq2db is a type-safe object-relational mapper that translates LINQ expressions into optimized SQL queries for multiple database providers. It functions as a database mapper that links classes to tables and includes a SQL query builder and a command-line schema tool for generating data classes from existing databases. The project provides high-performance bulk data processing for inserting and loading large volumes of records via batch or binary copy methods. It also supports advanced SQL operations, including window functions, common table expressions for recursive hierarchical querying, an
Inserts large collections of records using efficient batching or binary copy interfaces.
Drift is a type-safe SQL persistence library and relational mapper that provides a structured way to map database tables to classes and execute SQL queries with build-time validation. It functions as a type-safe query builder and a wrapper for SQLite and PostgreSQL, eliminating manual result set parsing by binding query outputs to native objects. The project distinguishes itself through a build-time code generation system that produces type-safe APIs and validates raw SQL statements against database versions before execution. It features reactive query streaming, which transforms SQL queries
Groups multiple insert statements into a single operation to improve write performance.