4 Repos
Mechanisms that identify and skip records that would cause primary key or unique constraint collisions during bulk insertion.
Distinct from Bulk Record Insertions: Focuses on skipping duplicates to prevent errors, rather than updating them (upsert) or handling them via specific conflict clauses.
Explore 4 awesome GitHub repositories matching data & databases · Duplicate Insert Suppression. Refine with filters or upvote what's useful.
FreeSql is a .NET object-relational mapper and data access layer that translates object-oriented code into SQL for multiple relational database providers. It functions as a fluent SQL query builder and database schema synchronizer, allowing developers to align database table and index structures with entity class definitions. The framework is specifically optimized for .NET Native AOT to ensure reduced memory footprints and faster startup times. It includes a database traffic manager to distribute load through read-write splitting, dynamic table sharding, and tenant-based data isolation. Bro
Skips records that would cause primary key or unique constraint collisions during insertion.
nano-graphrag ist ein Retrieval-System, das Wissensgraphen nutzt, um strukturierten Kontext für Antworten von Large Language Models (LLMs) bereitzustellen. Es fungiert als Wissensgraph-Indexer, der unstrukturierten Text in ein Netzwerk aus Entitäten und Beziehungen transformiert, sowie als hybrides Graph-Retrieval-System. Das Projekt unterscheidet sich durch die Kombination von lokalen Nachbarschaftssuchen mit globalen Community-Zusammenfassungen, um komplexe Fragen in natürlicher Sprache zu beantworten. Es enthält einen Wissensgraph-Visualisierer, der HTML-Repräsentationen von Entitäten und deren Beziehungen generiert, um indiziertes Wissen abzubilden. Das Framework deckt ein breites Spektrum an Funktionen ab, einschließlich Entitäts-Beziehungs-Extraktion, Community-basiertem Graph-Clustering und Hash-basiertem inkrementellem Indexing. Es bietet eine Integrationsschicht zur Anbindung von Open-Source-Modellen und lokalen Embedding-Providern, unterstützt durch austauschbare Storage-Backends für Key-Value-, Vektor- und Graph-Daten. Zusätzlicher Nutzen entsteht durch argumentbasierte Antwort-Cachings und Post-Processing-Funktionen zur Reparatur instabiler JSON-Ausgaben von Sprachmodellen.
Implements incremental indexing using content hashing to prevent duplicate storage and redundant processing.
Trillian is a distributed, multi-tenant verifiable data store that maintains cryptographically verifiable logs and maps using Merkle tree structures. It functions as a scalable backend for transparency logs, providing a system where data integrity is ensured through append-only records and mathematical proofs of inclusion and consistency. The system distinguishes itself by decoupling core storage from application-specific logic through a personality layer, which handles admission criteria and data canonicalization. It employs a consensus-based leader election mechanism for high availability a
Prevents duplicate entries by calculating a semantic hash of the content before committing it to the log.
Jet is a schema-driven code generation tool and type-safe SQL builder for Go. It introspects database schemas to automatically generate builders and data models, enabling compile-time type checking for table and column references to prevent runtime errors. The project distinguishes itself through a fluent interface that mirrors native SQL syntax, allowing for the orchestration of complex queries including common table expressions, recursive queries, and nested JSON structures. It further optimizes data retrieval by binding query outputs directly into generated Go structures or raw byte slices
Prevents duplicate errors by ignoring inserts or updating existing rows when constraints are violated.