zdennis/activerecord-import

Features

• Bulk Record Insertions - Provides high-performance capabilities to insert large datasets into a database using single or batched queries.
• ActiveRecord Bulk Importers - Provides a specialized library for inserting large datasets into a database using ActiveRecord through single or batched queries.
• Batch Size Management - The project limits the number of rows processed per single statement to optimize memory and avoid database packet size limits.
• Conflict Resolution - Handles data collisions during bulk writes by updating existing records or ignoring duplicates.
• Data Upsert Operations - Manages the upsert pattern to ensure data presence and currency when importing records that may violate unique constraints.
• Upsert Engines - Implements a data import tool that handles unique constraint conflicts by updating existing records or ignoring duplicates.
• Data Validation Layers - Provides a data validation layer to enforce integrity and validity before records are persisted to the database.
• Pre-Insertion Record Validations - Checks the validity of large datasets before insertion and tracks which specific records failed the validation process.
• Pre-Insertion Validations - Runs model validation logic on each record in a collection before committing the bulk operation.
• Batch Statement Generation - Chunks large datasets into smaller groups to prevent database packet size errors and manage memory usage.
• Upsert Strategies - Implements strategies to either update existing records or ignore conflicts based on unique constraints during imports.
• Import Chunking - Splits massive datasets into smaller chunks to manage memory usage and enable progress tracking via callbacks.
• Bulk Record Validations - Verifies record validity before insertion and generates detailed reports for instances that fail the validation process.
• Multi-Format Input Support - Processes bulk data from various source types including arrays of values, hashes, and model collections.
• Import Result Summaries - Returns detailed summaries containing the count of successful inserts and identifiers for the created records.
• Batch-Driven Data Migration - Processes massive datasets in controlled batches to manage memory usage and avoid database packet size limits.
• Association Record Management - Manages the recursive insertion of parent records and their nested child associations in a single database operation.
• Association Traversals - Processes parent and child records in a sequence that maintains foreign key integrity across related tables.
• Related Record Importers - Resolves and associates related database records recursively during the bulk import process.
• Dialect-Specific SQL Generation - Generates SQL syntax tailored to specific database dialects for optimized bulk inserts and upserts.
• Import Progress Tracking - Tracks and displays the completion status of bulk imports as data batches are processed.
• ORM Extensions - Bulk inserts data into databases efficiently.
• Database Tools - Handles bulk data insertion.

Star history