46 repository-uri
Automated systems and scripts designed to move, transform, and validate data during transitions between storage environments.
Explore 46 awesome GitHub repositories matching data & databases · Data Migration. Refine with filters or upvote what's useful.
gstack is an AI agent framework and development workflow system designed to automate the software development lifecycle. It coordinates specialized AI personas to manage tasks across product design, engineering management, and quality assurance, transforming product intent into technical specifications and final releases. The project is distinguished by its deep integration of headless browser automation and semantic code memory. It utilizes a persistent Chromium daemon for web scraping and visual auditing, and implements a searchable knowledge base that logs architectural decisions and repos
Enables the migration of stored pages and embeddings between local and cloud database providers without data loss.
Laravel is a comprehensive full-stack web framework designed for building scalable server-side applications. It provides an integrated development environment that centers on an object-relational mapper for database abstraction, a robust routing system, and a sophisticated service container for dependency injection. The framework is built to handle complex application requirements through a modular architecture that emphasizes convention over configuration. What distinguishes Laravel is its deep integration of background processing and event-driven communication. It features a task queue orch
Provides a fluent, programmatic interface for defining and modifying database schemas without writing raw SQL.
This project is a command-line storage manager that provides a unified interface for performing file operations across local filesystems and diverse cloud storage providers. It functions as a cross-platform storage abstraction, utilizing a modular backend architecture to map heterogeneous cloud storage APIs into a standard set of file system operations. This allows for consistent data management and movement regardless of the underlying storage service. The tool serves as a network data transfer engine designed for automated data migration and cloud storage synchronization. It distinguishes i
Automates the migration of large data volumes between disparate storage systems while preserving file metadata.
Appwrite is a backend-as-a-service platform that provides a unified development environment for building full-stack applications. It integrates essential infrastructure components—including authentication, databases, storage, and serverless functions—into a single, centralized interface to simplify application development and resource management. The platform distinguishes itself through a container-based microservices architecture that ensures consistent execution across diverse infrastructure. It features a versatile connectivity layer that links frontend applications with third-party servi
Transfers users, databases, and files between external platforms and new project instances.
Faceswap is a comprehensive framework for automated media manipulation and neural face synthesis. It provides a modular pipeline that manages the entire lifecycle of facial feature extraction, deep learning model training, and image conversion. By coordinating complex computer vision workflows, the system enables users to map facial identities between source and destination datasets while maintaining structural alignment and lighting consistency across video frames. The project distinguishes itself through a highly extensible plugin-based architecture that handles hardware-accelerated process
Updates legacy alignment data structures to current standards by modifying formats and inserting required metadata keys.
This project is a build orchestration engine and development toolkit designed for managing large-scale monorepos. It provides a unified workspace environment that maps project relationships and dependencies, enabling the system to perform intelligent impact analysis and execute only the tasks affected by specific code changes. The system distinguishes itself through a persistent daemon that monitors file changes for near-instant feedback and a content-addressable caching mechanism that stores task outputs to prevent redundant computation across local and remote environments. It further suppor
Provides interactive control to selectively apply, reorder, or skip migration tasks during workspace updates.
This project is a distributed, document-oriented database system designed to store information in flexible, hierarchical structures. It supports horizontal scaling through automated sharding and maintains high availability across global clusters using a multi-node replication protocol. By executing multi-document operations as atomic units, the system ensures data integrity and consistency across distributed environments. The platform distinguishes itself by integrating advanced vector-based indexing, which enables semantic similarity searches alongside traditional geospatial and lexical quer
Automates the movement and synchronization of data between different storage environments and clusters.
Karakeep is a self-hosted, open-source platform designed for personal knowledge management and web content archiving. It functions as a centralized repository where users can capture, organize, and preserve bookmarks, notes, and media files, ensuring long-term access to digital information even if original sources are removed or modified. The system distinguishes itself through its automated content processing and security-focused architecture. It utilizes headless browser crawling and optical character recognition to ingest and index web content, while a modular artificial intelligence pipel
Facilitates the movement of large bookmark datasets between local storage and external files for backups.
This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha
Provides automated systems and scripts designed to move and validate data during transitions.
Gofr is a comprehensive framework for building production-ready microservices in Go. It provides a unified toolkit for developing RESTful APIs and gRPC services, offering built-in support for observability, database management, and distributed system communication. The framework distinguishes itself through its focus on developer productivity and system resilience. It automates common backend tasks such as CRUD handler generation, schema-driven code creation, and database migration orchestration, while preventing race conditions in clustered environments. To maintain stability, it includes in
Prevents race conditions in clustered environments by coordinating distributed database migration execution.
Wekan este un instrument de management de proiect Kanban open-source, auto-găzduit, utilizat pentru organizarea fluxurilor de lucru prin panouri, liste și carduri. Este o aplicație web în timp real care permite echipelor să gestioneze sarcinile pe infrastructură privată. Platforma se distinge prin instrumente extinse de migrare a datelor, în special pentru importul panourilor și cardurilor din Trello. Suportă integrarea identității la nivel enterprise prin LDAP, OpenID Connect și OAuth2 și oferă opțiuni flexibile de stocare, inclusiv PostgreSQL ca backend relațional primar și stocare cloud conectabilă pentru atașamente. Sistemul acoperă o gamă largă de capabilități de gestionare a sarcinilor, inclusiv vizualizări de diagrame Gantt, urmărirea timpului și agregarea sarcinilor între panouri. Include instrumente administrative pentru controlul accesului bazat pe roluri, programarea automată a backup-urilor și extensibilitate programatică printr-un API REST și webhook-uri bazate pe evenimente. Aplicația este disponibilă pentru implementare prin Docker și suportă configurații multi-tenant.
Implements extensive data migration tools specifically for importing boards and cards from Trello.
ShardingSphere is a distributed SQL database middleware that provides sharding, read-write splitting, and distributed transaction management for relational databases. It functions as a layer that intercepts SQL queries to distribute data across multiple physical database instances for horizontal scaling. The project is distinguished by its ability to operate as either a standalone transparent database proxy or via direct integration as a JDBC driver. It features a SQL dialect translator that parses queries into abstract syntax trees to convert syntax between different database engines, enabli
Moves data between different database instances using change data capture and automated migration pipelines.
Vitess is a database clustering system for horizontal scaling of MySQL. It functions as a middleware layer that abstracts complex sharding and physical topology, allowing applications to interact with a distributed database environment through a unified interface. By intercepting and routing SQL queries across multiple shards, it enables large-scale data management while maintaining the appearance of a single database instance. The platform distinguishes itself through its ability to perform online schema migrations and distributed transaction coordination without requiring application downti
Moves tables or data subsets between database instances online to facilitate infrastructure transitions without requiring application downtime.
Wagtail is an open-source content management system built on the Django web framework. It provides a structured, tree-based approach to content modeling, allowing developers to define custom page types and reusable content components that are managed through a highly customizable administrative interface. The platform distinguishes itself through its flexible, block-based content composition system, which enables editors to assemble complex page layouts dynamically. It also offers robust support for multi-site and multi-lingual environments, allowing organizations to manage distinct websites
Automates the restructuring of complex content schemas during database updates.
Taipy is a Python data application framework designed for building production-ready web interfaces that integrate AI algorithms and data pipelines. It provides a comprehensive suite for developing AI dashboards and data application interfaces. The framework includes a data pipeline orchestrator for managing complex workflows and dependencies, alongside a job scheduler for executing background tasks on recurring timetables. It also features a scenario analysis tool for performing what-if simulations by comparing different data input versions to evaluate their impact on algorithm outcomes. Add
Handles the transition of data between application versions to maintain consistency during system updates.
This project is an open-source, self-hosted helpdesk system designed to centralize customer support operations. It functions as an omnichannel platform that aggregates inquiries from email, social media, and messaging services into a unified dashboard, while providing a dedicated portal for customers to track requests and access self-service documentation. The system distinguishes itself through deep integration with e-commerce platforms, allowing agents to view customer order history and profile data directly within the ticketing interface. It features a modular architecture that supports cu
Supports the migration of users, mailboxes, and ticket history from external platforms into the system.
Wasp is a declarative full-stack web framework that enables developers to build and deploy applications by defining their architecture in a centralized configuration. By using a high-level specification, the framework automates the orchestration of frontend, backend, and database components, ensuring that infrastructure concerns like routing, authentication, and data modeling are handled consistently across the entire stack. The framework distinguishes itself through its compiler-driven approach, which translates declarative configurations into cohesive, production-ready codebases. It provide
Applies complex schema changes using incremental strategies to prevent data loss during breaking updates.
This project is a command-line utility designed to manage database schema versioning and automate incremental schema updates. It functions as a version control system for database structures, ensuring consistency across environments by tracking applied migrations in a dedicated metadata table and executing scripts in a sequential, reliable manner. The tool distinguishes itself through a driver-based abstraction layer that supports a wide range of database engines, including various SQL and distributed cloud databases. It provides robust concurrency control through advisory locking, which prev
Splits migration scripts into individual statements using delimiters to support complex schema modifications.
Ent is a statically typed entity framework for Go that models database structures as a graph of nodes and edges. It functions as a code generation engine that transforms schema definitions into type-safe database clients, query builders, and migration scripts. By representing data as interconnected entities, the framework enables intuitive traversal of complex relationships and ensures that database interactions remain consistent with the application model at compile time. The framework distinguishes itself through its graph-based approach to data modeling and its reliance on compile-time cod
Supports delegation of schema management to external migration engines for complex database operations.
LinkSwift is a cloud storage link extractor and file transfer utility designed to retrieve direct download addresses from various cloud storage providers. By generating raw download links, the tool enables users to bypass web interface restrictions and simplify the process of moving files between cloud platforms, local systems, or remote servers. The project functions as a direct download URL generator that automates file acquisition by programmatically parsing cloud storage pages. It employs a combination of headless browser automation and HTTP request interception to navigate dynamic web pa
Facilitates moving files between cloud storage providers by obtaining direct access links.