37 repositorios
Comprehensive systems for maintaining data structure consistency over time.
Distinguishing note: Focuses on the overall management of migration files and synchronization.
Explore 37 awesome GitHub repositories matching data & databases · Database Migration Management. Refine with filters or upvote what's useful.
This project is a comprehensive Java backend engineering guide and technical reference focused on high-concurrency design, distributed systems, and microservices architecture. It provides detailed strategies for decomposing monolithic applications, managing service discovery, and implementing the architectural patterns required for scalable backend environments. The repository distinguishes itself through an extensive collection of big data algorithmic references and database scaling strategies. It covers memory-efficient techniques for analyzing massive datasets, such as Top-K element extrac
Implements dual-writes and background synchronization to transfer data to sharded environments without downtime.
Payload is a headless content management system and application framework that uses a code-first approach to define data schemas and administrative interfaces. By utilizing a centralized, type-safe configuration object, it automatically generates database schemas, API endpoints, and a fully customizable admin panel. The system is built on a database-agnostic architecture, allowing it to interface with various storage engines while providing a unified, type-safe API for server-side operations, REST, and GraphQL. What distinguishes Payload is its deep extensibility and developer-centric design.
Synchronizes application data structures with the database engine through managed migration files.
This project is a human resources management system built using Spring Boot and Vue. It serves as a platform for managing employee records, professional titles, and organizational hierarchies. The system features a role-based access control framework that maps users to specific roles and resources to secure API endpoints and user interface elements. It includes a real-time communication hub utilizing WebSockets for internal corporate chat and system notifications, as well as a dedicated manager for defining and modifying nested organizational department structures. Additional capabilities co
Automates SQL script execution to synchronize database schemas and track version history across environments.
Firefly III is a self-hosted personal finance management system built on a double-entry bookkeeping engine. It provides a comprehensive platform for tracking income, expenses, and account balances while maintaining financial integrity through structured accounting principles. Designed for private use, the system supports multi-user access, allowing independent financial administrations to coexist within a single installation. The platform distinguishes itself through extensive automation and integration capabilities. It features a robust REST JSON API and webhook system that enables programma
Maintains data structure consistency by managing and rolling back failed schema updates.
Activepieces is an open-source, self-hosted workflow automation platform designed to connect third-party applications through modular triggers and actions. It provides a low-code integration framework that allows users to build, manage, and execute complex business logic sequences within isolated, sandboxed environments. The platform distinguishes itself through its focus on embeddability and enterprise-grade security. It features an embedded automation builder that can be integrated into external applications via iframes, supported by comprehensive identity and access management tools such a
Executes incremental database updates using versioned migration files with forward and rollback logic.
This project is a command-line utility designed to manage database schema versioning and automate incremental schema updates. It functions as a version control system for database structures, ensuring consistency across environments by tracking applied migrations in a dedicated metadata table and executing scripts in a sequential, reliable manner. The tool distinguishes itself through a driver-based abstraction layer that supports a wide range of database engines, including various SQL and distributed cloud databases. It provides robust concurrency control through advisory locking, which prev
Manages and automates schema updates across environments using versioned scripts and concurrency-safe advisory locking.
WeKnora is a multi-tenant retrieval-augmented generation (RAG) knowledge platform and autonomous AI agent framework. It transforms raw documents into queryable knowledge bases and integrates large language models with vector databases to provide grounded AI responses. The system also functions as a Model Context Protocol (MCP) tool server, exposing knowledge search and agentic capabilities to external AI clients. The platform distinguishes itself through an autonomous agent framework that utilizes iterative reasoning, tool calling, and web search to solve multi-step tasks. It implements a sta
Executes and rolls back database schema changes to maintain data structure consistency across environments.
This toolkit provides an asynchronous interface for interacting with relational databases, offering a unified driver-agnostic layer for managing connection pools and executing transactions. It is designed to integrate with asynchronous runtimes, enabling non-blocking database operations while maintaining secure, encrypted communication between the application and the database server. The project distinguishes itself through its compile-time validation capabilities, which use procedural macros to inspect SQL syntax and parameter types against a live database schema during the build process. Th
Manages and executes versioned schema migration scripts embedded directly within the application binary.
FoundationDB is an ACID-compliant distributed transactional key-value store. It functions as a scalable database engine that ensures strict serializability and data consistency across a cluster of servers using a shared-nothing architecture. The system is distinguished by its multi-region replication capabilities, allowing data to be synchronized across different datacenters for high availability and disaster recovery. It utilizes optimistic concurrency control to manage distributed transactions and employs a majority-based coordination system to maintain cluster state. The platform provides
Moves database state and coordination roles to new machines without downtime by adding nodes.
Entity Framework Core is an object-relational mapper that enables developers to interact with database systems using strongly-typed code. It serves as a comprehensive data access framework, providing a unified interface for mapping application objects to relational and non-relational database schemas while managing the lifecycle of data operations through a central context. The project distinguishes itself through a provider-based architecture that decouples core data access logic from specific database engines, allowing for consistent interaction across diverse storage systems. It features a
Organizes schema changes into distinct sets by associating migration histories with specific database providers or context configurations.
This project is a TypeScript-based architectural framework designed to implement domain-driven design and hexagonal architecture in enterprise applications. It provides a structured boilerplate that isolates core business logic from infrastructure concerns, ensuring that domain entities, use cases, and external technology adapters remain decoupled and maintainable. The framework distinguishes itself by enforcing strict architectural boundaries and dependency inversion, preventing unauthorized access to core logic from external layers. It utilizes a command-query responsibility segregation pat
Executes schema changes and versioning scripts to keep database structures synchronized across development, testing, and production environments.
FreshRSS is an open-source, self-hosted web feed aggregator designed to collect, organize, and display content from multiple websites in a single, centralized interface. It functions as a comprehensive reader for standard syndication formats, allowing users to track updates from various sources while maintaining full control over their data and privacy. The platform supports multi-user environments, enabling individual account management and personalized reading experiences. The application distinguishes itself through its robust synchronization and extensibility capabilities. It provides a s
Manages the transfer of user data between different database engines through export and import procedures.
gh-ost is a triggerless online schema migration tool for MySQL. It functions as a replication client and table management utility that synchronizes data from a source table to a shadow table using binary logs, allowing for table structure modifications without locking original tables or causing downtime. The tool distinguishes itself by using binary-log-based replication instead of triggers to stream row-based events to a shadow table. It implements load-aware throttling and dynamic performance tuning to adjust migration speed based on server load and replication lag. Users can monitor and ad
Enables updating large database tables with new columns or indexes without causing service outages.
Encore is a distributed systems framework designed to unify backend development, infrastructure provisioning, and observability. It functions as an infrastructure-as-code platform that allows developers to define cloud resources, databases, and messaging topics directly within their application code. By analyzing these declarations at compile-time, the system automatically manages the deployment of cloud resources and security policies, ensuring parity between local development and production environments. The platform distinguishes itself through its integrated development experience, which
Automatically executes SQL migration files during startup to ensure schema synchronization.
This project provides educational materials and courseware focused on the theoretical and practical foundations of distributed systems design. It serves as a comprehensive curriculum covering the disciplines of consensus, data consistency, reliability engineering, and scalability. The instructional content focuses on achieving cluster agreement through consensus algorithms and managing system-wide state via coordination frameworks. It includes a dedicated guide to data theory, exploring replication strategies, consistency models, and data convergence. The courseware covers a broad capability
Covers techniques for transferring data between environments without interrupting service availability.
Mikro-ORM is a TypeScript-based object-relational mapping system that provides a unified persistence layer for Node.js applications. It translates TypeScript entities into relational or document-based database schemas, supporting a variety of engines including PostgreSQL, MySQL, MariaDB, MS SQL Server, SQLite, and MongoDB. The project implements the data mapper pattern to decouple in-memory domain models from the database persistence layer. It utilizes a unit of work pattern to track entity changes in memory and commit them in a single coordinated database transaction. The library covers com
Ships a comprehensive system for tracking and applying schema changes using versioned migration files.
Dropwizard is a Java web application stack and REST framework used to build production-ready web services. It integrates an embedded HTTP server with operational tools to provide a complete environment for developing network interfaces that exchange JSON data. The project provides a set of core technology stacks for relational data mapping, schema migration, and operational performance monitoring. It includes a relational database migration tool to track schema updates and execute versioned scripts before an application starts. The framework covers a broad capability surface including reques
Provides a comprehensive system for managing and versioning database schema migrations.
Realtime is a real-time data distribution and synchronization engine that enables applications to stream database changes and coordinate state between clients. It functions as a synchronization layer that monitors database write-ahead logs to provide change data capture and pushes updates to authorized clients via WebSockets. The project features a real-time presence server for tracking the online status of active users and a broadcast service for sending ephemeral messages without database persistence. It organizes communication through channel-based message routing and uses a structured JSO
Applies schema changes across environments to maintain consistency between local and production databases.
KnowStreaming is a centralized Kafka cluster management platform that unifies multi-cluster federation, load balancing, disaster recovery, and resource governance through a web-based graphical interface. It provides a single control plane for administering brokers, topics, partitions, consumer groups, ACLs, and connectors across heterogeneous Kafka clusters without requiring CLI commands or agent deployment on brokers. The platform distinguishes itself through automated load balancing that redistributes partition leaders and replicas to eliminate hotspots and improve throughput, combined with
Executes partition reassignment plans by coordinating leader election and data copying across brokers with zero downtime.
SQLDelight is a Kotlin database library that validates SQL schema, statements, and migrations at compile time, generating type-safe Kotlin query functions from labeled SQL files. It treats SQL as the source of truth for database definitions, catching schema errors during the build process before they reach production. The library supports multiple database dialects including SQLite, MySQL, PostgreSQL, HSQL, and H2, and generates platform-specific code for Android, iOS, JVM, and JavaScript targets. It provides a platform-specific driver abstraction that handles database connectivity difference
Applies numbered SQL migration files in order to evolve the database schema.