electric-sql/electric
Electric
Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates.
The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework to ensure consistent state synchronization across different users.
The project covers a broad range of capabilities, including reactive state management and real-time data streaming to client interfaces and server-side renders. It provides tools for data shaping and transformation, database integration across various cloud and serverless Postgres providers, and security primitives such as token-based authorization and end-to-end encryption.
The service can be deployed as a containerized web service on cloud platforms with support for rolling deployment management.
Features
- Data Synchronization Engines - Connects to Postgres databases using logical replication to synchronize data across clients in real time.
- Local-First Architectures - Implements a local-first architecture by synchronizing remote Postgres data to embedded client databases for offline access.
- Client-Side State Synchronizers - Synchronizes remote Postgres database data into local client-side stores for low-latency web and mobile access.
- Change Data Capture Streams - Consumes a real-time log of database changes into a stream to materialize local data structures.
- Data Shaping - Allows defining specific rows and columns from tables to be synchronized using filters and selections.
- Dynamic Sync Shapes - Allows the creation of personalized data structure definitions at runtime based on the current session and request.
- Database Replication Middleware - Configures how database tables publish for replication using automated or manual privilege control modes.
- Remote Synchronization - Implements background synchronization of data from an embedded local database to a remote server.
- Client-Side Materializers - Streams database changes to a JavaScript environment using an interface that materializes change logs into local state.
- Log Materialization - Transforms a stream of logical database operations into a local data structure to maintain current state.
- Real-time Data Synchronization - Streams database changes from Postgres to client applications to update user interfaces instantly.
- Shape-Based Synchronization - Defines data shapes at compile time to synchronize specific database subsets in real time using authentication middleware.
- Live Update Subscriptions - Pushes database changes to clients via persistent connections to reduce latency and request overhead.
- Local-First Synchronization - Provides a synchronization layer that streams database changes to embedded client stores for offline-first functionality.
- Database Change Streaming - Streams specific subsets of Postgres database rows and columns to clients or edge functions via logical replication.
- Conflict-Free Replicated Data Types - Integrates a conflict-free replicated data type (CRDT) framework with a database to enable consistent multi-user collaboration.
- Real-Time Data Streaming - Offers an API that pushes live database updates to connected clients via persistent connections.
- Real-time Collaboration - Transports conflict-free replicated data types (CRDTs) to support multi-user editing and shared application state.
- Incremental Sync - Provides a mode to omit the initial data snapshot and stream only new changes after connection.
- Database Integrations - Integrates with cloud-hosted Postgres instances via connection strings to synchronize data with clients.
- Shard Routing Logic - Directs client connections to specific database shards using client-side logic or edge routing to distribute load.
- Edge-to-Cloud Synchronization - Hydrates data into edge workers or durable objects to enable low-latency processing at the network edge.
- On-Demand Subset Loading - Reduces initial load times by loading specific data subsets on-demand via snapshots or query-driven synchronization.
- Postgres Adapters - Establishes direct connections to serverless Postgres hosts to enable logical replication.
- Real-Time Data Streaming - Pushes real-time database updates into serverless edge functions to generate dynamic responses.
- Mobile Synchronization - Synchronizes data from a remote database into mobile applications to provide real-time updates and offline access.
- Synchronization Snapshot Optimizations - Optimizes data transfer by selecting between full historical snapshots or streaming only new changes for initial synchronization.
- API Access Security - Requires secret tokens for all HTTP API calls to prevent unauthorized access to the synchronization service.
- API Authentication - Secures data access using gatekeepers or proxies to authorize requests between the client and synchronization engine.
- End-to-End Encryption - Protects sensitive information by encrypting data at the client before transmission and decrypting it upon receipt.
- Reverse Proxy Security - Validates user credentials and shape parameters through a reverse proxy before streaming data to clients.
- Token-based Authorization - Issues shape-scoped tokens that are validated by a proxy to ensure only authorized data subsets are requested.
- Component Data Bindings - Binds materialized data shapes to component state using hooks to manage the subscription lifecycle.
- Reactive Data Stores - Synchronizes database state to a local reactive store that automatically triggers UI updates.
- User-to-Shard Migrations - Moves users between shards by updating mapping and triggering a client-side resync via control messages.
- Reactive Data Bindings - Triggers callbacks or broadcasts when data changes to automatically update the user interface.
- Collaborative Editing Frameworks - Combines real-time synchronization with conflict-resolution frameworks to enable multi-user collaborative editing.
- Reactive State Management - Binds database change streams to client-side stores to automatically trigger UI updates.
- Observable State Patterns - Streams database changes into an observable state container to enable automatic user interface updates.
- Real-Time Collaboration Tools - Provides a framework for building multi-user applications with shared state and real-time synchronization.
- Sync-Aware Rendering - Integrates real-time data synchronization with server-side rendering to ensure seamless transitions to client-side updates.
