43 Repos
Mechanisms for replicating data between edge devices and centralized cloud storage.
Distinguishing note: Focuses on distributed synchronization, distinct from local-only data management.
Explore 43 awesome GitHub repositories matching data & databases · Edge-to-Cloud Synchronization. Refine with filters or upvote what's useful.
TDengine is a distributed time-series database designed for the high-speed ingestion, compression, and retrieval of timestamped metrics and sensor data. It functions as a SQL-compatible analytics engine, allowing users to perform complex operations on massive volumes of time-ordered information using standard relational syntax. The platform is built to serve as a backend foundation for industrial IoT environments, managing real-time data streams and device metadata through a cluster-based architecture. The system distinguishes itself through a distributed sharding architecture that uses consi
Synchronizes time-series data automatically between edge and cloud nodes to ensure consistent information.
This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
Deploys database branches to geographically distributed servers to minimize latency and maintain data consistency.
iosched is an Android event scheduling application designed for browsing conference sessions, reserving seats, and managing personal event schedules. It serves as a cloud-synced event manager that keeps user preferences and reservations synchronized across devices. The project provides specialized tools for conference navigation, including integrated maps and informational pages to guide attendees through physical venue spaces. It also functions as a real-time notification client that delivers live updates regarding schedule changes and session availability. The application covers broad capa
Synchronizes starred and reserved events between the local device and a cloud database for offline access.
NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t
Communicates interest in specific subjects across servers to ensure messages are routed only to nodes with active subscribers.
This is a mobile object database and NoSQL local data store that replaces relational tables with a schema-based model. It functions as a reactive data store, using live object observations and change notifications to trigger automatic user interface refreshes. The system provides built-in mobile cloud data synchronization to keep local datasets consistent with a remote server across multiple devices. It also includes security features for encrypted local storage, protecting sensitive on-disk data using at-rest encryption keys and fine-grained access control. Broad capabilities include object
Synchronizes local database state with a remote server to keep datasets consistent across devices.
Zotero is reference management software designed for collecting, organizing, and citing bibliographic research sources and digital documents for academic work. It functions as a web bibliographic collector, a citation generator, and a collaborative research platform. The system integrates tools for capturing metadata and archiving web pages into a centralized research library. It provides a specialized environment for reading and marking up PDF and EPUB files with highlights and notes linked directly to research sources. The software covers a broad range of capabilities including bibliograph
Syncs local database changes and attached files to a remote server for collaboration and multi-device access.
KeyDB is a multithreaded in-memory key-value store and distributed cache. It functions as a NoSQL database utilizing multi-version concurrency control to execute non-blocking queries and scans. The project is a multithreaded fork of Redis that maintains protocol compatibility while utilizing a multithreaded architecture to scale across multi-core hardware. It distinguishes itself with flash-tiered storage, allowing the system to offload data from primary RAM to SSD or flash storage to increase total capacity. The system supports high availability through active-active mesh replication and mu
Provides a mechanism to dump and load database snapshots directly to and from remote cloud storage buckets.
Grist is a relational spreadsheet platform that combines the flexibility of a spreadsheet with the power of a relational database. At its core, it manages structured data across multiple linked tables, using a relational database engine to organize information while providing a familiar grid interface. The platform supports Python-based formulas for complex calculations and data transformations, with automatic recalculation when referenced cells change. The system is designed for self-hosted deployment, storing data in either portable SQLite files or enterprise-grade PostgreSQL databases. It
Links to Google Drive using API credentials to manage remote file access.
Velero is a backup and recovery tool for Kubernetes cluster resources and persistent volumes. It functions as a disaster recovery solution and a utility for migrating applications and their associated data between different clusters. The project enables the replication of production environments by cloning cluster resources into development or testing environments for validation and debugging. It provides capabilities for backing up system objects, restoring resources to a known good state, and transferring applications across environments to facilitate system transitions.
Saves Kubernetes resource manifests and volume snapshots as serialized files in remote cloud object storage buckets.
Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates. The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework t
Hydrates data into edge workers or durable objects to enable low-latency processing at the network edge.
immudb is a tamperproof database that maintains an immutable record of entries using cryptographic commit logging. It ensures verifiable database integrity by utilizing Merkle trees to generate membership and consistency proofs that detect unauthorized data alterations. The system employs a multi-model storage engine that unifies key-value, document, and relational data structures within a single immutable backend. It provides compatibility with the PostgreSQL wire protocol, allowing it to integrate with standard SQL clients, ORMs, and database tools. The project covers broad capabilities in
Replicates database state to third-party cloud storage providers to enable remote storage for ephemeral deployments.
gdrive is a command line interface client for managing files, folders, and permissions within Google Drive. It functions as a backup utility and synchronization tool, allowing users to interact with their cloud storage through a terminal instead of a web browser. The project supports server-to-server integration by using service accounts and JSON credential files. This enables non-interactive identity verification and session establishment without requiring interactive user login prompts. The tool covers bidirectional and one-way synchronization between local directories and remote storage,
Acts as a backup utility by uploading local data and tracking file revisions for secure remote storage.
Floccus is a browser extension that synchronizes bookmarks across different browsers and devices. It runs directly within the browser to access the native bookmark API, and offers both one-way and two-way sync modes to control how changes flow between devices. The extension includes an optional end-to-end encryption layer that encrypts bookmark data client-side before transmission, ensuring only the user can decrypt it on other devices. The extension supports multiple storage backends through a unified sync interface, including WebDAV, Google Drive, and Nextcloud, allowing users to choose bet
Synchronizes bookmarks using Google Drive as the storage backend, with an option to encrypt the data.
KubeEdge is a distributed edge computing framework that extends Kubernetes to manage containerized workloads and hardware devices at the edge. It functions as a Kubernetes edge orchestration system, allowing the deployment and management of applications across distributed edge nodes using native Kubernetes APIs and workflows. The project distinguishes itself through a specialized focus on IoT integration and node autonomy. It employs digital-twin state modeling to represent physical hardware devices as virtual objects, utilizing an MQTT-based messaging bus for communication with heterogeneous
Provides mechanisms for replicating resource states and metadata between edge devices and centralized cloud storage.
Noms is a distributed version control database and content-addressable data store. It identifies data by cryptographic hashes to ensure integrity and deduplication, while tracking dataset state changes through a sequence of immutable commits to enable branching, forking, and historical recovery. The system functions as a peer-to-peer data synchronizer, reconciling state between disconnected database instances to ensure all nodes converge on the same data. It distinguishes itself as a schema-flexible document store that supports self-describing types, allowing schemas to evolve and widen as ne
Exports database snapshots to remote cloud object storage buckets for recovery and durability.
This is a Golang client library for interacting with a cloud native distributed messaging system. It provides the necessary tools for Go applications to exchange messages using publish-subscribe and request-reply patterns, as well as specialized clients for managing persistent streams and distributed storage. The library includes a JetStream client for durable message streaming and replay, a Key-Value store client for managing distributed state with versioning and watchers, and an Object Store client for the storage and retrieval of large binary files via chunked delivery. The implementation
Proxies local client traffic through edge servers to route messages from edge nodes to cloud controllers.
Dropbox-Uploader is a Bash-based command-line client for managing files on Dropbox. It provides a full set of file operations including upload, download, copy, move, rename, delete, directory creation, and listing, all through shell scripts that use curl to interact with the Dropbox API. The tool authenticates via OAuth tokens and includes a chunked upload mechanism for handling large files reliably. The project distinguishes itself with an interactive shell interface that lets users browse and manage remote files without re-entering commands, and a file-name-based password derivation system
Compresses a local directory into an encrypted archive and uploads it to a remote cloud storage service.
ZeroByte is a backup management platform built around the Restic backup engine, providing encrypted, deduplicated, and compressed snapshots across multiple storage backends. It offers a web interface for scheduling, monitoring, and managing backup operations, with support for cron-based job scheduling and configurable retention policies that automatically prune older snapshots. The platform distinguishes itself through comprehensive multi-protocol volume mounting, allowing backup ingestion from NFS, SMB, WebDAV, SFTP, and rclone-backed sources alongside local directories. It includes a snapsh
Supports backing up to S3, Google Cloud Storage, Azure Blob, and 40+ rclone backends for remote storage.
Exports and saves note data automatically to cloud storage services like Dropbox or Google Drive.
Airweave is a unified AI knowledge base platform that syncs data from external APIs into a searchable layer for retrieval-augmented generation. It provides a pre-built data connector library and a framework for building custom connectors, enabling the extraction, transformation, and synchronization of structured and unstructured data from SaaS applications. The platform includes a hybrid vector retrieval system that combines semantic, neural, and keyword search strategies to deliver grounded context for AI agents. The platform distinguishes itself through an agentic search engine that iterati
Polls the Google Drive changes feed to incrementally sync only modified or new presentations.