8 dépôts
Fast synchronization techniques using recent state snapshots to bootstrap nodes.
Distinguishing note: Focuses on fast-sync via snapshots rather than full re-execution.
Explore 8 awesome GitHub repositories matching data & databases · Snapshot Synchronization. Refine with filters or upvote what's useful.
This project is a containerized implementation of a blockchain node for the Base network. It manages the execution and consensus processes of an Ethereum Virtual Machine node within an isolated environment to maintain the network and process blockchain data. The infrastructure includes a specialized interface for processing pending blocks via websockets to reduce latency. It also provides a state synchronization tool that uses pre-computed snapshots to accelerate the process of reaching the current block height. The system covers network identity configuration for mainnet and testnet environ
Uses snapshot-based synchronization to quickly bootstrap the node and reach the current block height.
Geth is a comprehensive execution client for the Ethereum network, serving as a foundational node implementation that processes transactions, maintains the distributed ledger state, and participates in peer-to-peer consensus. It provides a robust infrastructure for synchronizing, validating, and serving blockchain data, utilizing a persistent Merkle Patricia Trie database to ensure the cryptographic integrity of historical records. As a sandboxed smart contract runtime, it executes bytecode according to deterministic protocol rules, enabling the deployment and interaction of decentralized appl
Downloads recent state snapshots to enable fast synchronization from network checkpoints.
This project is a containerized node orchestrator and deployment tool designed to manage execution clients and rollup nodes on a blockchain network. It provides a coordinated stack of isolated virtual environments to establish and maintain network connections. The system includes specialized data provisioning tools for initializing local directories and fetching verified archival snapshots to bypass sequential synchronization. It also features a monitoring suite with health check services and dashboards to track synchronization progress and overall system performance. The orchestrator covers
Enables fast node bootstrapping by downloading verified archival state snapshots to bypass sequential sync.
This project is a Go language library that provides a programmatic interface for interacting with the Kubernetes API server. It serves as a client for managing cluster resources, offering both typed interfaces for compile-time safety and dynamic interfaces for unstructured data and custom resource management. The library includes a controller framework designed for building event-driven automation. This framework utilizes informers to maintain local resource caches and rate-limited work queues to decouple event detection from state reconciliation. High availability is supported through a lead
Combines initial snapshots with a continuous stream of changes to maintain a local cache of resources.
This is a Raft consensus library and distributed consensus engine implemented in Go. It provides the primitives necessary to build fault-tolerant distributed services by implementing a replicated state machine that ensures a group of servers agree on a shared system state through leader election and log replication. The project distinguishes itself through a pluggable architecture for storage backends and snapshot storage, decoupling the consensus logic from physical persistence. It includes specialized mechanisms for leadership transfer, protocol version management to support rolling upgrade
Implements fast synchronization of nodes using state snapshots to bootstrap the cluster state.
This project is a streaming data integration framework that captures real-time database changes and synchronizes them with downstream systems. It operates as a distributed streaming ETL and database synchronizer, reading database logs and snapshots to propagate row-level modifications to target sinks. The system supports declarative data integration, allowing users to define source-to-sink data flows using SQL or YAML configurations. It distinguishes itself by automating schema evolution to maintain synchronization when source structures change and ensuring exactly-once delivery and processin
Captures historical data using snapshots and transitions to real-time capture to bootstrap synchronization.
Reth is a modular, high-performance Ethereum execution layer client written in Rust. It serves as a full Ethereum node that syncs, validates, and serves blockchain data, functioning as an archive node implementation, a high-throughput RPC node server, and a snapshot sync tool. The project is built around a modular component architecture that allows assembling custom node behavior by swapping independent Rust crates for consensus, execution, mempool, and networking. The client distinguishes itself through a staged sync pipeline that downloads headers and bodies online before processing the res
Bootstraps nodes by fetching compressed snapshot files from remote sources instead of syncing from genesis.
Mutagen is a cross-platform file synchronization engine and development environment orchestrator. It provides real-time bidirectional file synchronization between local machines, remote servers, and Docker containers, often utilizing SSH for transport. The tool functions as a high-performance alternative to Docker bind mounts by mirroring files between a local host and a running container. It also operates as a network port forwarding utility, routing TCP traffic and Unix domain sockets between local and remote environments via SSH. Its capability surface includes filesystem event monitoring
Generates a representation of files in a root using scans or watching to identify changes.