awesome-repositories.com
Blog
awesome-repositories.com

Entdecke die besten Open-Source-Repositories mit KI-gestützter Suche.

EntdeckenKuratierte SuchenOpen-Source-AlternativenSelf-hosted SoftwareBlogSitemap
ProjektÜber unsRanking-MethodikPresseMCP-Server
RechtlichesDatenschutzAGB
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

7 Repos

Awesome GitHub RepositoriesIncremental Sync

Streaming only the changes occurring after a connection is established, omitting snapshots.

Distinct from Change Data Capture Streams: Focuses on the omission of initial state in favor of delta-only streaming

Explore 7 awesome GitHub repositories matching data & databases · Incremental Sync. Refine with filters or upvote what's useful.

Awesome Incremental Sync GitHub Repositories

Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
  • nangohq/nangoAvatar von NangoHQ

    NangoHQ/nango

    10,772Auf GitHub ansehen↗

    Nango is an open-source platform that connects applications to external APIs by managing authentication, data synchronization, and custom function execution. It provides a managed runtime for TypeScript integration functions, handling OAuth flows, credential storage, and token refresh for hundreds of external APIs while keeping secrets isolated from application code. The platform distinguishes itself by exposing integration functions as discoverable tools for AI agents through an MCP server or API, with per-user credential isolation that keeps provider secrets out of the agent loop. It offers

    Runs recurring sync functions that paginate APIs, save checkpoints, and emit changed records.

    TypeScriptaccess-tokenapiapi-client
    Auf GitHub ansehen↗10,772
  • electric-sql/electricAvatar von electric-sql

    electric-sql/electric

    9,909Auf GitHub ansehen↗

    Electric is a Postgres data synchronization engine and replication proxy designed to enable local-first software. It replicates data from Postgres databases to client-side stores in real time using logical replication, allowing applications to maintain a local embedded database for offline access and low-latency updates. The system distinguishes itself by using shapes to filter and authorize specific subsets of database rows and columns before streaming them to clients or edge workers. It further supports multi-user collaboration by integrating a conflict-free replicated data type framework t

    Provides a mode to omit the initial data snapshot and stream only new changes after connection.

    Elixircrdtcrdtselixir
    Auf GitHub ansehen↗9,909
  • openmediavault/openmediavaultAvatar von openmediavault

    openmediavault/openmediavault

    6,802Auf GitHub ansehen↗

    OpenMediaVault is a Debian-based network attached storage (NAS) operating system that turns a computer into a dedicated storage appliance, managed entirely through a browser-based control panel. It provides a unified configuration interface for managing file sharing protocols, RAID arrays, user permissions, and network services, with extensibility through a modular plugin system that adds new capabilities without modifying the core installation. The platform supports setup and management of multiple RAID levels for data redundancy or performance, and exposes file storage over protocols includ

    Schedules recurring file transfers between local and remote systems using RSync for backup or mirroring tasks.

    PHPdebiannas
    Auf GitHub ansehen↗6,802
  • airweave-ai/airweaveAvatar von airweave-ai

    airweave-ai/airweave

    6,453Auf GitHub ansehen↗

    Airweave is a unified AI knowledge base platform that syncs data from external APIs into a searchable layer for retrieval-augmented generation. It provides a pre-built data connector library and a framework for building custom connectors, enabling the extraction, transformation, and synchronization of structured and unstructured data from SaaS applications. The platform includes a hybrid vector retrieval system that combines semantic, neural, and keyword search strategies to deliver grounded context for AI agents. The platform distinguishes itself through an agentic search engine that iterati

    Configures automatic refresh schedules for collections of synced data sources.

    Pythonagent-infrastructureaiai-agents
    Auf GitHub ansehen↗6,453
  • weiye-jing/datax-webAvatar von WeiYe-Jing

    WeiYe-Jing/datax-web

    6,009Auf GitHub ansehen↗

    DataX Web is a web-based management platform for scheduling, building, executing, and monitoring distributed data synchronization jobs powered by DataX. It provides a visual console for creating and managing DataX tasks without manual JSON configuration, with a distributed executor cluster that auto-registers worker nodes and supports configurable routing and blocking strategies for task distribution. The platform offers cron-based task scheduling with dynamic start, stop, and immediate status changes, along with incremental sync capabilities that pass dynamic parameters to extract only new o

    Automates periodic extraction of only new or changed records from source databases using dynamic parameter injection.

    Java
    Auf GitHub ansehen↗6,009
  • justinwalsh/daux.ioAvatar von justinwalsh

    justinwalsh/daux.io

    4,530Auf GitHub ansehen↗

    Dieses Projekt ist ein Markdown-Static-Site-Generator und Dokumentationsserver, der Dateien und Ordnerstrukturen in responsive Websites umwandelt. Es fungiert als Confluence-Dokumentations-Publisher und ermöglicht das direkte Hochladen und Synchronisieren von verarbeiteten Markdown-Inhalten auf Enterprise-Knowledge-Base-Server. Das System zeichnet sich durch seine technischen Rendering-Fähigkeiten aus, indem es Mermaid-Syntax in visuelle Diagramme und TeX-Ausdrücke in mathematische Formeln umwandelt. Es unterstützt mehrsprachige Dokumentation durch die Erstellung lokalisierter Versionen einer Website mit dedizierten Navigationsstrukturen. Die Plattform bietet eine vollständige Content-Pipeline, die hierarchische Navigationsmenüs, Code-Syntax-Highlighting und ein System zur Verwaltung der Seitenreihenfolge über Dateinamen-Präfixe umfasst. Zudem verfügt sie über ein flexibles Erweiterungs-Framework für benutzerdefinierte Content-Generatoren und die programmatische Manipulation von Seitenbäumen. Die Umgebung kann als portabler Docker-Container bereitgestellt werden und enthält einen lokalen Preview-Server zum Testen der Dokumentation vor der finalen Generierung.

    Optimizes updates by synchronizing only the changes that occur between the source and destination.

    Auf GitHub ansehen↗4,530
  • kananinirav/aws-certified-cloud-practitioner-notesAvatar von kananinirav

    kananinirav/AWS-Certified-Cloud-Practitioner-Notes

    3,829Auf GitHub ansehen↗

    This project is a collection of structured study notes and conceptual breakdowns designed for the AWS Certified Cloud Practitioner exam. It serves as a technical reference and study guide, organizing cloud service details and architectural principles to assist in certification preparation. The knowledge base is built using markdown files and includes curated cheat sheets and interactive mind-map visualizations. These tools map complex certification topics into visual hierarchies to enable drill-down study paths and rapid revision. The materials cover a wide range of cloud capabilities, inclu

    Covers scheduled sync engines used to move and synchronize large volumes of data from on-premises environments.

    HTMLamazon-web-servicesawsaws-certified-cloud-practitioner
    Auf GitHub ansehen↗3,829
  1. Home
  2. Data & Databases
  3. Change Data Capture Streams
  4. Incremental Sync

Unter-Tags erkunden

  • Dynamic Parameter InjectionsPasses runtime field values and intervals to sync tasks so only new or changed records are extracted each run. **Distinct from Incremental Sync:** Distinct from Incremental Sync: focuses on the mechanism of injecting dynamic parameters into tasks, not the general delta-only streaming concept.
  • Scheduled Incremental Sync ConfigurationsDefines parent-child task dependencies and configures incremental sync with auto-calculated date ranges for scheduled runs. **Distinct from Incremental Sync:** Distinct from Incremental Sync: adds task dependency chaining and auto-calculated date ranges for scheduled execution, not just delta-only streaming.
  • Scheduled Sync Engines2 Sub-TagsRecurring sync functions that paginate external APIs, save checkpoints, and emit only changed records. **Distinct from Incremental Sync:** Distinct from Incremental Sync: adds scheduling and checkpoint-based resumption for recurring runs.