How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
Netflix's distributed Data Pipeline
The main features of netflix/suro are: Data Ingestion, Data Ingestion and Integration, Data Ingestion Pipelines, Data Pipelines, Data Processing and Analytics.
Projects with overlapping indexed features include: gazette/core — Build platforms that flexibly mix SQL, batch, and stream processing paradigms. rudderlabs/rudder-server — Rudder Server is a customer data platform and event routing pipeline designed to collect, transform, and route… bruin-data/ingestr — ingestr is a command-line tool for copying and syncing data between different database engines and third-party… bruin-data/bruin — Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end… linkedin/gobblin — A distributed data integration framework that simplifies common aspects of big data integration such as data… aklivity/zilla — 🦎 A multi-protocol edge & service proxy. Seamlessly interface web apps, IoT clients, & microservices to Apache Kafka®…
Build platforms that flexibly mix SQL, batch, and stream processing paradigms
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
ingestr is a command-line tool for copying and syncing data between different database engines and third-party platforms without writing custom code. It functions as an ETL pipeline utility that extracts data from diverse sources and loads it into destinations. The tool features a schema-agnostic data loader that maps source fields to destination columns dynamically, removing the need for predefined static table definitions. It also operates as an incremental data synchronizer, updating destination tables by appending new records or merging changes to maintain current datasets. The system pr
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.