How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
Iceberg is an open table format and big data table manager designed for huge analytic datasets in cloud storage. It provides a specification for tracking large-scale datasets to maintain transactional consistency and structural integrity. The project utilizes a standardized REST catalog interface to manage table metadata, ensuring interoperability between different compute engines. This allows diverse query engines to connect to a single table interface and maintain consistency across different processing frameworks. Its core capabilities include managing large-scale analytic tables, coordin
Kylin is a distributed OLAP engine designed for executing fast SQL queries on massive datasets. It utilizes multi-dimensional data cubes to pre-calculate data aggregates, enabling sub-second response times for large-scale analytical queries and big data analytics. The system focuses on large-scale data warehousing and multi-dimensional data modeling. It allows for the organization and querying of vast amounts of structured data to support business intelligence and reporting workflows through distributed SQL querying.
This repository is a collection of reference implementations and distributed data processing algorithms implemented in Java and Scala for cluster computing frameworks. It provides computational recipes for solving complex data processing problems, including large-scale dataset joins, aggregations, and word count tasks. The implementations cover both MapReduce paradigms and Apache Spark integrations, enabling programmatic job submission and execution across distributed node infrastructures. The collection includes specialized utilities for statistical analysis and text processing, such as data
Stellar Core is the primary software implementation of the Stellar blockchain network, serving as a distributed ledger and a Federated Byzantine Agreement system. It functions as a core node that maintains the shared state of the network and provides a runtime environment for executing WebAssembly smart contracts. The project enables the creation and management of digital assets, including the implementation of decentralized exchanges through distributed orderbooks and automated liquidity pools. It facilitates cross-border payment settlement by routing assets via path payments and bridging di
A Scala API for Apache Beam and Google Cloud Dataflow.
The main features of spotify/scio are: Big Data, Big Data and Analytics.
Projects with overlapping indexed features include: apache/iceberg — Iceberg is an open table format and big data table manager designed for huge analytic datasets in cloud storage. It… apache/kylin — Kylin is a distributed OLAP engine designed for executing fast SQL queries on massive datasets. It utilizes… stellar/stellar-core — Stellar Core is the primary software implementation of the Stellar blockchain network, serving as a distributed ledger… mahmoudparsian/data-algorithms-book — This repository is a collection of reference implementations and distributed data processing algorithms implemented in… apache/flink — Apache Flink is a distributed processing engine designed for both high-throughput, low-latency data streams and finite… apache/kafka — Kafka is a distributed event streaming platform designed for capturing, storing, and processing real-time data streams…