Open-source alternatives to Awesome Splunk

A curated list of awesome Qlik extensions and resources for Qlik Sense and QlikView

a curated list of awesome streaming frameworks, applications, etc

A curated list of amazingly awesome Hadoop and Hadoop ecosystem resources

Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.

Elasticsearch权威指南中文版

Scala library for accessing various file, batch systems, job schedulers and grid middlewares.

This project is a curated directory of software, frameworks, and educational resources designed for building, scaling, and maintaining distributed data processing and storage architectures. It serves as a comprehensive index for the distributed computing ecosystem, helping users identify the appropriate tools for managing large-scale information systems. The repository functions as a central hub for data engineering, offering categorized access to technologies that support batch and stream processing, machine learning, and interactive querying. By organizing these resources, it assists in the

A curated list of awesome pipeline toolkits inspired by Awesome Sysadmin

A Scala API for Apache Beam and Google Cloud Dataflow.

Low-code tool for automating actions on real time data | Stream processing for the users.

A schema-aware Scala library for data transformation

Apache Flink is a distributed processing engine designed for both high-throughput, low-latency data streams and finite batch workloads. It functions as a stateful stream processor and a SQL stream processing engine, providing a unified runtime to execute relational queries and event-based transformations. The system is distinguished by its ability to manage persistent operator state to ensure exactly-once processing guarantees and consistency during failures. It features specialized capabilities for complex event processing to detect temporal patterns and handles out-of-order events using eve

Kafka is a distributed event streaming platform designed for capturing, storing, and processing real-time data streams across interconnected nodes. It functions as a distributed commit log, providing a fault-tolerant storage mechanism that records state changes sequentially to ensure data consistency and durability across distributed environments. The platform distinguishes itself through a partitioned commit log architecture that enables horizontal scaling and parallel processing of data streams. It integrates a stream processing engine for continuous transformations and aggregations, while

Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation graphs. It functions as a distributed machine learning framework, a graph processing system, a real-time stream processor, and a SQL analytics engine. The system enables the execution of distributed SQL querying, large-scale graph analysis, and real-time stream analytics across clusters of machines. It also provides a scalable environment for implementing machine learning algorithms and predictive model development on massive datasets. The engine incorporates relational query e

This project is a collection of specialized study guides and roadmaps centered on computer science, data engineering, and machine learning fundamentals. It provides a structured curriculum of technical competencies, tools, and skills required to transition into professional data engineering roles. The project features a data engineering skill map that visually organizes databases, processing architectures, and infrastructure tools. It also includes a machine learning learning path covering supervised and unsupervised learning techniques alongside model operations. The curriculum covers broad