Awesome GitHub RepositoriesData Warehousing

ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring. The platform distinguishes itself through ad

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

Doris is a distributed SQL data warehouse designed for high-performance analytical workloads and real-time data processing. It functions as a unified platform that integrates traditional relational warehousing with lakehouse query capabilities, allowing users to execute analytical operations directly against external data lakes without requiring data migration. The system distinguishes itself through a shared-nothing, massively parallel processing architecture that utilizes vectorized query execution and columnar storage to maintain sub-second latency. It supports dynamic schema evolution, en

Databend is a cloud-native data warehouse and OLAP database designed for large-scale analytics. It functions as a SQL-compliant engine and serverless analytics platform that separates compute from storage to allow for independent scaling. The system integrates vector database capabilities, indexing high-dimensional embeddings to enable semantic, hybrid, and full-text searches across massive datasets. It further distinguishes itself through serverless compute management that automatically scales resources based on demand and shuts them down during idle periods. The platform covers a broad set

Connect is a Kafka data integration platform and stream processing engine used to build declarative pipelines that move and transform messages between Kafka topics and external sources. It functions as a Kafka Connect framework and a change data capture tool, streaming real-time database modifications to synchronize data across distributed environments. The project differentiates itself through a dedicated mapping language for mutating and reshaping message payloads and the ability to execute custom processing logic within a sandboxed WebAssembly runtime. It also provides an observability pip

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

JanusGraph is a distributed, elastically scalable graph database designed to store and query highly connected data across a cluster of machines. It supports the property graph data model with ACID consistency and integrates multi-model search capabilities including geo, numeric range, and full-text queries. The database also includes a Graph OLAP engine for running batch analytics and global graph computations on large datasets using the Hadoop framework. The project distinguishes itself through a masterless cluster architecture that eliminates single points of failure, allowing every node to

HBase एक वितरित, वाइड-कॉलम NoSQL स्टोर और बिग डेटा स्टोरेज इंजन है जिसे स्पार्स डेटासेट के लिए डिज़ाइन किया गया है। यह विशाल मात्रा में स्ट्रक्चर्ड और अनस्ट्रक्चर्ड डेटा तक रीयल-टाइम रीड और राइट एक्सेस प्रदान करने के लिए Hadoop Distributed File System के ऊपर निर्मित एक स्केलेबल कॉलम-आधारित डेटाबेस के रूप में कार्य करता है। सिस्टम एक क्रॉस-लैंग्वेज डेटाबेस गेटवे के रूप में कार्य करता है, जो नेटिव रिमोट प्रोसीजर कॉल्स, REST और Thrift इंटरफेस के माध्यम से कनेक्टिविटी प्रदान करता है। यह एक मास्टर-वर्कर समन्वय मॉडल के माध्यम से खुद को अलग करता है जो एक क्लस्टर में क्षैतिज स्केलिंग और फॉल्ट टॉलरेंस को सक्षम बनाता है। प्रोजेक्ट सेल-लेवल विजिबिलिटी लेबल्स, प्लगेबल डेटा कम्प्रेशन और सर्वर-साइड डेटा एग्रीगेशन के माध्यम से फाइन-ग्रेन्ड एक्सेस कंट्रोल सहित क्षमताओं के एक व्यापक सेट को कवर करता है। यह मैप-रिड्यूस एकीकरण के माध्यम से बिग डेटा एनालिटिक्स वर्कफ़्लो का भी समर्थन करता है और कस्टम सर्वर-साइड लॉजिक के निष्पादन की अनुमति देता है। ऑपरेशनल मॉनिटरिंग सिस्टम मेट्रिक ट्रैकिंग और प्लगइन-आधारित मेट्रिक एक्सपोर्टिंग के माध्यम से प्रदान की जाती है।

Side-Menu.Android, Android एप्लिकेशन्स के लिए एक पुन: प्रयोज्य UI कंपोनेंट है जो स्लाइड-आउट नेविगेशन ड्रॉअर प्रदान करता है। इसे डेवलपर्स को एप्लिकेशन सेक्शन्स और यूज़र ऑप्शन्स को एक स्ट्रक्चर्ड, छिपे हुए पैनल में व्यवस्थित करने में मदद करने के लिए डिज़ाइन किया गया है, जो प्राथमिक कंटेंट क्षेत्र के लिए एक साफ़ इंटरफ़ेस बनाए रखता है। यह कंपोनेंट अपने विज़ुअल प्रेजेंटेशन के माध्यम से अलग है, जो सुसंगत और सहज यूज़र अनुभव सुनिश्चित करने के लिए Material Design दिशानिर्देशों का पालन करता है। इसमें एक डेटा-ड्रिवन मेनू पदानुक्रम है जो नेविगेशन आइटम्स के तार्किक समूहीकरण की अनुमति देता है, और इसमें मेनू खोले या बंद किए जाने पर पॉलिश विज़ुअल ट्रांज़िशन प्रदान करने के लिए फ्लूइड सर्कुलर रिवील एनिमेशन शामिल हैं। जटिल लेआउट और इंटरैक्शन लॉजिक को एक ही मॉड्यूलर क्लास में एनकैप्सुलेट करके, यह लाइब्रेरी कई स्क्रीन्स पर नेविगेशन के कार्यान्वयन को सरल बनाती है।

Kylin is a distributed OLAP engine designed for executing fast SQL queries on massive datasets. It utilizes multi-dimensional data cubes to pre-calculate data aggregates, enabling sub-second response times for large-scale analytical queries and big data analytics. The system focuses on large-scale data warehousing and multi-dimensional data modeling. It allows for the organization and querying of vast amounts of structured data to support business intelligence and reporting workflows through distributed SQL querying.

tech-vault is a command-line technical interview bank and knowledge base designed for practicing engineering questions across various technical domains. It functions as a terminal-based application that stores structured study materials and interview questions as markdown files, which are then rendered directly within the system console. The project distinguishes itself through a delivery model that uses command-line argument parsing to filter content by topic or difficulty. It also includes a random selection algorithm to pick individual questions from the collection for spontaneous study se

OpenAddresses is an open-source geospatial data aggregator and directory that collects public domain and open-license address, parcel, and building datasets from governments and organizations worldwide. It functions as a global index and data warehouse for locating and distributing free geospatial records. The project operates a normalization pipeline that cleans and standardizes diverse source formats into a consistent global coordinate and attribute schema. This process includes a crowdsourced curation pipeline and programmatic quality validation to verify the spatial accuracy and formattin

cve-search is a vulnerability search engine and database manager designed to index, synchronize, and query CVE and CPE security vulnerability data. It functions as a security data warehouse that imports vulnerability feeds into a local database to enable fast, keyword-based discovery of security flaws. The project provides a web-based vulnerability browser and a programmatic JSON API for retrieving records and risk scores. It utilizes full-text indexing for vulnerability descriptions and implements an identity-verified security portal using the OpenID Connect standard for user authentication.