Awesome GitHub RepositoriesData Warehousing

ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring. The platform distinguishes itself through ad

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

Doris is a distributed SQL data warehouse designed for high-performance analytical workloads and real-time data processing. It functions as a unified platform that integrates traditional relational warehousing with lakehouse query capabilities, allowing users to execute analytical operations directly against external data lakes without requiring data migration. The system distinguishes itself through a shared-nothing, massively parallel processing architecture that utilizes vectorized query execution and columnar storage to maintain sub-second latency. It supports dynamic schema evolution, en

Databend is a cloud-native data warehouse and OLAP database designed for large-scale analytics. It functions as a SQL-compliant engine and serverless analytics platform that separates compute from storage to allow for independent scaling. The system integrates vector database capabilities, indexing high-dimensional embeddings to enable semantic, hybrid, and full-text searches across massive datasets. It further distinguishes itself through serverless compute management that automatically scales resources based on demand and shuts them down during idle periods. The platform covers a broad set

Connect is a Kafka data integration platform and stream processing engine used to build declarative pipelines that move and transform messages between Kafka topics and external sources. It functions as a Kafka Connect framework and a change data capture tool, streaming real-time database modifications to synchronize data across distributed environments. The project differentiates itself through a dedicated mapping language for mutating and reshaping message payloads and the ability to execute custom processing logic within a sandboxed WebAssembly runtime. It also provides an observability pip

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

JanusGraph is a distributed, elastically scalable graph database designed to store and query highly connected data across a cluster of machines. It supports the property graph data model with ACID consistency and integrates multi-model search capabilities including geo, numeric range, and full-text queries. The database also includes a Graph OLAP engine for running batch analytics and global graph computations on large datasets using the Hadoop framework. The project distinguishes itself through a masterless cluster architecture that eliminates single points of failure, allowing every node to

HBase est un magasin NoSQL distribué à colonnes larges et un moteur de stockage de big data conçu pour les jeux de données creux. Il fonctionne comme une base de données colonnaire évolutive construite au-dessus du Hadoop Distributed File System pour fournir un accès en lecture et écriture en temps réel à des volumes massifs de données structurées et non structurées. Le système agit comme une passerelle de base de données multi-langage, offrant une connectivité via des appels de procédure distante (RPC) natifs, REST et des interfaces Thrift. Il se distingue par un modèle de coordination maître-esclave qui permet une mise à l'échelle horizontale et une tolérance aux pannes à travers un cluster. Le projet couvre un large ensemble de capacités, notamment le contrôle d'accès granulaire via des étiquettes de visibilité au niveau de la cellule, la compression de données enfichable et l'agrégation de données côté serveur. Il supporte également les workflows d'analyse de big data via l'intégration map-reduce et permet l'exécution de logique personnalisée côté serveur. La surveillance opérationnelle est fournie via le suivi des métriques système et l'exportation de métriques basée sur des plugins.

Side-Menu.Android est un composant d'interface utilisateur réutilisable pour les applications Android qui fournit un tiroir de navigation coulissant. Il est conçu pour aider les développeurs à organiser les sections de l'application et les options utilisateur dans un panneau structuré et masqué qui maintient une interface propre pour la zone de contenu principale. Le composant se distingue par sa présentation visuelle, qui suit les directives Material Design pour garantir une expérience utilisateur cohérente et intuitive. Il dispose d'une hiérarchie de menu pilotée par les données qui permet un regroupement logique des éléments de navigation, et il intègre des animations de révélation circulaire fluides pour fournir des transitions visuelles polies lorsque le menu est ouvert ou fermé. En encapsulant une logique de mise en page et d'interaction complexe dans une seule classe modulaire, la bibliothèque simplifie l'implémentation de la navigation sur plusieurs écrans. Elle prend en charge les transitions pilotées par les événements, permettant aux développeurs de découpler les interactions de menu des mises à jour de contenu pour maintenir une architecture d'application propre et réactive.

Kylin is a distributed OLAP engine designed for executing fast SQL queries on massive datasets. It utilizes multi-dimensional data cubes to pre-calculate data aggregates, enabling sub-second response times for large-scale analytical queries and big data analytics. The system focuses on large-scale data warehousing and multi-dimensional data modeling. It allows for the organization and querying of vast amounts of structured data to support business intelligence and reporting workflows through distributed SQL querying.

tech-vault is a command-line technical interview bank and knowledge base designed for practicing engineering questions across various technical domains. It functions as a terminal-based application that stores structured study materials and interview questions as markdown files, which are then rendered directly within the system console. The project distinguishes itself through a delivery model that uses command-line argument parsing to filter content by topic or difficulty. It also includes a random selection algorithm to pick individual questions from the collection for spontaneous study se

OpenAddresses is an open-source geospatial data aggregator and directory that collects public domain and open-license address, parcel, and building datasets from governments and organizations worldwide. It functions as a global index and data warehouse for locating and distributing free geospatial records. The project operates a normalization pipeline that cleans and standardizes diverse source formats into a consistent global coordinate and attribute schema. This process includes a crowdsourced curation pipeline and programmatic quality validation to verify the spatial accuracy and formattin

cve-search is a vulnerability search engine and database manager designed to index, synchronize, and query CVE and CPE security vulnerability data. It functions as a security data warehouse that imports vulnerability feeds into a local database to enable fast, keyword-based discovery of security flaws. The project provides a web-based vulnerability browser and a programmatic JSON API for retrieving records and risk scores. It utilizes full-text indexing for vulnerability descriptions and implements an identity-verified security portal using the OpenID Connect standard for user authentication.