Awesome GitHub RepositoriesData Warehousing

ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring. The platform distinguishes itself through ad

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

Doris is a distributed SQL data warehouse designed for high-performance analytical workloads and real-time data processing. It functions as a unified platform that integrates traditional relational warehousing with lakehouse query capabilities, allowing users to execute analytical operations directly against external data lakes without requiring data migration. The system distinguishes itself through a shared-nothing, massively parallel processing architecture that utilizes vectorized query execution and columnar storage to maintain sub-second latency. It supports dynamic schema evolution, en

Databend is a cloud-native data warehouse and OLAP database designed for large-scale analytics. It functions as a SQL-compliant engine and serverless analytics platform that separates compute from storage to allow for independent scaling. The system integrates vector database capabilities, indexing high-dimensional embeddings to enable semantic, hybrid, and full-text searches across massive datasets. It further distinguishes itself through serverless compute management that automatically scales resources based on demand and shuts them down during idle periods. The platform covers a broad set

Connect is a Kafka data integration platform and stream processing engine used to build declarative pipelines that move and transform messages between Kafka topics and external sources. It functions as a Kafka Connect framework and a change data capture tool, streaming real-time database modifications to synchronize data across distributed environments. The project differentiates itself through a dedicated mapping language for mutating and reshaping message payloads and the ability to execute custom processing logic within a sandboxed WebAssembly runtime. It also provides an observability pip

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

JanusGraph is a distributed, elastically scalable graph database designed to store and query highly connected data across a cluster of machines. It supports the property graph data model with ACID consistency and integrates multi-model search capabilities including geo, numeric range, and full-text queries. The database also includes a Graph OLAP engine for running batch analytics and global graph computations on large datasets using the Hadoop framework. The project distinguishes itself through a masterless cluster architecture that eliminates single points of failure, allowing every node to

HBase este un magazin NoSQL distribuit de tip wide-column și un motor de stocare big data conceput pentru seturi de date rare (sparse). Funcționează ca o bază de date coloanară scalabilă construită deasupra Hadoop Distributed File System pentru a oferi acces în timp real la citirea și scrierea unor volume masive de date structurate și nestructurate. Sistemul acționează ca un gateway de bază de date cross-language, oferind conectivitate prin apeluri de procedură remote native, REST și interfețe Thrift. Se distinge printr-un model de coordonare master-worker care permite scalarea orizontală și toleranța la erori într-un cluster. Proiectul acoperă un set larg de capabilități, inclusiv controlul accesului granular prin etichete de vizibilitate la nivel de celulă, compresia datelor pluggable și agregarea datelor pe partea de server. De asemenea, suportă fluxuri de lucru de analiză big data prin integrarea map-reduce și permite execuția logicii personalizate pe partea de server. Monitorizarea operațională este oferită prin urmărirea metricilor de sistem și exportul de metrici bazat pe plugin-uri.

Side-Menu.Android este o componentă UI reutilizabilă pentru aplicațiile Android care oferă un meniu de navigare de tip slide-out. Este concepută pentru a ajuta dezvoltatorii să organizeze secțiunile aplicației și opțiunile utilizatorului într-un panou structurat, ascuns, care menține o interfață curată pentru zona de conținut principal. Componenta se distinge prin prezentarea sa vizuală, care urmează ghidurile Material Design pentru a asigura o experiență de utilizator consistentă și intuitivă. Dispune de o ierarhie de meniu bazată pe date care permite gruparea logică a elementelor de navigare și încorporează animații fluide de tip circular reveal pentru a oferi tranziții vizuale rafinate atunci când meniul este deschis sau închis. Prin încapsularea logicii complexe de layout și interacțiune într-o singură clasă modulară, biblioteca simplifică implementarea navigării pe mai multe ecrane. Suportă tranziții bazate pe evenimente, permițând dezvoltatorilor să decupleze interacțiunile din meniu de actualizările de conținut pentru a menține o arhitectură de aplicație curată și responsivă.

Kylin is a distributed OLAP engine designed for executing fast SQL queries on massive datasets. It utilizes multi-dimensional data cubes to pre-calculate data aggregates, enabling sub-second response times for large-scale analytical queries and big data analytics. The system focuses on large-scale data warehousing and multi-dimensional data modeling. It allows for the organization and querying of vast amounts of structured data to support business intelligence and reporting workflows through distributed SQL querying.

tech-vault is a command-line technical interview bank and knowledge base designed for practicing engineering questions across various technical domains. It functions as a terminal-based application that stores structured study materials and interview questions as markdown files, which are then rendered directly within the system console. The project distinguishes itself through a delivery model that uses command-line argument parsing to filter content by topic or difficulty. It also includes a random selection algorithm to pick individual questions from the collection for spontaneous study se

OpenAddresses is an open-source geospatial data aggregator and directory that collects public domain and open-license address, parcel, and building datasets from governments and organizations worldwide. It functions as a global index and data warehouse for locating and distributing free geospatial records. The project operates a normalization pipeline that cleans and standardizes diverse source formats into a consistent global coordinate and attribute schema. This process includes a crowdsourced curation pipeline and programmatic quality validation to verify the spatial accuracy and formattin

cve-search is a vulnerability search engine and database manager designed to index, synchronize, and query CVE and CPE security vulnerability data. It functions as a security data warehouse that imports vulnerability feeds into a local database to enable fast, keyword-based discovery of security flaws. The project provides a web-based vulnerability browser and a programmatic JSON API for retrieving records and risk scores. It utilizes full-text indexing for vulnerability descriptions and implements an identity-verified security portal using the OpenID Connect standard for user authentication.