apache/hbase
Hbase
HBase este un magazin NoSQL distribuit de tip wide-column și un motor de stocare big data conceput pentru seturi de date rare (sparse). Funcționează ca o bază de date coloanară scalabilă construită deasupra Hadoop Distributed File System pentru a oferi acces în timp real la citirea și scrierea unor volume masive de date structurate și nestructurate.
Sistemul acționează ca un gateway de bază de date cross-language, oferind conectivitate prin apeluri de procedură remote native, REST și interfețe Thrift. Se distinge printr-un model de coordonare master-worker care permite scalarea orizontală și toleranța la erori într-un cluster.
Proiectul acoperă un set larg de capabilități, inclusiv controlul accesului granular prin etichete de vizibilitate la nivel de celulă, compresia datelor pluggable și agregarea datelor pe partea de server. De asemenea, suportă fluxuri de lucru de analiză big data prin integrarea map-reduce și permite execuția logicii personalizate pe partea de server.
Monitorizarea operațională este oferită prin urmărirea metricilor de sistem și exportul de metrici bazat pe plugin-uri.
Features
- Columnar Databases - Implements a distributed NoSQL wide-column store built on top of the Hadoop ecosystem for sparse datasets.
- Big Data Storage - Functions as a distributed engine for storing and querying massive volumes of structured and unstructured data.
- Column Family Management - Organizes sparse data into grouped column families for efficient distributed storage and retrieval.
- Hadoop - Integrates with the Hadoop Distributed File System to provide a columnar store for large-scale data analysis.
- Distributed File System Backends - Relies on the Hadoop Distributed File System for durable, replicated persistent storage of data files.
- Sparse Dataset Management - Provides scalable storage and versioning for massive, sparse, column-oriented datasets across a cluster.
- LSM-Tree Storage Engines - Utilizes an LSM-tree storage engine to provide high write throughput via in-memory buffering and sorted flushes.
- Region-Based Partitioning - Implements region-based partitioning by splitting the sorted keyspace into contiguous ranges for horizontal scaling.
- Wide-Column Stores - Organizes data into column families to provide real-time read and write access to high-scale datasets.
- Column-Oriented Disk Storage - Organizes sparse datasets into column-oriented disk storage for scalable, versioned data management.
- Distributed Data Stores - Provides a cluster-based storage system with horizontal scaling and fault tolerance for scalable data retrieval.
- Cell-Level Controls - Enforces fine-grained access control using visibility labels at the individual data cell level.
- Master-Worker Coordination - Employs a master-worker coordination model to manage cluster metadata and region assignments.
- Distributed Storage Clusters - Implements a scalable architecture that aggregates multiple nodes into a unified storage system for massive datasets.
- Distributed File Systems - Relies on a distributed file system like HDFS for durable and replicated storage of underlying data files.
- Big Data Processing - Supports big data processing workflows using map-reduce patterns for large-scale data transformation.
- Cross-Language Data Interfaces - Provides consistent data interaction interfaces via native RPC, REST, and Thrift APIs for clients in multiple programming languages.
- MapReduce Processing Engines - Integrates with MapReduce processing engines to transform and migrate large volumes of data between tables.
- Server-Side Aggregations - Calculates summaries and statistics directly on the server to minimize data transfer to the client.
- Application REST API Gateways - Exposes database operations and cluster status through a standardized REST API gateway.
- Thrift RPC Servers - Ships a dedicated Thrift server to enable cross-language connectivity for database operations.
- Cross-Language Service Gateways - Acts as an entry point that translates REST, Thrift, and RPC requests into internal database protocols.
- Storage Block Compression - Applies pluggable block compression to reduce the physical storage footprint of datasets on disk.
- Multi-Protocol Communication Bridges - Provides a multi-protocol gateway allowing clients to connect via RPC, HTTP, and Thrift.
- Remote Procedure Calls - Uses remote procedure calls for low-latency communication between clients, master nodes, and region servers.
- Remote Procedure Call Protocols - Implements structured messaging protocols for standardized communication between cluster nodes and clients.
- Database Systems - Distributed big data store modeled after Bigtable.
