awesome-repositories.com
Blog
awesome-repositories.com

Entdecke die besten Open-Source-Repositories mit KI-gestützter Suche.

EntdeckenKuratierte SuchenOpen-Source-AlternativenSelf-hosted SoftwareBlogSitemap
ProjektÜber unsRanking-MethodikPresseMCP-Server
RechtlichesDatenschutzAGB
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
apache avatar

apache/hive

0
View on GitHub↗
6,012 Stars·4,791 Forks·Java·apache-2.0·8 Aufrufehive.apache.org↗

Hive

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interface for submitting Hive, MapReduce, and Pig jobs and managing HCatalog metadata.

Hive distinguishes itself through its multi-engine query execution, allowing queries to run on Apache Spark, Tez, or MapReduce to balance performance and resource usage across different workloads. It supports external data federation, enabling direct querying of remote databases, Druid, HBase, and Iceberg tables without moving data. Enterprise security integration provides authentication via Kerberos, LDAP, SAML, JWT, or OAuth2, with fine-grained access control through Apache Ranger. The cost-based optimizer, materialized views, and LLAP persistent daemon work together to deliver sub-second query responses on large datasets.

The platform offers comprehensive data management capabilities including ACID transactions, multiple storage formats such as ORC, Parquet, Avro, and RCFile, and support for cloud storage on S3, Azure Data Lake, and Google Cloud Storage. It includes a pluggable SerDe abstraction layer for custom data formats and a storage handler interface for connecting to external systems like HBase, Druid, Kudu, and JDBC sources. Advanced SQL features cover windowed aggregation, grouping sets, common table expressions, and geospatial calculations, while extensibility is provided through user-defined functions, custom MapReduce scripts, and procedural SQL execution.

Hive can be deployed via stable release tarballs, Docker containers, or Amazon EMR, and includes command-line tools like Beeline and HCatalog for interactive and batch query execution. Monitoring and observability features allow inspection of query execution plans, job status tracking, and runtime metrics viewing.

Features

  • Hadoop - Provides a SQL-on-Hadoop data warehouse for querying petabytes of distributed data.
  • Distributed SQL Querying - Executes SQL queries against petabytes of data in distributed storage for batch analytics.
  • Hadoop Data Warehouses - Provides a SQL-on-Hadoop data warehouse that queries and manages petabytes of data stored in distributed storage.
  • Thrift-Based Query Executions - Provides a Thrift-based service for executing SQL queries from remote clients with authentication.

KI-Suche

Entdecke weitere awesome Repositories

Beschreibe in einfachen Worten, was du brauchst — die KI bewertet tausende kuratierte Open-Source-Projekte nach Relevanz.

Start searching with AI
  • Concurrent Read-Write Transactions - Executes concurrent read and write transactions on tables with optimistic or pessimistic locking and lockless reads.
  • Data Lake Transaction Managers - Implements an ACID transaction manager for concurrent reads and writes on ORC tables.
  • ORC Transaction Managers - Provides ACID transactions on ORC tables for concurrent read-write operations.
  • Columnar Storage Engines - Uses the Optimized Row Columnar (ORC) format to improve read, write, and compression performance.
  • Metastore Integrations - Provides a central metadata repository serving table schemas to Hive, Spark, and Impala.
  • Hive Metastore Services - Provides a central metadata repository that stores table schemas and partition information for Hive and other data lake tools.
  • Cloud Native Object Storage - Natively supports S3, Azure Data Lake, and Google Cloud Storage for cloud-native data warehousing.
  • Automatic Partition Insertions - Automatically creates partitions during data insertion, eliminating manual partition management.
  • Partition Key Filters - Restricts query scans to specific partitions using predicates on partition key columns.
  • Data Serialization Formats - Converts data between storage and in-memory representations using SerDes for multiple formats.
  • SQL Data Loading and Transformation - Loads data into tables and transforms it using INSERT, UPDATE, DELETE, and scripting operations.
  • SQL Data Analysis - Provides a SQL interface for ad-hoc querying, summarization, and analysis of large datasets.
  • Table Schemas - Creates, renames, adds columns to, and drops tables and partitions while preserving schema.
  • Tez-Based Query Executions - Executes SQL queries using the Tez framework for lower latency than MapReduce.
  • Database Object Management - Creates, alters, drops, and describes databases, tables, views, functions, indexes, and partitions.
  • Data Lake Table Metadata Stores - Stores and serves table and partition metadata via a service API for Hive, Spark, Impala, and other data lake tools.
  • SQL Query Interfaces - Queries files stored in HDFS, HBase, or other storage systems through a unified SQL interface.
  • Federated Data Query Engines - Links Hive to remote databases via data connectors for federated query execution across disparate sources.
  • File-Based Data Import - Imports data from HDFS files into Hive tables using external table definitions or load commands.
  • Key-Based Partitioning - Divides table data into storage units based on column values for faster query analysis.
  • Low-Latency Analytical Queries - Delivers sub-second query responses by reusing persistent query infrastructure and optimized data caches.
  • In-Memory Caching Engines - Provides an in-memory caching engine for sub-second query responses on large datasets.
  • Parquet Data Processing - Reads and writes data in the Parquet columnar format for efficient storage and querying.
  • Multi-Engine Execution Backends - Executes queries on Spark, Tez, or MapReduce to balance performance and resource usage.
  • Cost-Based Optimizers - Uses a cost-based optimizer with table statistics and materialized views for query planning.
  • Low-Latency Query Engines - Ships the LLAP interactive engine for sub-second query responses on large datasets.
  • Remote Table Connectivity - Maps remote databases or schemas to Hive databases, making all tables queryable without manual mapping.
  • Row Aggregations - Combines multiple input rows into a single output row, computing statistics like count, sum, average, and percentile.
  • Data Lake - Provides a central metastore for managing schemas across Hive, Spark, Impala, and other tools.
  • SQL Query Optimizations - Uses a cost-based optimizer to automatically improve SQL query performance and resource usage.
  • Managed and External Tables - Defines tables whose data is either managed by Hive or stored at a user-specified location.
  • Query Caching Daemons - Runs persistent daemon workers that cache data in memory for sub-second SQL query responses.
  • SQL Query Executions - Provides a REST endpoint for executing Hive queries and returning results.
  • WebHCat - Ships a WebHCat REST API for submitting Hive, MapReduce, and Pig jobs and managing HCatalog metadata.
  • Apache Thrift Integrations - Uses Apache Thrift for remote procedure calls to submit queries to a Hive instance.
  • Thrift RPC Servers - Implements a Thrift RPC server that accepts SQL queries and returns results over binary or HTTP transport.
  • Thrift Service Implementations - Implements a Thrift-based server that accepts SQL queries from remote clients with authentication.
  • Directed Acyclic Graph Execution Engines - Uses a DAG execution engine (Tez) for lower-latency interactive SQL queries on Hadoop.
  • Authorization Mode Managers - Restricts user actions on data and metadata through configurable authorization modes.
  • Enterprise Security Integrations - Integrates with Kerberos, LDAP, SAML, JWT, OAuth2, and Apache Ranger for enterprise-grade security.
  • Multi-Method API Authentication - Supports SAML, JWT, Kerberos, and LDAP authentication for HiveServer2 and the metastore.
  • Enterprise Access Controls - Integrates Kerberos, Apache Ranger, and Apache Atlas for enterprise-grade access control.
  • Schema-on-Read Formats - Applies a schema to various file formats like CSV, Parquet, and ORC, enabling SQL queries on unstructured data.
  • Table Bucketing - Distributes data across fixed buckets using hash of a column for efficient sampling and query pruning.
  • Window Ranking Functions - Applies analytic functions like RANK, ROW_NUMBER, and SUM over a defined window of rows.
  • BI Tool Integration - Provides JDBC and ODBC drivers that connect BI tools directly to Hive for querying.
  • RCFile Data Access - Stores and retrieves table data in the Record Columnar File format for Hive tables.
  • Druid Datasource Querying - Indexes data from Hive into Druid and runs SQL queries against Druid datasources directly.
  • Database Namespace Organization - Groups tables, views, and partitions into namespaces to avoid naming conflicts and enforce security boundaries.
  • Interactive - Executes HiveQL statements from a shell utility in both interactive and batch script modes.
  • Iceberg Table Lifecycle Management - Manages large analytic datasets in Iceberg format with snapshot isolation, branching, tagging, and full DML support.
  • Common Table Expressions - Defines a temporary result set within a WITH clause that can be referenced by a single SELECT, INSERT, or CREATE statement.
  • ACID Table Compaction Optimizations - Rebalances and prioritizes compaction requests for ACID and Iceberg tables to improve storage and query performance.
  • Continuous Data Imports - Streams small batches of records into an existing Hive partition or table, making them immediately visible to subsequent queries.
  • Custom MapReduce Script Embedding - Embeds user-provided mapper and reducer scripts within SQL queries using the TRANSFORM clause.
  • Storage Compaction Utilities - Rebalances and prioritizes compaction of ACID and Iceberg tables to optimize storage and query performance.
  • Catalog Integration - Connects to an Iceberg REST catalog secured with OAuth2 to manage tables with snapshots and branches.
  • Data Import and Export - Exports and imports table or partition data and metadata between Hive instances.
  • Spark Execution Backends - Supports running Hive queries on Apache Spark for accelerated performance.
  • S3 Querying - Declares tables over S3 data and runs SQL queries against it using EC2 compute resources.
  • Cross-Cluster Table Replication - Bootstraps and checkpoints replication of tables, including external tables, with tracking metrics for consistency.
  • Avro Table Reads and Writes - Reads and writes Avro-encoded data as Hive tables, inferring the table schema from the Avro schema and supporting nested structures.
  • Beeline Query Execution - Connects to HiveServer2 via the Beeline CLI client to run SQL queries interactively.
  • Map Join Optimizations - Automatically selects map joins for star-schema queries by loading smaller tables into memory as hash tables.
  • Predicate Pushdown Joins - Applies filter predicates to all joined tables to avoid scanning unnecessary partitions.
  • Date and Time Libraries - Provides built-in SQL functions for parsing, formatting, and extracting date and time components.
  • Disaster Recovery Solutions - Performs bootstrap and incremental replication for robust backup and business continuity.
  • SQL Geospatial Analysis - Processes spatial data using ESRI-based functions for geometry operations and analysis.
  • Multi-Dimensional Aggregations - Computes subtotals and grand totals across multiple dimensions using CUBE, ROLLUP, and GROUPING SETS operators.
  • Tez Execution Backends - Supports running queries on Apache Tez for lower-latency DAG-based execution.
  • Materialized Views - Pre-computes and stores summary results, then rewrites incoming queries to use those summaries for faster execution.
  • Pluggable Connector Frameworks - Provides a pluggable connector framework for mapping external systems like HBase and Druid to Hive tables.
  • Distributed Row Distributions - Orders or distributes rows across reducers using ORDER BY, SORT BY, DISTRIBUTE BY, or CLUSTER BY clauses.
  • Custom SQL Functions - Adds user-defined functions, aggregates, and table functions to SQL for custom data processing.
  • Custom Aggregation Function Writing - Extends built-in aggregation by implementing user-defined aggregation functions in Java.
  • Custom Scalar Function Writing - Extends built-in SQL functions by implementing user-defined scalar functions in Java.
  • Procedural SQL Execution - Executes procedural SQL code, including PL/SQL and T-SQL, on Hive and other engines.
  • Cost-Based Compilation Optimizations - Compiles SQL queries using materialized views, anti-joins, branch pruning, and histogram statistics for faster execution.
  • Pluggable Serialization Frameworks - Provides a pluggable SerDe layer that supports multiple storage formats like ORC, Parquet, and Avro.
  • Procedural SQL Execution - Executes procedural SQL code alongside standard queries for complex data transformations.
  • Query Result Persistence - Writes the output of a SELECT statement into another table, a local file, or a distributed filesystem directory.
  • Union All Result Combinations - Merges the result sets of multiple SELECT statements into a single result set using UNION.
  • Transactional Data Mutations - Inserts, updates, and deletes individual records in ACID tables without rewriting entire partitions.
  • Iceberg Table Ingestion - Supports Apache Iceberg tables natively via a storage handler for cloud-native, high-performance data lake architectures.
  • JDBC External Table Connectors - Creates external tables that read from any JDBC data source with predicate pushdown support.
  • Kudu Table Connectors - Provides a storage handler for querying Apache Kudu tables directly from Hive.
  • Table Function Invocation - Applies a user-defined table-generating function to each input row and joins the output rows to the original input.
  • SerDe-Based Format Plugins - Uses a custom SerDe to serialize and deserialize table data in any format for reading and writing.
  • Metadata REST Endpoints - Provides a REST API for managing table metadata and submitting Hadoop jobs.
  • Beeline Shell Connectivity - Provides a JDBC-based command-line shell for connecting to HiveServer2 in embedded or remote mode.
  • Compaction Worker Pools - Routes compaction requests to dedicated worker pools so high-priority tables are processed without waiting in the default queue.
  • JDBC Driver Loading - Ships a JDBC driver that enables external applications to connect and run queries against Hive.
  • Conditional Logic - Provides built-in SQL functions for conditional logic like CASE, IF, and NULL handling.
  • Automatic Query-Time Decompression - Imports text files compressed with Gzip or Bzip2 and decompresses them automatically during query execution.
  • SQL Type Conversions - Ships a comprehensive set of SQL type conversion functions for query-time data transformation.
  • SQL Mathematical Functions - Provides a wide range of mathematical functions callable directly within SQL queries.
  • Sampling Retrievals - Returns a random sample of rows from a table by selecting a specific bucket.
  • String Manipulation - Provides built-in SQL functions for string concatenation, extraction, encoding, and pattern matching.
  • HBase Table Connectors - Accesses HBase tables with HiveQL for both reading and writing data alongside native tables.
  • Multi-Table Insertions - Sends the results of a single query to several tables or HDFS files in one pass.
  • Big Data Storage - Data warehouse software for querying and managing large distributed datasets.
  • Data Engineering - Data warehouse software for managing large datasets.

    • Star-Verlauf

    Star-Verlauf für apache/hiveStar-Verlauf für apache/hive

    Open-Source-Alternativen zu Hive

    Ähnliche Open-Source-Projekte, sortiert nach der Anzahl der gemeinsamen Funktionen mit Hive.
    • apache/sparkAvatar von apache

      apache/spark

      43,467Auf GitHub ansehen↗

      Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation graphs. It functions as a distributed machine learning framework, a graph processing system, a real-time stream processor, and a SQL analytics engine. The system enables the execution of distributed SQL querying, large-scale graph analysis, and real-time stream analytics across clusters of machines. It also provides a scalable environment for implementing machine learning algorithms and predictive model development on massive datasets. The engine incorporates relational query e

      Scalabig-datajavajdbc
      Auf GitHub ansehen↗43,467
    • apache/pinotAvatar von apache

      apache/pinot

      6,098Auf GitHub ansehen↗

      Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

      Java
      Auf GitHub ansehen↗6,098
    • lancedb/lancedbAvatar von lancedb

      lancedb/lancedb

      9,031Auf GitHub ansehen↗

      LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters

      HTMLapproximate-nearest-neighbor-searchimage-searchnearest-neighbor-search
      Auf GitHub ansehen↗9,031
    • hazelcast/hazelcastAvatar von hazelcast

      hazelcast/hazelcast

      6,570Auf GitHub ansehen↗

      Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

      Javabig-datacachingdata-in-motion
      Auf GitHub ansehen↗6,570
    Alle 30 Alternativen zu Hive anzeigen→

    Häufig gestellte Fragen

    Was macht apache/hive?

    Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write…

    Was sind die Hauptfunktionen von apache/hive?

    Die Hauptfunktionen von apache/hive sind: Hadoop, Distributed SQL Querying, Hadoop Data Warehouses, Thrift-Based Query Executions, Concurrent Read-Write Transactions, Data Lake Transaction Managers, ORC Transaction Managers, Columnar Storage Engines.

    Welche Open-Source-Alternativen gibt es zu apache/hive?

    Open-Source-Alternativen zu apache/hive sind unter anderem: apache/spark — Apache Spark is a unified distributed data processing engine designed for large-scale data analysis and computation… apache/pinot — Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It… lancedb/lancedb — LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector… hazelcast/hazelcast — Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to… prestodb/presto — Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data… apache/kylin — Kylin is a distributed OLAP engine designed for executing fast SQL queries on massive datasets. It utilizes…