24 个仓库
Populates new tables from query results with configurable storage properties.
Distinct from Virtual Table Querying: Distinct from Virtual Table Querying: focuses on persistent table creation rather than virtual view aggregation.
Explore 24 awesome GitHub repositories matching data & databases · Table Creation. Refine with filters or upvote what's useful.
DataX is a distributed data integration framework and plugin-based ETL tool designed for synchronizing large datasets between heterogeneous sources and destinations. It functions as a JDBC data migration engine and offline synchronization tool, enabling the movement of data between relational databases, NoSQL stores, and object storage. The system utilizes a plugin-based connector architecture that decouples reader and writer logic, allowing it to map and transform data types across different storage engines using a standardized internal representation. This design supports heterogeneous data
Directs data ingestion into specific leaf-level partitions within a partitioned table structure.
Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Enables creating and populating new tables directly from query results with custom storage settings.
Perspective is a columnar data analytics engine and high-performance visualization component powered by WebAssembly. It provides a system for analyzing and visualizing large or streaming datasets through interactive data grids and charts, utilizing a compiled binary to achieve near-native performance within the browser. The project distinguishes itself through a WebSocket-based data streaming interface and deep Apache Arrow integration, which minimize memory overhead when synchronizing tables between servers and clients. It acts as a remote query proxy capable of translating visualization con
Constructs transient in-memory tables from input data to serve as the foundation for real-time analysis.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Maintains an up-to-date, queryable view of external database tables by applying incoming change records to distributed maps.
Ibis is a portable Python dataframe library and multi-backend query engine that provides a unified interface for executing data transformations across diverse compute engines. It functions as a Python SQL expression compiler and dialect transpiler, allowing users to define data logic once and execute it across cloud warehouses, embedded databases, and distributed clusters without rewriting code. The project distinguishes itself through a database backend abstraction that decouples transformation logic from the underlying execution engine. It enables polyglot data workflows by mixing raw SQL s
Enables adding, dropping, and altering table partitions to optimize data organization and query performance.
This project is a streaming data integration framework that captures real-time database changes and synchronizes them with downstream systems. It operates as a distributed streaming ETL and database synchronizer, reading database logs and snapshots to propagate row-level modifications to target sinks. The system supports declarative data integration, allowing users to define source-to-sink data flows using SQL or YAML configurations. It distinguishes itself by automating schema evolution to maintain synchronization when source structures change and ensuring exactly-once delivery and processin
Defines new primary keys or partition keys for the downstream target table to optimize storage.
Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer
Provides a unified query interface that abstracts multiple physical tables, including those distributed across different clusters or storage locations.
Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac
Defines tables whose data is either managed by Hive or stored at a user-specified location.
GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without
Ships a DDL syntax for declaring observability concepts and lineage directly in CREATE TABLE statements.
CKEditor 4 is a browser-based WYSIWYG rich text editor that enables users to create and format HTML content directly in the browser. It operates on a plugin-based architecture with a configurable toolbar system, DOM-based content editing, and an event-driven lifecycle, all delivered through a CDN-based distribution model. The editor supports skin-based theming and includes a legacy plugin compatibility layer for backward compatibility. The editor distinguishes itself as a cross-platform framework that integrates natively with Angular, React, Vue, Electron, Android, and iOS environments. It of
Enables creating and nesting tables directly within the editor for structured content.
SqlSugar is an object-relational mapping library for .NET that translates C# and VB objects into database queries and tables without requiring raw SQL. It is designed as a multi-database ORM supporting SQL Server, MySQL, PostgreSQL, Oracle, MongoDB, ClickHouse, and other databases through a unified API, and it is compatible with .NET AOT compilation for native ahead-of-time deployment. The library distinguishes itself through high-speed bulk data operations that can insert or update millions of records in seconds using batch processing instead of row-by-row handling. It also provides multi-te
Automatically routes inserts and queries to the correct physical table partition based on date fields.
pg is a PostgreSQL object-relational mapper (ORM) for Go that maps Go structs to database tables and provides a fluent query builder for constructing SQL statements programmatically. At its core, it automatically generates CREATE TABLE statements from Go struct definitions using struct tags and naming conventions, and builds queries through method chaining with placeholder-based parameter binding to prevent SQL injection. The library distinguishes itself through relation-aware join generation that automatically constructs JOIN clauses for has-one, has-many, many-to-many, and polymorphic assoc
Creates parent tables with PARTITION BY clauses using struct tags.
AliSQL is a fork of MySQL by Alibaba that extends the relational database management system with enhancements for high performance, scalability, and enterprise-grade availability. It retains the core MySQL identity as a SQL-based database for storing, organizing, and retrieving structured data, while adding optimizations for large-scale transactional and analytical workloads. The project differentiates itself through a set of Alibaba-specific improvements, including a columnar engine for accelerating analytical queries directly on MySQL tables, and a distributed, shared-nothing NDB Cluster en
Divides tables into smaller physical segments based on a key to improve query performance and manageability.
Perfetto is a platform for system-level performance tracing and analysis on Linux and Android. It combines a high-throughput trace recorder, a SQL-based query engine, and a browser-based visualizer into a single toolchain. The platform covers CPU scheduling and call-stack profiling, native and Java heap memory allocation tracking, GPU and graphics events, and system-wide counters such as CPU frequency and power consumption. The architecture decouples trace recording from offline analysis, using a compact protobuf format for event encoding and columnar storage for efficient SQL queries. The we
Creates read-only tables from SQL queries optimized for analytic performance on trace data.
Blog.Core 是一个生产就绪的后端样板,用于使用 ASP.NET Core 构建企业级 API 和微服务。它为分布式系统提供了基础架构,包括用于数据库优先(database-first)脚手架和实现多租户 API 框架的工具。 该项目的特色在于自动数据层生成,可直接从数据库模式生成实体模型和存储库层。它使用标准身份服务器协议实现了集中式身份管理系统,以处理跨多个客户端和项目的身份验证和授权。 该框架涵盖了广泛的企业级功能,包括通过事件总线进行的异步消息处理、分布式内存缓存以及读写数据库流量分离。它结合了具有部门数据限制的基于角色的访问控制,并通过 API 性能分析和活动审计提供系统可观测性。 该系统还包括对用于推送通知的双向服务器-客户端通信、全文搜索集成以及集中式服务配置的支持。
Implements paginated queries and operations across key-based database table partitions to optimize performance.
Sequel is a relational database toolkit for Ruby that provides object-relational mapping, a fluent SQL query builder, and schema migration capabilities. It maps database tables to Ruby classes with support for associations, validations, lifecycle hooks, and eager loading, offering a comprehensive ORM layer for building data-centric applications. Sequel distinguishes itself through a plugin-based extension architecture that allows composable customization of models, databases, and datasets without relying on deep inheritance hierarchies. It includes a thread-safe connection pool with support f
Populates new tables from SELECT query results without explicit column type definitions.
me_cleaner 是一组专门的工具,用于解析闪存描述符、剥离固件二进制大对象(blob)以及配置用于管理和执行引擎的硬件级关机。它提供了分析 BIOS 内存转储、提取特定固件区域以及删除非必要二进制模块以减少系统交互面的实用程序。 该项目专门针对 Intel 管理引擎(Management Engine)和可信执行引擎(Trusted Execution Engine)固件映像的清理。这涉及删除二进制 blob 并修改配置位,以强制这些子系统在硬件初始化过程后自动关闭。 该工具集涵盖了固件修改功能,例如从工厂分区中删除压缩块、删除非基础分区以及重新计算分区表以保持映像完整性。
Updates internal offsets and sizes after removing firmware partitions to maintain image integrity.
Arroyo is a high-performance stream processing platform built in Rust. It executes continuous SQL queries on streaming data with event-time semantics, enabling accurate windowed aggregations, joins, and stateful computations on unbounded event streams. The platform uses native Rust execution for high throughput and low latency, with periodic checkpointing for exactly-once fault tolerance and horizontal scaling across distributed workers. The system integrates deeply with Kafka for reading and writing topics with exactly-once delivery and supports change data capture (CDC) from MySQL and Postg
Defines transient in-memory tables within a pipeline for intermediate state storage.
supabase-js 是一个全面的客户端库,旨在将前端应用程序与托管的后端即服务 (BaaS) 集成。它提供了一个统一的接口,用于与 PostgreSQL 数据库、身份管理系统、云对象存储和实时数据同步进行交互。 该库具有同构客户端设计,可在浏览器和服务器环境中运行。它通过类型安全的方法脱颖而出,利用 TypeScript 将数据库模式直接映射到客户端定义,并采用基于 PostgREST 的 API 将 JavaScript 调用转换为 RESTful 请求。 该客户端涵盖了广泛的功能,包括通过 OAuth、OIDC 和通行密钥 (passkeys) 进行用户身份验证,以及使用签名令牌进行会话管理。它通过 S3 兼容的存储接口管理大规模二进制数据,并通过基于 WebSocket 的订阅实现实时应用程序更新,以进行数据库更改和状态同步。其他功能包括无服务器边缘函数的调用以及使用向量嵌入执行相似性搜索。
Creates and manages Iceberg tables to optimize large-scale analytical queries and data processing.
TablePro is a cross-platform database management client designed for browsing, querying, and administering both SQL and NoSQL databases. It functions as a unified workspace that integrates a code-centric SQL editor with schema visualization tools, allowing developers to manage complex data models and execute queries across diverse database engines. The application distinguishes itself through an agentic AI integration layer that connects language models directly to database tools, enabling automated query generation, optimization, and error fixing with configurable approval gates. It features
Performs maintenance actions on data partitions, including optimizing tables and dropping or detaching partitions.