22 个仓库
Mathematical strategies for distributing data across partitions, such as hash-based or range-based.
Distinguishing note: Focuses on the algorithms used for partitioning, distinct from the general concept of partitioning.
Explore 22 awesome GitHub repositories matching data & databases · Partitioning Algorithms. Refine with filters or upvote what's useful.
This project is a comprehensive educational resource focused on the principles, patterns, and trade-offs required to design scalable, reliable, and high-performance distributed systems. It provides a structured curriculum that covers the fundamental architectural strategies necessary for building modern software infrastructure, ranging from high-level system decomposition to low-level networking and data management. The repository distinguishes itself by offering deep dives into complex architectural patterns, such as microservices-based decomposition, event-driven communication, and command-
Details strategies like hash-based partitioning for distributing data.
This project is a comprehensive collection of common computer science algorithms and data structures implemented in Swift. It serves as an educational reference and library for studying computational complexity, algorithmic logic, and data structure engineering through practical code examples. The repository provides a wide suite of data structure implementations, including various types of linked lists, heaps, hash tables, and an extensive range of hierarchical trees such as Red-Black, B-Tree, and Splay trees. It also covers diverse sorting and searching techniques, from basic bubble sort to
Implements efficient array partitioning around pivots to facilitate sorting operations.
This project is a comprehensive, community-maintained knowledge base and toolkit designed for competitive programming. It serves as a centralized repository for algorithmic theory, data structures, and mathematical techniques, providing a structured reference for informatics and collegiate programming competitions. The project distinguishes itself by integrating educational content with a robust suite of automation utilities. It provides a complete workflow for competitive programming, including tools for automated test case generation, solution verification, and direct interaction with onlin
Organizes elements by recursively partitioning a sequence around a pivot value to ensure sub-sequences are ordered.
TimescaleDB is an open-source PostgreSQL extension that adds native time-series capabilities to the database. At its core, it transforms standard PostgreSQL tables into hypertables—automatically partitioned by time intervals—so data is stored in fixed-size chunks without manual sharding. The extension includes a library of over 200 built-in SQL functions purpose-built for time-series workloads, such as time bucketing, gap filling, percentile estimation, and time-weighted averages. What distinguishes TimescaleDB from generic PostgreSQL is its set of integrated time-series features that work th
Splits large tables into smaller partitions based on time or a key to improve query performance.
This library is a collection of generic utilities for the Go programming language designed to simplify the manipulation of slices and maps. It provides a functional toolkit that enables developers to perform data transformations, such as filtering, mapping, and reducing, while maintaining strict type safety through the use of language-level generics. The project distinguishes itself by offering a dual approach to data processing that balances functional programming patterns with performance-oriented execution. It supports both immutable functional pipelines for predictable state transitions a
Provides utilities for grouping, chunking, and partitioning complex data collections.
Bullet3 is a professional physics simulation engine designed for calculating rigid body, soft body, and collision dynamics within 3D environments and robotics applications. It functions as a computational framework for determining complex geometric intersections and contact manifolds between objects in simulated space. The library distinguishes itself through a distributed rendering framework that scales heavy graphical workloads and scene generation tasks across large clusters of machines. This capability enables the production of massive datasets by distributing complex scene generation acr
Groups independent clusters of interacting objects into isolated islands to allow parallel processing of physics updates.
Dask 是一个并行计算框架和分布式任务调度器,旨在将 Python 数据科学工作流从单机扩展到大型集群。它作为一个集群资源管理器,通过将任务及其依赖项表示为有向无环图来编排计算逻辑。这种架构允许系统在管理复杂执行要求的同时,自动将工作负载分配到可用硬件上。 该项目通过一个延迟评估引擎脱颖而出,该引擎将数据操作推迟到明确请求时才执行,从而实现全局图优化和高效的资源分配。它结合了内存感知数据溢出功能,以防止在处理超过可用内存的数据集时系统崩溃,并利用任务图融合将操作序列组合成单个执行步骤,从而最大限度地减少调度开销和节点间通信。 该平台为大规模数据分析提供了全面的功能面,包括对分布式机器学习、高性能计算集成和并行数据处理的支持。它提供了用于集群生命周期管理、性能分析和任务执行实时监控的广泛工具。用户可以在各种基础设施上部署这些环境,包括本地硬件、云提供商、容器化系统和高性能计算集群。
Adjusts the number of output partitions during grouping operations to balance memory usage and computational efficiency.
Citus is a PostgreSQL extension that transforms a standard database into a distributed system. It functions as a sharding framework and distributed SQL engine, enabling horizontal scaling by partitioning tables across a cluster of nodes. By utilizing a coordinator-worker topology, the system manages metadata and routes queries to the appropriate nodes, allowing for parallel execution of complex operations across distributed data shards. The platform distinguishes itself through its specialized support for multi-tenant architectures and real-time analytical processing. It enables tenant-based
Automates the creation and removal of time-based table partitions using scheduled maintenance tasks to ensure continuous data ingestion and efficient retention.
Mac app that shows all open files, directories, sockets, pipes and devices in use by all running processes. Nice GUI for lsof.
Filters the displayed list by name, access mode, volume, type, location, or regular expression to narrow results.
pgloader is a command-line tool that automates the migration of data and schema from various source databases and file formats into PostgreSQL. It combines schema discovery, parallel data pipelines, and type casting into a single, declarative workflow, using PostgreSQL's COPY protocol for high-throughput bulk loading. The tool distinguishes itself by compiling a dedicated command language into concurrent reader-writer pipelines that handle schema introspection, data transformation, and error-resilient batch processing. It supports migrating entire databases from MySQL, MS SQL, SQLite, and Pos
Selectively includes or excludes tables from migration using name lists or regular expressions.
Implements spatial hashing and distance-based interest management to reduce network traffic in multiplayer games.
GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without
Lists tables in a database with optional name filtering for discovery.
CodeIgniter is a PHP web framework built on the Model-View-Controller pattern, designed for building full-stack web applications. It provides a lightweight toolkit with minimal configuration, organizing application logic into controllers, models, and views for clean separation of concerns. The framework includes a fluent query builder for constructing SQL statements programmatically, PSR-4 autoloading with namespace mapping, and a service-based dependency injection container for managing shared class instances. The framework distinguishes itself through its comprehensive set of built-in tools
Provides a utility method to enumerate all table names in the currently connected database.
Cortex is an open-source, horizontally scalable metrics platform that ingests, stores, and queries Prometheus-compatible time-series data with multi-tenant isolation. It accepts metrics via Prometheus remote write and OpenTelemetry, executes PromQL queries against both recent and historical data, and provides a Prometheus-compatible alerting and recording rule engine with an integrated Alertmanager. The system is built as a set of independently scalable microservices that use hash-ring-based sharding, gossip-based cluster membership, and tenant-aware object storage to distribute workloads acro
Divides timeseries across multiple sub-blocks using a hash of their labels for parallel compaction.
FastStream is an asynchronous Python framework designed for building event-driven microservices. It provides a unified abstraction layer for interacting with various message brokers, enabling developers to manage event production and consumption through a consistent interface while maintaining access to native provider-specific features. The framework centers on a decorator-based routing model that binds application logic directly to broker topics, supported by a built-in dependency injection container that resolves resources at runtime. The framework distinguishes itself through its deep int
Supports routing messages by partition keys to ensure data grouping and processing order.
This repository is a comprehensive collection of fully worked solutions to exercises and problems from the standard algorithms textbook by Cormen, Leiserson, Rivest, and Stein (CLRS). It serves as an educational reference for algorithm design and analysis, providing step-by-step reasoning, pseudocode, and mathematical proofs for a wide range of topics. The content spans core computer science areas: algorithm analysis with asymptotic notation, recurrence solving, and amortized cost analysis; data structure implementation and operations for binary search trees, red-black trees, B-trees, Fibonac
Illustrates the quicksort partition subroutine including pivot selection and array rearrangement.
Jitsu 是一个客户数据平台,旨在收集、转换并将应用程序事件路由到数据仓库和营销工具。它作为一个事件摄取引擎和数据仓库路由器,通过 API 和 SDK 捕获行为数据以进行实时处理和存储。 该平台具有可编程的 JavaScript 数据流水线,允许在传输过程中对事件数据进行过滤、丰富和重塑。它包含一个客户身份拼接器,可合并匿名和已知的用户标识符,以在仓库中维护持久的客户画像。 系统涵盖了广泛的功能,包括来自 Web 和移动环境的多源事件收集、目标仓库的自动模式演进,以及到 SaaS 平台和 SQL 数据库的多目标路由。它为开发人员提供了一套用于测试转换逻辑的工具,并支持通过 Kubernetes 或自托管 Docker 环境进行部署。
Organizes records into time-based segments using a timestamp column to speed up analytical queries and reduce data scanned during lookups.
GAM is a command-line tool for administering Google Workspace and Cloud Identity. It translates command-line arguments into structured API calls, enabling administrators to manage users, groups, organizational units, and domain settings across a Google Workspace environment. The tool handles authentication through OAuth2 flows, service accounts, and workload identity federation, and supports multi-tenant configurations for managing multiple domains or cloud projects from a single installation. GAM distinguishes itself through its batch processing and automation capabilities. It can process la
Filters users, groups, files, or messages using regular expressions based on name or email.
This repository is a curated guide and implementation library of coding patterns used to solve data structures and algorithms problems. It serves as a technical interview study resource, providing a comprehensive set of strategies and computational logic examples for optimizing time and space complexity. The project focuses on standardized algorithmic patterns, including sliding windows, two pointers, and dynamic programming. It features specific implementations for a wide range of challenges, such as LeetCode problem solutions and specialized techniques like cyclic sort and bitwise XOR opera
Implements array partitioning that organizes three distinct values into separate zones in a single pass.
Screw 是一个数据库模式分析工具,用于自动生成技术文档和 Java 数据对象。它扫描数据库表以导出其定义,将模式元数据转换为结构化报告和应用程序代码。 该项目根据现有的数据库表结构生成普通 Java 对象 (POJO) 以实现数据建模自动化。它还生成 HTML、Word 和 Markdown 格式的数据库模式文档。 该系统包括使用名称模式进行表导出过滤、基于 JDBC 的模式提取,以及将数据库列类型映射到 Java 语言类型的模板化代码生成功能。
Provides the ability to include or exclude specific database tables from the export process using name patterns.