18 个仓库
Mechanisms for grouping multiple computational queries into a single execution call to improve throughput.
Distinct from Batch Matrix Multiplication Utilities: Distinct from Batch Matrix Multiplication Utilities: focuses on general task/query batching rather than specific linear algebra operations.
Explore 18 awesome GitHub repositories matching data & databases · Query Batching. Refine with filters or upvote what's useful.
TanStack Table is a headless, framework-agnostic engine designed for building complex data grids and managing tabular state. By decoupling data processing logic from the visual rendering layer, it allows developers to implement custom user interfaces while offloading sophisticated operations like sorting, filtering, grouping, and pagination to a unified, performant core. The library distinguishes itself through its commitment to type safety and environment flexibility. It leverages strict type definitions to ensure data integrity across the entire application and utilizes an adapter pattern t
Groups individual operations into batches to improve system efficiency and reduce processing overhead.
Hystrix is a latency and fault tolerance library designed to prevent cascading failures in distributed systems. It functions as a circuit breaker implementation that monitors failure thresholds and opens circuits to isolate remote calls when downstream services degrade. The project distinguishes itself by providing multiple isolation mechanisms, utilizing dedicated thread pools and semaphores to ensure that latency in one dependency does not saturate the entire system. It also features a request collapsing and batching engine that groups concurrent calls into single executions to reduce the t
Runs multiple remote calls concurrently while caching identical requests and collapsing them into a single batch.
Apollo Client is a GraphQL client library and data fetching framework used to request data from a GraphQL server and synchronize that state within a frontend application. It functions as a remote state manager and a local state management tool, allowing developers to define client-side schemas and resolvers for data that does not reside on a remote server. The project features a normalized GraphQL cache that identifies objects by ID to ensure referential equality and consistent data updates across different queries. It also includes a GraphQL API mocking tool to simulate server responses and
Combines multiple GraphQL operations into a single network request to reduce overhead and round trips.
Horovod is a distributed deep learning framework designed to scale machine learning training across multiple GPUs and nodes. It functions as an orchestrator for multi-GPU scaling and a tool for distributed gradient averaging, allowing users to increase compute capacity without rewriting core model logic. The project provides a consistent communication interface that supports multi-framework model distribution across TensorFlow, PyTorch, Keras, and MXNet. It leverages an MPI distributed training library to synchronize gradients across processes using collective communication operations. The s
Groups multiple small gradient updates into a single large buffer to reduce network communication frequency.
Horovod is a distributed deep learning framework and gradient synchronizer designed to scale model training across multiple GPUs and compute nodes. It functions as a distributed training orchestrator and an elastic training engine, utilizing an MPI collective communication library to synchronize weights and gradients across TensorFlow, PyTorch, Keras, and MXNet models. The system distinguishes itself through dynamic elastic scaling, which allows it to adjust the number of active workers at runtime and recover from node failures. It optimizes communication efficiency using tensor fusion batchi
Groups multiple small tensors into larger buffers to reduce network overhead during gradient synchronization.
Bytebase is a database DevSecOps platform and management console designed to orchestrate schema migrations, deployments, and security audits across multiple database engines. It serves as a SQL GitOps tool that synchronizes database states with configurations stored in Git repositories to manage infrastructure as code. The platform distinguishes itself through a multi-database management console that provides a single interface for relational and NoSQL databases. It includes a security layer for role-based access control, database activity auditing, and column-level data masking to protect se
Pushes schema or data updates to multiple databases and tenants simultaneously using a centralized interface.
Dask 是一个并行计算框架和分布式任务调度器,旨在将 Python 数据科学工作流从单机扩展到大型集群。它作为一个集群资源管理器,通过将任务及其依赖项表示为有向无环图来编排计算逻辑。这种架构允许系统在管理复杂执行要求的同时,自动将工作负载分配到可用硬件上。 该项目通过一个延迟评估引擎脱颖而出,该引擎将数据操作推迟到明确请求时才执行,从而实现全局图优化和高效的资源分配。它结合了内存感知数据溢出功能,以防止在处理超过可用内存的数据集时系统崩溃,并利用任务图融合将操作序列组合成单个执行步骤,从而最大限度地减少调度开销和节点间通信。 该平台为大规模数据分析提供了全面的功能面,包括对分布式机器学习、高性能计算集成和并行数据处理的支持。它提供了用于集群生命周期管理、性能分析和任务执行实时监控的广泛工具。用户可以在各种基础设施上部署这些环境,包括本地硬件、云提供商、容器化系统和高性能计算集群。
Groups multiple queries into a single execution call to enable parallel processing and reuse of shared intermediate results.
Falcor is a JavaScript library that models remote data as a single virtual JSON graph, providing a path-based query engine for efficient client-side data retrieval and updates. It represents multiple remote data sources as a unified document where entities are accessed via globally unique identity paths. The system distinguishes itself by treating the remote data model as a virtual JSON resource, allowing the client to query specific paths without managing individual endpoints. It uses a reference-aware graph model to handle many-to-many relationships and prevents data duplication. Network ef
Merges multiple concurrent requests within a time window into a single network call to eliminate redundancy.
GraphQL.NET 是一个用于在 C# 应用程序中构建和执行 GraphQL API 的服务端框架。它提供了一套全面的工具包,用于模式构建、分布式数据图的联合引擎,以及用于管理实时数据流的订阅处理器。 该项目的独特之处在于其灵活的模式构建器,支持程序化的代码优先定义和使用标准模式定义语言的声明式模式优先方法。它包括一个专门的联合引擎,用于将数据图拆分为子图并将其组合成统一的网关,以及一个专门设计用于通过批处理和缓存解决 N+1 查询问题的数据加载器实现。 该框架涵盖了广泛的操作功能,包括用于服务生命周期管理的依赖注入集成、用于字段解析拦截的中间件流水线,以及针对值类型优化以减少内存分配的执行流水线。它还提供用于查询复杂度分析、文档缓存和基于角色的访问控制的工具,以保护 API 端点。 对提前(AOT)模式编译的支持允许该框架在禁止动态代码生成的环境中执行。
Implements the DataLoader pattern to batch requests and cache results, eliminating N+1 query problems.
HotChocolate is a GraphQL server implementation for .NET that handles queries, mutations, and subscriptions over HTTP. It provides code-first schema generation directly from C# classes and attributes, keeping the schema and business logic in sync without manual type definitions. The project also includes a GraphQL gateway that composes multiple GraphQL and REST endpoints into a single unified schema, along with a GraphQL IDE for exploring, testing, and debugging GraphQL APIs interactively. The project distinguishes itself through its comprehensive tooling ecosystem, which includes a GraphQL c
Implements DataLoader-based batching to eliminate N+1 query problems in GraphQL resolvers.
GraphQL Platform is a comprehensive GraphQL ecosystem for .NET that provides a spec-compliant server framework for building APIs and gateways, along with a typed C# client for consuming GraphQL services. At its core, it translates C# classes and methods into a GraphQL schema using code-first or fluent descriptor approaches, enabling developers to define their API structure directly from their existing .NET code. The platform distinguishes itself through several integrated capabilities that address common GraphQL production concerns. It includes a DataLoader batching and caching engine that gr
Ships a DataLoader batching engine that groups data fetching requests by key to solve the N+1 problem in resolvers.
GraphQL-Ruby is a Ruby library for building GraphQL APIs with a strongly typed schema and a dedicated query execution engine. It provides a comprehensive framework for mapping application objects to a formal type system, enabling structured data fetching through defined resolvers. The project distinguishes itself with advanced performance and delivery mechanisms, including a data loader for batching and caching to prevent N+1 query patterns. It supports high-performance data delivery through incremental response streaming, deferred query responses, and parallel data fetching using fibers. Add
Provides a DataLoader engine to batch data fetching requests and eliminate N+1 query problems.
obs-websocket 是一个用于 OBS Studio 的 WebSocket 远程控制插件和扩展。它作为一个 JSON-RPC 网络 API,允许外部设备和应用程序管理软件设置、场景和流媒体操作。 该项目提供了一个标准化的网络协议,支持远程命令执行和事件驱动的状态同步。它使用加盐挑战身份验证来保护这些连接,以验证客户端身份。 该接口涵盖了广泛的生产功能,包括场景和源管理、音频电平控制以及用于录制和流媒体的输出管理。它还支持系统统计查询和软件状态监控,以促进实时广播监控和工作流自动化。
Provides the ability to group multiple network commands into a single request for efficient execution.
LMQL 是一种编程语言和概率接口,将算法逻辑与随机文本生成相结合。它作为一个约束引导的提示框架和结构化输出生成器,允许用户强制模型响应遵循严格的格式和数据类型。 该系统作为推理优化器,通过专门的执行策略(包括基于树的提示缓存和异步批处理)提高了 Token 吞吐量并降低了延迟。 该项目涵盖了广泛的生成控制功能,例如基于 Logit 的 Token 掩码、束搜索选择以及交错逻辑提示执行。它还为通过 WebSockets 和服务器发送事件 (SSE) 进行实时响应流式传输提供了基础设施。
Groups multiple computational queries into single execution calls to increase token throughput and reduce latency.
该项目是一个全栈 Monorepo 样板和 Node.js GraphQL API 模板。它提供了一个种子项目,在一个仓库中集成了模式优先的 GraphQL API、PostgreSQL 数据库和 React 前端,以便在后端和用户界面之间共享类型。 该架构包含一个用于云部署和自动扩缩容的无服务器包装器。它实现了一个请求批处理层,通过防止 N+1 查询问题来优化数据库性能。 该项目通过基于迁移的模式版本控制和种子文件来管理数据库。它还包括用户身份集成功能,涵盖身份验证、授权和基于权限的访问控制。
Implements a request batching layer using DataLoader to eliminate N+1 query problems in GraphQL resolvers.
Apollo Android 是一个强类型的 GraphQL 客户端库,用于在 JVM、Android 和 Kotlin Multiplatform 上执行查询和变更。它包含一个代码生成工具,可从 GraphQL 模式和操作文件中创建不可变类和解析器,以确保编译时的类型安全。 该项目具有一个规范化缓存,将响应扁平化为查找表以维护单一事实来源。它还提供了一个订阅客户端,用于通过 WebSocket 或 SSE 连接进行实时服务器端数据更新,以及一个用于模拟服务器响应和生成测试用假类型安全数据的 Mock 框架。 该库涵盖了广泛的功能,包括通过请求批处理和持久化查询进行网络优化,以及通过乐观更新和分页管理本地数据状态。它支持集成自定义 HTTP 引擎,并将自定义标量类型映射到原生语言类。 该工具与开发 IDE 集成,提供从编辑器到定义的自动代码生成和导航功能。
Groups multiple GraphQL operations into a single network request to reduce round-trip latency.
Lightly is a self-supervised learning framework and computer vision data curation tool designed to manage large image datasets and train models on unlabeled data. It functions as a PyTorch vision library and dataset management SDK, providing tools to convert raw images into high-dimensional vectors for similarity search, visualization, and feature extraction. The project implements a variety of self-supervised architectures, including MoCo, SimCLR, VICReg, Barlow Twins, and masked image modeling. It distinguishes itself by combining these learning frameworks with active learning capabilities,
Groups multiple augmented views of the same image into separate tensors within a single batch.
MadelineProto is an asynchronous PHP library that provides a programmatic interface for interacting with the Telegram API using the MTProto protocol, the same protocol used by official Telegram clients. It functions as both a Telegram bot SDK and a userbot automation library, enabling PHP applications to connect to Telegram as either a bot account or a regular user account, sending and receiving messages, media, and other data directly without relying on the Bot API intermediary. The library is built on an event-driven architecture with Amp v3 fiber-based concurrency, allowing for non-blockin
Queues multiple concurrent Telegram API calls into a single container for efficient batch processing.