awesome-repositories.com
博客
awesome-repositories.com

通过 AI 驱动的搜索,发现最优秀的开源仓库。

探索精选搜索开源替代品自托管软件博客网站地图
项目关于排名机制媒体报道MCP 服务器
法律隐私政策服务条款
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

24 个仓库

Awesome GitHub RepositoriesRequest Deduplication

Techniques to collapse multiple concurrent network requests for the same resource.

Distinguishing note: Focuses on request collapsing, not general caching.

Explore 24 awesome GitHub repositories matching web development · Request Deduplication. Refine with filters or upvote what's useful.

Awesome Request Deduplication GitHub Repositories

用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
  • vercel/swrvercel 的头像

    vercel/swr

    32,406在 GitHub 上查看↗

    SWR is a data fetching library that provides a collection of hooks for managing remote data synchronization, caching, and state updates in web applications. It employs a declarative approach to handle complex network request lifecycles and dependency chains, ensuring that client-side application state remains consistent with server data through automatic revalidation and background updates. The library distinguishes itself through a reactive cache layer that automatically synchronizes local state with remote sources based on component lifecycle events. It features event-driven revalidation, w

    Collapses multiple concurrent requests for the same resource into a single network call using unique keys.

    TypeScriptcachedatadata-fetching
    在 GitHub 上查看↗32,406
  • vectordotdev/vectorvectordotdev 的头像

    vectordotdev/vector

    22,071在 GitHub 上查看↗

    Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

    Eliminates redundant logs or metrics by comparing incoming data against a configurable cache of recent events.

    Rusteventsforwarderhacktoberfest
    在 GitHub 上查看↗22,071
  • cube-js/cubecube-js 的头像

    cube-js/cube

    20,251在 GitHub 上查看↗

    Cube is a semantic data layer that provides a unified framework for defining business metrics, dimensions, and relationships across diverse data sources. By acting as a headless business intelligence engine, it transforms raw data into a governed model that can be queried via SQL, REST, and GraphQL interfaces. This architecture ensures consistent data definitions and logic across all downstream analytical applications and reporting tools. The platform distinguishes itself through its integrated conversational AI capabilities, which allow users to explore data using natural language. It orches

    Protects data sources by deduplicating redundant analytical queries through managed queues.

    Rustagentic-analyticsagentsai
    在 GitHub 上查看↗20,251
  • victoriametrics/victoriametricsVictoriaMetrics 的头像

    VictoriaMetrics/VictoriaMetrics

    16,343在 GitHub 上查看↗

    VictoriaMetrics is a high-performance, scalable time series database and observability platform designed for long-term storage and analysis of metric, log, and trace data. It functions as a unified backend for monitoring ecosystems, offering full compatibility with industry-standard protocols and query languages. The system is built to handle massive data volumes through a distributed architecture that supports horizontal scaling and efficient data lifecycle management. The platform distinguishes itself through a storage engine that utilizes consistent hashing for data sharding and log-struct

    Collapses redundant data points retrieved from multiple storage replicas into single samples to ensure accurate query results.

    Godatabasegrafanagraphite
    在 GitHub 上查看↗16,343
  • improbable-eng/thanosimprobable-eng 的头像

    improbable-eng/thanos

    14,105在 GitHub 上查看↗

    Thanos is a CNCF cloud native monitoring tool that provides a highly available and scalable extension to the Prometheus ecosystem. It functions as a global query engine, a long-term storage system, and a metric downsampler. The project enables a unified interface to aggregate and query metrics across multiple distributed clusters from a single view. It maintains historical data beyond local retention limits by persisting time-series metrics in object storage and eliminates data gaps by merging metrics from redundant server pairs. The system includes capabilities for reducing the resolution o

    Implements metric sample deduplication to collapse redundant data points from multiple storage replicas for storage accuracy.

    Go
    在 GitHub 上查看↗14,105
  • golang/groupcachegolang 的头像

    golang/groupcache

    13,326在 GitHub 上查看↗

    Groupcache is a distributed caching library designed to coordinate data retrieval and storage across a cluster of nodes. It functions as a peer-to-peer data store that uses consistent hashing to assign specific keys to canonical owners, ensuring that cached items remain predictable and accessible throughout the network. The system distinguishes itself through a request coalescing engine that merges concurrent requests for the same missing key into a single upstream fetch. This mechanism prevents redundant backend load by ensuring that only one process retrieves the required data while sharing

    Prevents redundant backend fetches by merging concurrent requests for the same data into a single operation.

    Go
    在 GitHub 上查看↗13,326
  • coil-kt/coilcoil-kt 的头像

    coil-kt/coil

    11,819在 GitHub 上查看↗

    Coil is an image loading and caching pipeline designed for Android and Compose Multiplatform applications. It functions as a comprehensive loader, caching engine, and rendering utility that asynchronously fetches and displays images from network URLs, local storage, and multiplatform resource systems. The library distinguishes itself through a flexible fetcher-decoder pipeline and an interface-driven component registry, allowing for the integration of custom networking clients and decoders. It provides specialized support for rendering scalable vector graphics, animated formats such as GIF an

    Implements request collapsing to combine multiple simultaneous requests for the same image resource into a single execution.

    Kotlinandroidandroidxcompose
    在 GitHub 上查看↗11,819
  • netflix/falcorNetflix 的头像

    Netflix/falcor

    10,572在 GitHub 上查看↗

    Falcor is a JavaScript library that models remote data as a single virtual JSON graph, providing a path-based query engine for efficient client-side data retrieval and updates. It represents multiple remote data sources as a unified document where entities are accessed via globally unique identity paths. The system distinguishes itself by treating the remote data model as a virtual JSON resource, allowing the client to query specific paths without managing individual endpoints. It uses a reference-aware graph model to handle many-to-many relationships and prevents data duplication. Network ef

    Deduplicates requests during virtual graph traversal to reduce redundant data fetches.

    JavaScript
    在 GitHub 上查看↗10,572
  • prometheus-operator/prometheus-operatorprometheus-operator 的头像

    prometheus-operator/prometheus-operator

    9,941在 GitHub 上查看↗

    The Prometheus Operator is a Kubernetes monitoring orchestrator and controller that manages Prometheus clusters and observability components through declarative custom resources. It functions as a custom resource controller that translates high-level Kubernetes resource definitions into the configuration files required by the underlying monitoring software. The project automates the deployment, scaling, and lifecycle of an observability stack, including the integration of components like Thanos and Alertmanager. It distinguishes itself by syncing monitoring targets, alerting rules, and scrape

    Implements external labels and relabeling rules to prevent duplicate data samples in high-availability backends.

    Gohacktoberfestkubernetesmonitoring
    在 GitHub 上查看↗9,941
  • xiaomi/soarXiaoMi 的头像

    XiaoMi/soar

    8,770在 GitHub 上查看↗

    Soar is a suite of specialized tools designed for analyzing MySQL performance, advising on indexing, and optimizing SQL syntax. It functions as a performance analyzer, index advisor, and query optimizer to identify bottlenecks and suggest structural improvements for faster execution. The project distinguishes itself through a system for rewriting SQL statements into optimized equivalent versions using custom heuristic rules and patterns. It also features a dedicated index advisor that evaluates query patterns and database metadata to recommend the creation of new indexes. Its broader capabil

    Provides query fingerprinting by replacing literals with placeholders for deduplication and grouping.

    Goadvisorauditorcommand-line
    在 GitHub 上查看↗8,770
  • kean/nukekean 的头像

    kean/Nuke

    8,600在 GitHub 上查看↗

    Nuke is an image loading system designed to fetch, cache, and render images and short video clips within a user interface. It functions as an asset caching engine and a resumable download manager to handle the retrieval of remote media files. The framework includes a request priority manager to coordinate duplicate requests and order loading based on visual importance. It employs an image prefetching strategy to load visuals into memory before they are required and uses an LRU asset cache to reduce network traffic. The system covers broader capabilities in traffic management and data storage

    Collapses multiple concurrent network requests for the same image resource into a single shared promise.

    Swift
    在 GitHub 上查看↗8,600
  • andeya/pholcusandeya 的头像

    andeya/pholcus

    7,578在 GitHub 上查看↗

    Pholcus is a distributed web crawling system designed for large-scale data scraping. It employs a master-worker distribution model to coordinate high-concurrency scraping tasks across a network of remote client nodes, enabling both horizontal and vertical data collection. The system features a hot-loadable rule engine that allows extraction and navigation logic to be updated at runtime without restarting the process. It handles dynamic content through headless browser integration and bypasses bot detection using proxy rotation, automated user authentication, and simulated human behavior. The

    Includes a request deduplication pipeline to prevent redundant network calls and infinite crawling loops.

    Go
    在 GitHub 上查看↗7,578
  • chillicream/hotchocolateChilliCream 的头像

    ChilliCream/hotchocolate

    5,713在 GitHub 上查看↗

    HotChocolate is a GraphQL server implementation for .NET that handles queries, mutations, and subscriptions over HTTP. It provides code-first schema generation directly from C# classes and attributes, keeping the schema and business logic in sync without manual type definitions. The project also includes a GraphQL gateway that composes multiple GraphQL and REST endpoints into a single unified schema, along with a GraphQL IDE for exploring, testing, and debugging GraphQL APIs interactively. The project distinguishes itself through its comprehensive tooling ecosystem, which includes a GraphQL c

    Merges concurrent requests for the same key so the data source is queried only once per batch.

    C#
    在 GitHub 上查看↗5,713
  • day8/re-frameday8 的头像

    day8/re-frame

    5,532在 GitHub 上查看↗

    re-frame is a functional framework for building single-page applications in ClojureScript. It provides a centralized, immutable database that serves as the single source of truth for the entire application state, enforcing a strict unidirectional data flow where events trigger state transitions and subsequent view updates. The framework distinguishes itself through a reactive signal graph and an interceptor-based middleware pipeline. By treating application logic as a sequence of data-driven events and declarative side effects, it decouples business logic from the view layer. This architectur

    Collapses redundant data requests into single computation nodes to protect underlying sources and improve performance.

    Clojureclojurescriptre-framereact
    在 GitHub 上查看↗5,532
  • elbywan/wretchelbywan 的头像

    elbywan/wretch

    5,174在 GitHub 上查看↗

    Wretch 是一个可链式的 HTTP 客户端库与 Fetch API 封装器,旨在简化网络请求配置、响应解析与错误处理。它作为网络请求管理器与中间件框架,用于管理 HTTP 调用的端到端生命周期。 该项目以用于请求配置的流畅接口与模块化扩展系统而著称。它利用可编程的响应解析器来自动化解析,并利用插件式架构为请求与响应对象添加新能力。 该库涵盖了广泛的运营能力,包括带有指数退避的自动请求重试、响应缓存与请求去重。它提供了用于动态负载序列化、响应模式验证的工具,以及用于全局数据转换与请求监控的中间件流水线。安全特性包括基础认证支持与自动令牌刷新。 该库支持自定义 fetch 实现,以确保在不同环境下的兼容性。

    Collapses multiple identical outgoing requests into a single execution to prevent redundant network calls.

    TypeScript
    在 GitHub 上查看↗5,174
  • middleapi/orpcmiddleapi 的头像

    middleapi/orpc

    4,862在 GitHub 上查看↗

    orpc is a contract-first API development framework for TypeScript that starts with a shared contract definition and generates type-safe clients and servers from that single source of truth. It guarantees end-to-end type safety, meaning inputs, outputs, errors, and streaming data are all checked at compile time across the client–server boundary. What distinguishes orpc from typical RPC frameworks is its ability to export contracts as OpenAPI specifications, to optimize server-side rendering by calling API handlers directly inside the server process, and to support real‑time bidirectional commu

    Prevents redundant client-side requests by grouping similar ones so only one is sent to the server.

    TypeScriptapibunjscloudflare-worker
    在 GitHub 上查看↗4,862
  • onejs/oneonejs 的头像

    onejs/one

    4,475在 GitHub 上查看↗

    One is a full-stack, cross-platform React framework and multi-target rendering engine designed to build web and native applications from a single codebase. It utilizes a Vite-based build pipeline to manage shared components while providing a type-safe file-system router that automatically generates navigation definitions. The framework distinguishes itself by supporting a hybrid rendering strategy, allowing individual routes to be configured as static sites, server-rendered pages, or single-page applications. It employs platform-specific logic resolution and a shared Vite plugin to coordinate

    Implements request collapsing to share results of simultaneous calls to the same route and reduce server load.

    TypeScriptlocal-firstreactreact-native
    在 GitHub 上查看↗4,475
  • ht524/500lineorless_cnHT524 的头像

    HT524/500LineorLess_CN

    4,434在 GitHub 上查看↗

    This project is a Chinese localization repository and technical translation project designed to make concise programming projects and technical documentation accessible to Chinese speakers. It provides a collection of translated resources and curated mappings of computer science terminology to ensure consistent translation of technical concepts. The project implements a software localization workflow that converts English-language technical guides and codebase documentation into Chinese. This process utilizes a technical glossary resource and a resource-driven localization model to maintain t

    Filters out previously visited nodes during graph traversal to prevent redundant results.

    Jupyter Notebook
    在 GitHub 上查看↗4,434
  • apache/ageapache 的头像

    apache/age

    4,236在 GitHub 上查看↗

    Apache AGE is a graph database extension for PostgreSQL that adds openCypher graph query capabilities directly within the relational database environment. It functions as a loadable extension that translates Cypher graph traversal queries into SQL expressions, enabling users to run pattern matching and path analysis alongside standard SQL operations within a single database instance. The extension stores labeled, directed property graphs as isolated schemas with internal relational tables for vertices, edges, and labels, preventing cross-graph interference. It supports hybrid query execution

    Supports removing duplicate rows from graph query results using the DISTINCT modifier.

    Cage-databaseagensgraphanalytics
    在 GitHub 上查看↗4,236
  • pinterest/pinremoteimagepinterest 的头像

    pinterest/PINRemoteImage

    4,028在 GitHub 上查看↗

    PINRemoteImage 是一个用于 iOS 的图像下载与缓存框架。它作为一个远程图像加载器和多线程管理器,旨在从 URL 获取图像并将其渲染到用户界面中。 该项目包含用于渲染 GIF 和动画 WebP 格式的专用解码器。它还具有渐进式 JPG 渲染器,可在慢速连接下通过模糊效果显示扫描图像,以提升感知的加载速度。 该库通过请求去重管理并发下载任务,并利用混合内存与磁盘缓存系统。它处理高分辨率资产缩放,并在后台线程执行图像解码,以保持用户界面的响应速度。

    Collapses multiple concurrent network requests for the same image resource to prevent redundant data transfers.

    Objective-C
    在 GitHub 上查看↗4,028
上一个12下一个
  1. Home
  2. Web Development
  3. Request Deduplication

探索子标签

  • Query Deduplications3 个子标签Techniques for collapsing redundant data requests to protect underlying sources. **Distinct from Request Deduplication:** Distinct from Request Deduplication: focuses on query-level deduplication for analytical workloads rather than general network request collapsing.