40 Repos
Mechanisms for storing frequently accessed metadata locally to improve performance.
Distinguishing note: Focuses on local performance optimization for remote or large-scale data stores.
Explore 40 awesome GitHub repositories matching data & databases · Metadata Caching. Refine with filters or upvote what's useful.
This project is a comprehensive Java backend engineering guide and technical reference focused on high-concurrency design, distributed systems, and microservices architecture. It provides detailed strategies for decomposing monolithic applications, managing service discovery, and implementing the architectural patterns required for scalable backend environments. The repository distinguishes itself through an extensive collection of big data algorithmic references and database scaling strategies. It covers memory-efficient techniques for analyzing massive datasets, such as Top-K element extrac
Implements local caching of service provider addresses to maintain system availability when the central service registry is unavailable.
This project is a command-line utility designed for secure, content-addressable data archiving. It functions as an encrypted backup tool that stores data as deduplicated chunks, ensuring that every piece of information is identified by a cryptographic hash to maintain integrity across all backups. By applying strong encryption and message authentication codes to both data and metadata, the software prevents unauthorized access and detects potential tampering. The tool distinguishes itself through a backend-agnostic storage abstraction that allows users to maintain repositories across diverse
Stores frequently accessed repository indexes on the local disk to accelerate search operations.
SDWebImage is an asynchronous image downloader and caching library for iOS and macOS applications. It provides a core identity centered on a network utility for fetching images from URLs, a tiered memory and disk caching engine, and a processing framework for decoding and encoding media. The library features a specialized rendering engine for animated formats such as GIF and WebP, including support for progressive animation rendering. It distinguishes itself through a plugin system that allows for extended image format support and the ability to replace default loading or storage logic with c
Manages cache expiration and memory limits to prevent system memory pressure and ensure data freshness.
A2A is a standardized framework designed to enable interoperability, discovery, and orchestration among independent artificial intelligence agents. It provides a common communication protocol that allows heterogeneous agents to exchange data, verify identities, and collaborate across diverse programming languages and computing environments. By establishing a unified messaging standard, the project facilitates the creation of complex, multi-agent workflows where tasks are routed and managed between specialized services. The project distinguishes itself through a capability-based architecture t
Stores agent information locally using standard web caching headers to minimize redundant network requests while ensuring data remains accurate over time.
This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
Caches database metadata in local storage to reduce initialization time and improve application startup performance.
FastMCP is a Python framework designed for building servers that expose functions, resources, and prompts to AI models using the Model Context Protocol. It simplifies the development process by automatically deriving tool metadata, input schemas, and documentation directly from Python function signatures and type hints. The framework provides a unified container for managing these components, allowing developers to build modular applications that integrate seamlessly with AI assistants. The project distinguishes itself through its support for interactive, server-defined user interface compone
Stores backend tool and resource definitions in local memory to reduce latency.
Aerial is a macOS screensaver application and background manager that replaces static images with a rotating series of high-resolution aerial cinema shots. It functions as an Apple TV video streamer, fetching and displaying official aerial screensavers originally designed for Apple TV to be used as dynamic backgrounds on a Mac. The project integrates Apple TV content to enable desktop wallpaper customization and dynamic screensaver management. This allows the system to stream high-definition aerial views from remote sources to serve as a living background or an automated screensaver when the
Caches video URLs and associated metadata locally to improve performance and reduce network requests.
Bolt is a single-file embedded key-value store for Go applications. It is an ACID transactional database that organizes data in B+trees on disk to provide efficient sorted key retrieval and range scans. The system uses a memory-mapped model to map the database file directly into the process address space for fast random-access reads. The project distinguishes itself through a multi-version concurrency control architecture that allows multiple simultaneous readers to access a consistent snapshot of data without blocking a writer. It employs a single-writer multi-reader locking model and uses a
Provides a persistent on-disk store suitable for caching application configuration and metadata across restarts.
Duplicati is a self-hosted backup server designed to perform encrypted, incremental, and compressed backups to a wide range of local, network, and cloud-based storage providers. It functions as a background service that automates recurring data protection tasks, ensuring that only changed data blocks are stored to maximize efficiency and minimize bandwidth usage. The project distinguishes itself through a centralized management console that allows for the orchestration of multiple distributed backup agents from a single web-based dashboard. It supports multi-tenant management, enabling the or
Stores file paths and remote state information in a local database to speed up queries and avoid frequent remote requests.
Twemproxy is a lightweight proxy that routes and distributes requests across multiple Redis and Memcached backend servers. It functions as a protocol translation gateway and distributed cache shard manager, partitioning data across clusters to balance load and storage capacity. The system acts as a high-availability cache orchestrator, employing health monitoring and automatic server ejection to maintain continuous access to cached data. It integrates with sentinels for dynamic master and replica discovery and utilizes consistent hashing and tag-based key grouping to manage data distribution
Refreshes the expiration time of a cached item to keep it active without changing its value.
Falcor is a JavaScript library that models remote data as a single virtual JSON graph, providing a path-based query engine for efficient client-side data retrieval and updates. It represents multiple remote data sources as a unified document where entities are accessed via globally unique identity paths. The system distinguishes itself by treating the remote data model as a virtual JSON resource, allowing the client to query specific paths without managing individual endpoints. It uses a reference-aware graph model to handle many-to-many relationships and prevents data duplication. Network ef
Uses metadata to manage the time-to-live and expiration of cached graph fragments.
This project is a comprehensive sample library and implementation guide for ASP.NET Core. It provides a collection of practical examples and projects that demonstrate how to build web applications, RESTful APIs, and high-performance services. The repository focuses on a variety of architectural patterns, including the development of Minimal APIs, contract-first gRPC services, and real-time communication using WebSockets and Server-Sent Events. It includes detailed implementations for user identity and security, such as token-based authentication and CSRF protection. The codebase covers a bro
Executes registered callback functions automatically whenever a specific value is removed or expires from the cache.
DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met
Implements caching of routing decisions across inference iterations to reduce redundant CPU synchronization.
go-cache is a thread-safe, in-memory cache library for Go that stores arbitrary objects with per-item expiration timestamps. It provides a concurrent key-value store where multiple goroutines can safely read and write shared cached data without external synchronization, using a mutex-guarded map for access control. The library distinguishes itself through its expiration management and optional disk persistence. Each cached item carries its own time-to-live, and a background goroutine periodically purges expired entries. The cache can serialize its entire contents to disk using Go's gob encodi
Automatically removes cached items after a configurable duration to ensure data freshness.
go-cache ist ein thread-sicherer In-Memory-Key-Value-Speicher für Einzelmaschinen-Anwendungen. Er bietet einen lokalen Caching-Mechanismus, der Objekte im RAM speichert, um redundante Datenabrufe zu reduzieren und den Overhead von Netzwerkprotokollen oder Serialisierung zu vermeiden. Das System verfügt über einen persistenten Cache-Speicher, der den In-Memory-Zustand in einer lokalen Datei sichert und so die Datenwiederherstellung nach Anwendungsneustarts ermöglicht. Das Projekt verwaltet den gleichzeitigen Datenzugriff über mehrere Routinen hinweg und implementiert eine Time-to-Live (TTL)-Ablaufsteuerung für zwischengespeicherte Einträge. Ein Hintergrundprozess scannt den Speicher regelmäßig, um abgelaufene Elemente zu löschen und Speicher freizugeben.
Implements time-to-live expiration to automatically invalidate cached entries after a set duration.
Delta is a lakehouse table format that brings ACID transactions and data warehouse consistency to large scale data lakes on cloud object storage. It serves as an ACID transaction manager, coordinating atomic commits and serializable isolation for concurrent reads and writes across distributed compute engines. The project provides a multi-engine interoperability layer that uses format translation to allow diverse SQL engines and processing frameworks to read and write the same tables. It functions as a data versioning system, utilizing a transaction log to enable time travel, historical snapsh
Retrieves commit history and metadata from a catalog to bypass slow cloud storage lookups.
FastImageCache is an iOS image caching library that provides a persistent disk-based image store. It utilizes a persistent bitmap cache to store images in uncompressed formats and incorporates an image pre-processing pipeline to optimize assets before they are committed to storage. The library optimizes rendering performance by using memory-mapped image tables for constant-time retrieval and byte-aligned data layouts to prevent memory copies. It organizes images of identical dimensions into shared tables and manages disk space through a least-recently-used cache eviction system. The project
Automatically removes old image data from the store based on the recency of the last use.
Vendure is a Node.js e-commerce engine and headless commerce framework built with NestJS and TypeScript. It serves as a multi-channel commerce platform that manages product catalogs, orders, and customers via a strongly typed GraphQL API. The platform is distinguished by its highly extensible architecture, featuring a customizable administrative dashboard where developers can inject custom React components and entity views. It supports multi-channel commerce, allowing the isolation of products, currencies, and regional catalogs from a single unified backend. The engine covers a broad range o
Updates cached data using predefined refresh functions upon access after the expiration period.
SSHFS is a network filesystem client that maps remote server directories to local mount points. It functions as a userspace implementation of a filesystem, allowing users to access and manage remote files as if they were on a local disk using the SFTP protocol over SSH. The tool distinguishes itself by offering multiple transport options, including encrypted SSH tunnels for security and direct TCP socket connections to bypass encryption for higher throughput. It also supports connectivity via virtual sockets to access directories hosted inside virtual machines. The system includes capabiliti
Caches file attributes and metadata locally to reduce network round-trips for individual file information.
Mbed TLS is an open-source TLS and DTLS library with a small footprint, designed for embedded systems and IoT devices. It provides a portable cryptographic library that includes symmetric ciphers, hashing, and public-key cryptography, along with a reference implementation of the PSA Cryptography API for standardized cryptographic operations across platforms. The library also offers X.509 certificate management for parsing, validating, and managing certificate chains in secure communications. The library is built around a platform abstraction layer that decouples it from OS-specific services t
Provides custom timer callbacks for DTLS handshake retransmission in environments without a built-in timing module.