44 Repos
Architectures that allow registration of custom storage backends.
Distinguishing note: Focuses on the extensibility of the storage system.
Explore 44 awesome GitHub repositories matching data & databases · Pluggable Storage Drivers. Refine with filters or upvote what's useful.
OpenBBTerminal is a Python financial data platform and command line interface designed for aggregating and analyzing market data from diverse APIs. It serves as a quantitative analysis tool for processing stock, crypto, and derivative datasets to identify market trends and build investment strategies. The project utilizes a pluggable financial API framework with an adapter-based architecture, allowing external financial data providers to be integrated as independent modules. This system standardizes information from public and proprietary sources into a unified layer to support cross-asset an
Implements a pluggable connector framework to integrate heterogeneous financial data sources into a unified engine.
Fuel Core is a Rust-based implementation of a distributed ledger and a full node for the Fuel v2 protocol. It functions as a layer 2 blockchain client designed to execute binary encoded transactions and synchronize shared state across a decentralized network. The system utilizes a deterministic state machine to ensure all nodes reach the same ledger state and employs a binary-encoded message protocol for network communication. It provides the ability to operate full network nodes or bootstrap private local instances for development and testing. The project covers a broad set of capabilities
Supports pluggable storage backends to switch between permanent disk storage and volatile RAM.
graphql-engine is an automated GraphQL API engine that transforms database tables and relationships into a queryable GraphQL schema. It functions as a federation gateway and mapper, instantly generating APIs with built-in filtering, pagination, and mutations from existing databases and remote schemas. The project distinguishes itself through a fine-grained access control layer that enforces row-level and field-level permissions. It further provides a real-time data subscription server that converts standard queries into live streams and a system for triggering event-driven webhooks and notifi
Employs a pluggable middleware layer to translate a unified internal API into various database-specific dialects.
JumpServer is a privileged access management platform designed to manage and audit secure access to SSH, RDP, Kubernetes, and database endpoints. It functions as a centralized gateway that brokers remote terminal and graphical sessions to isolate users from critical infrastructure. The system utilizes a web-based protocol gateway to translate remote connections into browser-compatible streams and a protocol-based proxy layer to isolate end-user devices from target assets. It incorporates security watermarking to deter unauthorized screen captures and provides a Kubernetes access gateway for c
Supports a pluggable database backend for managing system configuration and logs.
TruffleHog is a secret scanning tool designed to identify leaked credentials and API keys across version control systems, cloud storage, and filesystems. It functions as a git secret detector that enumerates hidden commits and a cloud storage security auditor for inspecting container images and storage buckets. The project is distinguished by a credential verification engine that tests discovered secrets against service APIs to confirm they are active, which eliminates false positive alerts. It further analyzes these verified credentials to determine the specific access levels and resources t
Uses an abstract storage architecture to scan diverse targets like cloud buckets, docker images, and elastic clusters.
localForage is a browser storage wrapper and key-value store that provides a unified JavaScript API for persisting data. It acts as an abstraction layer over IndexedDB, WebSQL, and localStorage, allowing web applications to save data offline. The library features a priority-based driver fallback system that automatically detects and selects the most capable storage engine available in the browser. It simplifies data management by providing an asynchronous interface for storing complex JavaScript objects and binary buffers without requiring manual serialization. The project covers broad capab
Features a pluggable architecture that allows developers to inject custom storage drivers for data persistence.
This project is an asynchronous key-value store designed for client-side data persistence. It provides a unified interface that allows applications to save and retrieve complex data types, including binary objects, while maintaining responsiveness through non-blocking operations. By enabling offline-first functionality, it ensures that data remains accessible even when a network connection is unavailable. The library distinguishes itself through a driver-based abstraction layer that automatically detects the most efficient storage mechanism available in the current browser or mobile environme
Allows developers to register custom storage backends that adhere to a standardized interface.
Deepagents is an LLM agent orchestration platform and stateful application server designed for deploying and managing AI agents built with computational graphs. It provides a containerized runtime environment that handles agent execution, state persistence, and the versioning of AI assistants. The platform distinguishes itself through deep integration with the Model Context Protocol, allowing agents to function as servers that expose tools and capabilities to external clients. It features a sophisticated observability suite for capturing execution traces, performing LLM-based evaluations agai
Provides a modular architecture for registering custom storage backends for agent state and long-term memory.
SkyWalking is a comprehensive observability stack and application performance monitoring platform. It functions as a distributed tracing system and an AI application monitor, providing a centralized suite for collecting and analyzing logs, metrics, and traces to maintain the health of containerized architectures. The platform distinguishes itself through a service topology visualizer that renders interactive maps of infrastructure dependencies and communication patterns. It also includes specialized capabilities for generative AI workflow observation to track the execution flow and performanc
Provides a pluggable storage architecture that allows switching between different database engines for telemetry persistence.
Crawlee is a web scraping framework designed for building scalable, reliable, and distributed data extraction pipelines. It provides a unified interface for managing headless browser automation and lightweight HTTP requests, allowing developers to handle complex web navigation, dynamic content rendering, and large-scale data collection within a single, modular architecture. The project distinguishes itself through its resource-aware concurrency controller, which dynamically scales task execution based on real-time CPU and memory usage to prevent host machine exhaustion. It also features a rob
Decouples data persistence from application logic to allow swapping between local, memory, or cloud-based storage backends.
This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
Allows switching between various storage engines to optimize performance for different environments.
Jaeger is a distributed tracing platform used for collecting, storing, and visualizing request flows across microservices. It identifies performance bottlenecks and errors by tracking requests as they move through multiple service boundaries. The system includes telemetry collectors, a multi-tenant backend, and a trace visualizer. The platform provides a multi-tenant tracing infrastructure that isolates data and queries by tenant to support shared environments. It supports standardized telemetry ingestion via the OpenTelemetry Protocol over gRPC and HTTP. To manage storage costs and overhead,
Provides a provider interface that allows different database engines to store and retrieve trace records.
Node-RED is a visual, low-code automation framework designed for building event-driven data processing workflows. It provides a browser-based programming environment where users connect hardware devices, APIs, and online services by wiring together functional nodes in a directed graph. This visual approach allows for the creation of complex logic paths without the need for traditional source code. The platform is distinguished by its pluggable node architecture and portable flow serialization. Logic is represented as JSON-based data structures, enabling flows to be easily versioned, shared, a
Allows registration of custom storage backends for runtime data, context, and library assets.
LangChainJS is an AI agent orchestrator and application framework designed for building autonomous systems that use large language models to plan and execute tasks. It serves as an integration library that connects language models with tools, memory, and external data sources to create context-aware logic and complex workflows. The project provides a provider-agnostic interface and model provider abstraction, allowing applications to switch between different language model providers without rewriting core logic. It includes a toolkit for retrieval augmented generation, utilizing retrievers to
Ships a storage abstraction allowing the system to swap between different databases for persisting conversation history.
Cognee is an agentic memory management platform designed to provide autonomous agents with long-term semantic recall and structured knowledge. It functions as a framework for building persistent memory systems that connect large language models to graph-based knowledge and vector storage, enabling agents to maintain context across complex tasks and multiple sessions. The platform distinguishes itself through a hybrid approach that combines semantic similarity search with structural graph traversal, allowing for context-aware information retrieval. It features a modular architecture that orche
Abstracts database interactions to support diverse relational, vector, and graph storage backends.
Verdaccio is a registry server that implements the npm protocol to store, serve, and manage software dependencies. It functions as a private registry proxy for Node.js packages, allowing organizations to host internal packages while caching public modules. The system features a pluggable middleware architecture that enables the extension of registry behavior through custom plugins for authentication and storage backends. It supports scalable remote data management by offloading package files to S3-compatible object stores. The platform covers package lifecycle management, including publishin
Provides an extensible architecture allowing the registration of custom storage backends, including local filesystems and S3-compatible stores.
Zipkin is an open-source distributed tracing system designed to collect, store, and visualize timing data across complex service architectures. It provides a platform for monitoring request lifecycles, enabling developers to identify latency bottlenecks and performance issues by tracking operations as they move through heterogeneous service environments. The system distinguishes itself through a standardized data model and a pluggable storage architecture that supports various backend databases. It utilizes sampling strategies to manage telemetry volume and employs asynchronous collection met
Abstracts database interactions through a common interface to allow swapping between memory, relational, or NoSQL storage systems.
Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Uses a standardized service interface to integrate heterogeneous data sources and storage systems into a unified query engine.
Apache Pulsar is a cloud-native distributed pub-sub messaging system designed for high-performance data ingestion. It functions as a geo-replicated data streamer and a multi-tenant event streaming platform, providing a serverless stream processing engine and a tiered storage messaging broker. The system distinguishes itself by separating serving layers from storage layers to allow independent scaling of compute and data retention. It features native geo-replication to synchronize messages across different geographical regions and employs a multi-layered tenant isolation model using authentica
Features a modular connector framework to integrate heterogeneous external data sources and sinks.
Cayley is a graph database engine designed for storing and querying interconnected data using a quad-based data model. It functions as an RDF quad store, managing information through subjects, predicates, objects, and labels. The system features a modular graph store architecture with pluggable backends, allowing it to swap between in-memory storage and various external persistent databases. It includes a GraphQL-inspired API and a dedicated data visualizer for the interactive exploration of nodes and edges. Query capabilities cover bidirectional path traversal and multi-syntax execution usi
Implements a pluggable architecture allowing the storage backend to be swapped between in-memory and persistent databases.