awesome-repositories.com
Blog
awesome-repositories.com

Descubre los mejores repositorios open-source con nuestra búsqueda potenciada por IA.

ExplorarBúsquedas curadasAlternativas open-sourceSoftware autohospedableBlogMapa del sitio
ProyectoAcerca deCómo clasificamosPrensaServidor MCP
Aviso legalPrivacidadTérminos
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

840 repositorios

Awesome GitHub RepositoriesData Persistence and Storage

Technologies and architectures dedicated to the durable storage and long-term management of digital information.

Explore 840 awesome GitHub repositories matching data & databases · Data Persistence and Storage. Refine with filters or upvote what's useful.

Awesome Data Persistence and Storage GitHub Repositories

Encuentra los mejores repositorios con IA.Buscaremos los repositorios que mejor coincidan usando IA.
  • donnemartin/system-design-primerAvatar de donnemartin

    donnemartin/system-design-primer

    353,387Ver en GitHub↗

    Este proyecto es un recurso educativo integral y una guía de estudio centrada en la arquitectura de sistemas distribuidos y el diseño de infraestructura backend. Proporciona un plan de estudios estructurado para dominar los principios de escalabilidad, confiabilidad y rendimiento necesarios para diseñar sistemas de software complejos. El repositorio se distingue por ofrecer un enfoque metódico para la preparación de entrevistas técnicas, incorporando patrones de diseño, compensaciones arquitectónicas y herramientas de repetición espaciada para ayudar a los usuarios a retener conceptos complejos. Enfatiza el análisis basado en restricciones, enseñando a los usuarios cómo evaluar requisitos competitivos como latencia, consistencia y disponibilidad al redactar diseños arquitectónicos. El contenido cubre un amplio espectro de capacidades de diseño de sistemas, incluyendo estrategias para el escalado de bases de datos, gestión de tráfico y optimización de infraestructura. Detalla técnicas para el escalado horizontal, almacenamiento en caché multicapa, comunicación asíncrona y descubrimiento de servicios, al tiempo que proporciona marcos para realizar estimaciones de recursos y planificación de capacidad. La documentación está organizada como una guía de estudio, ofreciendo un camino sistemático a través de los fundamentos de la ingeniería backend y el diseño de sistemas a gran escala.

    Details mechanisms for storing frequently accessed data in memory to reduce latency and backend processing requirements.

    Pythondesigndesign-patternsdesign-system
    Ver en GitHub↗353,387
  • vinta/awesome-pythonAvatar de vinta

    vinta/awesome-python

    303,207Ver en GitHub↗

    Este proyecto es un directorio integral curado por la comunidad que organiza un vasto panorama de bibliotecas, frameworks y herramientas de software de Python. Sirve como una base de conocimientos centralizada diseñada para facilitar la navegación del ecosistema y acelerar el descubrimiento de desarrolladores en todo el ciclo de vida del desarrollo de software. El directorio se distingue por proporcionar un índice estructurado de recursos categorizados por dominio técnico, que van desde utilidades de desarrollo fundamentales hasta campos de ingeniería especializados. Cubre capacidades de alto nivel que incluyen inteligencia artificial, ciencia de datos, desarrollo web y gestión de infraestructura, lo que permite a los desarrolladores identificar soluciones verificadas para desafíos técnicos específicos. El proyecto abarca una amplia superficie de capacidades, incluyendo herramientas para la gestión de dependencias, análisis de código estático y pruebas automatizadas. También cataloga recursos para el almacenamiento de datos persistentes, orquestación de infraestructura en la nube y desarrollo de interfaces, proporcionando una referencia unificada para construir y mantener sistemas de software complejos.

    Boost system performance by memoizing frequently accessed data within memory-efficient storage structures.

    Pythonawesomecollectionspython
    Ver en GitHub↗303,207
  • torvalds/linuxAvatar de torvalds

    torvalds/linux

    237,355Ver en GitHub↗

    El kernel de Linux es un núcleo de sistema operativo monolítico que gestiona recursos de hardware, memoria y programación de procesos a través de diversas arquitecturas informáticas. Proporciona un entorno estandarizado y compatible con POSIX para la ejecución de aplicaciones, manteniendo al mismo tiempo un framework de controladores modular que permite la carga y eliminación dinámica de interfaces de hardware. El proyecto se distingue por su kit de herramientas de concurrencia de alto rendimiento, que utiliza primitivas de sincronización sin bloqueo y mecanismos de lectura-copia-actualización para gestionar el acceso a datos compartidos en entornos multinúcleo. Incorpora una suite completa de rastreo e instrumentación del kernel que permite el monitoreo no intrusivo de eventos del sistema, ejecución de funciones y métricas de latencia. Además, el kernel aplica garantías estrictas de estabilidad de interfaz y seguimiento del ciclo de vida para garantizar la compatibilidad con versiones anteriores para aplicaciones dependientes. Más allá de su identidad central, el sistema incluye amplias capacidades para la abstracción de hardware, implementación de protocolos de red y aplicación de políticas de seguridad. Admite requisitos de ingeniería especializados a través de la gestión del estado de energía, optimizaciones de sistemas integrados y procesos de arranque basados en firmware. La arquitectura también cuenta con marcos de diagnóstico robustos para el análisis de memoria, verificación de ejecución del sistema y validación de modelos de programación concurrente. El repositorio de origen proporciona un sistema de compilación completo para transformar código en imágenes binarias ejecutables, incluyendo herramientas para la selección de características del kernel y optimización de configuración para adaptar la salida a requisitos de hardware específicos.

    Manages filesystem operations to provide consistent data access and storage organization across physical media.

    C
    Ver en GitHub↗237,355
  • trimstray/the-book-of-secret-knowledgeAvatar de trimstray

    trimstray/the-book-of-secret-knowledge

    228,641Ver en GitHub↗

    Este proyecto sirve como un repositorio centralizado impulsado por la comunidad de conocimientos técnicos y recursos administrativos. Proporciona una taxonomía estructurada que agrega información dispar en un framework buscable, apoyando el aprendizaje continuo y la resolución rápida de problemas para administradores de sistemas y profesionales de ciberseguridad. Al mapear recursos a través de seguridad ofensiva, gestión de infraestructura y desarrollo de software, ofrece un camino unificado para la adquisición de habilidades y referencia profesional. El proyecto se define por una filosofía de diseño centrada en la línea de comandos, priorizando utilidades basadas en terminal e interfaces programables para facilitar una administración eficiente del sistema y flujos de trabajo de seguridad repetibles. Se distingue por un enfoque agnóstico a la plataforma, manteniendo documentación y guías operativas que siguen siendo aplicables a través de diversos entornos tipo Unix y basados en la nube. Esta integración modular de cadenas de herramientas permite a los usuarios componer entornos personalizados adaptados a tareas administrativas o de seguridad específicas. El repositorio cubre una amplia superficie de capacidades, incluyendo kits de herramientas integrales para auditoría de sistemas, gestión de redes y endurecimiento de infraestructura. Proporciona rutas de aprendizaje estructuradas para el desarrollo de habilidades en ciberseguridad, que van desde laboratorios de hacking ético y estándares de pruebas de penetración hasta evaluación de vulnerabilidades y mejores prácticas de configuración del sistema. La colección también abarca una amplia gama de herramientas de productividad, utilidades de diagnóstico y materiales educativos diseñados para agilizar el mantenimiento de rutina y mejorar la postura de seguridad general.

    Navigate and manage file systems through terminal-based interfaces that simplify directory operations.

    awesomeawesome-listbsd
    Ver en GitHub↗228,641
  • affaan-m/eccAvatar de affaan-m

    affaan-m/ECC

    221,981Ver en GitHub↗

    ECC es un framework de orquestación de agentes LLM y una suite de herramientas de IA multiplataforma diseñada para coordinar flujos de trabajo de múltiples modelos. Proporciona un sistema para gestionar roles de agentes especializados, habilidades reutilizables y planificación estructurada para ejecutar tareas complejas de desarrollo de software a través de diferentes editores de código impulsados por IA. El proyecto se distingue como un gestor de Protocolo de Contexto de Modelo, proporcionando una capa de configuración para integrar servidores externos y auditar la ejecución de herramientas. Además, implementa un sandbox de seguridad agentic que restringe el acceso a archivos confidenciales y escanea en busca de fugas de secretos para asegurar flujos de trabajo autónomos. El framework cubre amplias áreas de capacidad, incluyendo la automatización del flujo de trabajo de codificación de IA con barandillas de desarrollo impulsado por pruebas, optimización de costos de modelos a través de enrutamiento inteligente y gestión de memoria con estado aislado. También incluye herramientas para hacer cumplir los estándares de codificación específicos del lenguaje y gestionar los comportamientos de los agentes a través de varios entornos de desarrollo integrados. El sistema se gestiona a través de una interfaz de línea de comandos que maneja la instalación de herramientas, la reparación de configuración y la implementación de preajustes de herramientas.

    Manages the persistent storage of session summaries and learned skills under configurable root directories.

    JavaScript
    Ver en GitHub↗221,981
  • significant-gravitas/autogptAvatar de Significant-Gravitas

    Significant-Gravitas/AutoGPT

    184,973Ver en GitHub↗

    AutoGPT is an orchestration platform designed for building, managing, and deploying autonomous agents. It provides a visual canvas-based environment where users can assemble agents by connecting modular blocks that represent actions, data flows, and conditional logic. The platform supports the entire agent lifecycle, including task scheduling, execution monitoring, and configuration management, while offering a marketplace for discovering and sharing community-built workflows. The project includes a legacy framework for command-line agent execution and an extensible component system for devel

    Enables persistent file system interactions including directory navigation, reading, and writing with configurable storage paths.

    Pythonaiartificial-intelligenceautonomous-agents
    Ver en GitHub↗184,973
  • langchain-ai/langchainAvatar de langchain-ai

    langchain-ai/langchain

    139,458Ver en GitHub↗

    LangChain is an orchestration framework designed for building, managing, and deploying applications powered by large language models. It provides a unified integration layer that normalizes disparate model provider APIs into a consistent set of primitives, enabling developers to build complex, multi-step AI workflows that manage state, memory, and tool execution. The project distinguishes itself through a durable execution runtime that maintains persistent state across long-running processes by checkpointing progress to external storage. It models agent workflows as directed graphs, allowing

    Organize directory hierarchies to manage machine-specific state and persistent application data effectively.

    Pythonagentsaiai-agents
    Ver en GitHub↗139,458
  • chalarangelo/30-seconds-of-codeAvatar de Chalarangelo

    Chalarangelo/30-seconds-of-code

    128,121Ver en GitHub↗

    30-seconds-of-code is a comprehensive knowledge base and programming snippet library designed to support software engineering education and professional development. It provides a curated collection of reusable code units and technical guides that help developers master core language mechanics, design patterns, and architectural philosophies. The project distinguishes itself by offering a wide-ranging library of algorithmic solutions and web development patterns that are organized into modular, independently testable units. It emphasizes functional programming paradigms and declarative logic,

    Provides tools for serializing and persisting data to the local file system.

    JavaScriptastroawesome-listcss
    Ver en GitHub↗128,121
  • excalidraw/excalidrawAvatar de excalidraw

    excalidraw/excalidraw

    125,451Ver en GitHub↗

    This project is a virtual whiteboard component and vector graphics editor designed for creating diagrams with a hand-drawn aesthetic. It provides a canvas-based drawing engine that can be embedded directly into web applications, allowing users to manipulate shapes, upload images, and export visual data into standard formats like PNG, SVG, or JSON. The platform distinguishes itself through a real-time synchronization layer that supports multi-user collaboration across distributed environments. This engine utilizes end-to-end encryption to secure shared sessions and employs a local-first data p

    Leverages browser-based storage to maintain application state locally, ensuring data availability and persistence even during offline operation.

    TypeScriptcanvascollaborationdiagrams
    Ver en GitHub↗125,451
  • kubernetes/kubernetesAvatar de kubernetes

    kubernetes/kubernetes

    123,197Ver en GitHub↗

    Kubernetes is a distributed container orchestration platform that automates the deployment, scaling, and management of containerized applications across clusters of computing nodes. It functions as a declarative infrastructure controller, utilizing a control loop architecture that continuously monitors the current system state against user-defined configurations to ensure desired operational outcomes. The system relies on a centralized API-driven interface and a replicated key-value store to maintain a consistent source of truth for all cluster objects. The platform distinguishes itself throu

    Maintains a consistent, replicated data store that serves as the reliable source of truth for distributed system states.

    Gocncfcontainersgo
    Ver en GitHub↗123,197
  • immich-app/immichAvatar de immich-app

    immich-app/immich

    104,236Ver en GitHub↗

    Immich is a self-hosted media management platform designed to provide a centralized, private repository for photos and videos. It functions as a comprehensive system for organizing, backing up, and viewing personal media collections across mobile devices, web browsers, and external storage locations. By maintaining full control over data ownership and storage infrastructure, the platform ensures that users retain sovereignty over their digital assets. The system distinguishes itself through a distributed architecture that coordinates background media synchronization, real-time filesystem moni

    Mounts external storage volumes to provide containerized services with the necessary filesystem access for indexing media libraries.

    TypeScriptbackup-toolfluttergoogle-photos
    Ver en GitHub↗104,236
  • pytorch/pytorchAvatar de pytorch

    pytorch/pytorch

    100,814Ver en GitHub↗

    PyTorch is a machine learning framework centered on a GPU-ready tensor library that supports multi-dimensional array operations across both CPU and accelerator hardware. It provides a foundational infrastructure for mathematical computation and dynamic neural network construction, utilizing a tape-based automatic differentiation system that allows for flexible, non-static graph execution. The framework is designed for deep integration with Python, enabling natural usage alongside standard scientific computing ecosystems. It distinguishes itself through a comprehensive distributed training sui

    Persists tensors and complex data structures to disk through native loading and saving mechanisms.

    Pythonautograddeep-learninggpu
    Ver en GitHub↗100,814
  • chatgptnextweb/nextchatAvatar de ChatGPTNextWeb

    ChatGPTNextWeb/NextChat

    88,256Ver en GitHub↗

    NextChat is a self-hosted web application that provides a unified interface for interacting with multiple large language models. It functions as a conversational platform where users can manage and switch between diverse AI providers through configurable API backends, maintaining full control over their data and infrastructure. The platform features a persistent session layer designed to handle long-running dialogues by managing message history and context. It distinguishes itself through a structured prompt engineering environment that allows for the development and application of templates

    Utilizes browser-based storage to maintain chat logs and user preferences for offline access.

    TypeScriptcalclaudechatgptclaude
    Ver en GitHub↗88,256
  • macrozheng/mallAvatar de macrozheng

    macrozheng/mall

    83,878Ver en GitHub↗

    This project is an enterprise-grade Java framework designed for building scalable, full-stack e-commerce applications. It provides a comprehensive foundation for microservice-based distributed architectures, enabling the development of complex retail platforms that include product management, order processing, and secure user authentication. By leveraging modular service patterns and centralized API gateways, the framework supports the construction of resilient systems that decompose monolithic business logic into independent, manageable services. The platform distinguishes itself through a r

    Executes administrative tasks for object storage buckets, file transfers, and policy configurations directly through a terminal interface.

    Javadockerelasticsearchelk
    Ver en GitHub↗83,878
  • firehol/netdataAvatar de firehol

    firehol/netdata

    79,416Ver en GitHub↗

    Netdata is a real-time infrastructure monitoring tool and multi-node observability platform. It functions as a high-resolution monitoring agent, log and metric aggregator, and time-series database designed to provide full-stack visibility into server health. The system is distinguished by its per-second metric sampling and zero-configuration auto-discovery, which allows for immediate infrastructure tracking upon installation. It utilizes edge-based machine learning and unsupervised models to detect system anomalies and abnormal metric patterns locally on each node. For distributed environment

    Provides an efficient time-series database optimized for rapid retrieval and analysis of collected metrics.

    Go
    Ver en GitHub↗79,416
  • netdata/netdataAvatar de netdata

    netdata/netdata

    79,176Ver en GitHub↗

    Netdata is a distributed observability platform designed for real-time infrastructure monitoring and performance tracking. It functions as a high-frequency agent that collects system, container, and application metrics with per-second precision, providing both local visualization and centralized aggregation across complex, multi-cloud environments. The platform distinguishes itself through edge-based intelligence, utilizing local machine learning models to automatically detect performance anomalies without requiring manual configuration or external query engines. Its architecture prioritizes

    Persists high-resolution telemetry data directly on the host filesystem to ensure continuous availability during network outages.

    Caialertingcncf
    Ver en GitHub↗79,176
  • doocs/advanced-javaAvatar de doocs

    doocs/advanced-java

    78,987Ver en GitHub↗

    This project is a comprehensive Java backend engineering guide and technical reference focused on high-concurrency design, distributed systems, and microservices architecture. It provides detailed strategies for decomposing monolithic applications, managing service discovery, and implementing the architectural patterns required for scalable backend environments. The repository distinguishes itself through an extensive collection of big data algorithmic references and database scaling strategies. It covers memory-efficient techniques for analyzing massive datasets, such as Top-K element extrac

    Implements inverted index engines to map keywords to document identifiers for rapid distributed full-text search.

    Javaadvanced-javadistributed-search-enginedistributed-systems
    Ver en GitHub↗78,987
  • openhands/openhandsAvatar de OpenHands

    OpenHands/OpenHands

    77,330Ver en GitHub↗

    OpenHands is an autonomous agent framework designed for software engineering workflows. It provides a modular platform for orchestrating AI agents that reason, plan, and execute tasks within isolated, containerized development environments. By integrating with standard version control and development tools, the system enables agents to autonomously navigate codebases, implement features, and resolve issues through iterative reasoning and tool execution. The platform distinguishes itself through a model-agnostic orchestrator that connects diverse language models to a unified tool registry. It

    Store conversation history in structured directory formats with sequentially indexed event files for granular data access.

    Pythonagentartificial-intelligencechatgpt
    Ver en GitHub↗77,330
  • elastic/elasticsearchAvatar de elastic

    elastic/elasticsearch

    77,012Ver en GitHub↗

    Elasticsearch is a distributed search engine and document store designed for the high-performance indexing and retrieval of massive volumes of unstructured data. It functions as a centralized analytics platform, providing a schema-flexible architecture that organizes information into searchable indices while maintaining global cluster state through a distributed consensus mechanism. The platform distinguishes itself through its integrated approach to observability, security, and advanced analytics. It combines full-text, vector, and hybrid search capabilities with machine learning-driven insi

    Organizes schema-flexible data into searchable documents across distributed storage environments.

    Javaelasticsearchjavasearch-engine
    Ver en GitHub↗77,012
  • awesomedata/awesome-public-datasetsAvatar de awesomedata

    awesomedata/awesome-public-datasets

    75,979Ver en GitHub↗

    This project is a community-maintained, open-access directory of high-quality public datasets. It serves as a centralized reference point for researchers, developers, and data scientists to locate reliable information sources across a wide spectrum of industries and scientific fields. By providing a structured index, the repository facilitates the discovery of data necessary for exploratory analysis, machine learning model training, and the development of data-intensive applications. The directory distinguishes itself through a lightweight, platform-agnostic approach to resource indexing that

    Presents datasets related to global transportation systems, including traffic, logistics, and transit data.

    aaron-swartzawesome-public-datasetsdatasets
    Ver en GitHub↗75,979
Ant.123456…42Siguiente
  1. Home
  2. Data & Databases
  3. Data Engineering and Infrastructure
  4. Data Persistence and Storage

Explorar subetiquetas

  • Checkpoints and RecoveryMechanisms for creating consistent snapshots of data to enable rapid recovery after system restarts. **Distinct from Data Persistence and Storage:** Focuses on the recovery process via checkpoints rather than general long-term storage architecture
  • Contract Storage TieringManagement of contract state across different longevity tiers including persistent, instance, and temporary storage. **Distinct from Data Persistence and Storage:** Specific to smart contract runtime state rather than general data persistence or hardware tiering
  • Cross-Token Key-Value StoresPersistent key-value storage that can be shared and reused across different webhook endpoints and workflows. **Distinct from Data Persistence and Storage:** Distinct from Data Persistence and Storage: focuses on shared key-value storage across multiple endpoints rather than general durable storage.
  • Data Persistence Management10 sub-etiquetasSystems that manage the lifecycle and scheduling of data writing operations to ensure reliable storage.
  • Data Persistence Strategies5 sub-etiquetasApproaches for ensuring data remains available and consistent across system restarts or local storage environments.
  • Data Storage29 sub-etiquetasComponents and utilities that facilitate the saving, retrieving, and managing of data within an application environment.
  • Data Storage Architectures10 sub-etiquetasStructural designs and patterns that define how data is organized and accessed within a storage system.
  • Data Storage Layers4 sub-etiquetasSoftware abstractions that provide a dedicated interface for interacting with underlying database or storage systems.
  • Delivery AcknowledgementsMechanisms for confirming data persistence or delivery to upstream clients. **Distinct from Data Persistence and Storage:** Distinct from general persistence: focuses on the acknowledgement handshake protocol.
  • Filesystem Abstractions3 sub-etiquetasComponents focused on low-level file system logic, management, and containerized volume mounting rather than general data storage.
  • Hybrid Persistence EnginesStorage architectures that combine memory-based performance with disk-based durability. **Distinct from Data Persistence and Storage:** Focuses on the combination of memory and disk for performance, distinct from general persistence.
  • Near Real-Time StorageStorage architectures that optimize for immediate searchability of ingested data using memory buffers and logs. **Distinct from Data Persistence and Storage:** Distinct from Data Persistence and Storage: specifically focuses on the low-latency gap between ingestion and visibility, whereas the parent covers general durability.
  • Persistence & Durability3 sub-etiquetasMechanisms that ensure data remains intact and accessible over time, even during system failures or interruptions.
  • Persistence Integration Tests2 sub-etiquetasAutomated verification of database and cache operations against live storage instances. **Distinct from Data Persistence and Storage:** Distinct from Data Persistence and Storage: focuses on the validation of persistence operations rather than the storage technology itself.
  • Persistence-Ignorant ArchitecturesArchitectural patterns where business logic remains decoupled from specific database schemas or storage technologies. **Distinct from Data Persistence and Storage:** Distinct from general data persistence: focuses on the architectural decoupling of domain logic from storage, rather than the storage technology itself.
  • Specialized Storage Engines3 sub-etiquetasHigh-performance storage backends optimized for specific data structures like inverted indices or distributed key-value consensus.
  • Storage Command-Line InterfacesCLI tools for managing storage buckets and policies.
  • Storage Driver AbstractionsInterchangeable backend implementations for persisting application data and sessions. **Distinct from Data Persistence and Storage:** Focuses on the driver-based abstraction layer for persistence, rather than general storage technologies.
  • Storage Solutions2 sub-etiquetasInfrastructure platforms designed to store large volumes of data, typically in cloud or object-based environments.
  • Task List PersistenceDurable storage specifically for user-defined task lists and their completion status. **Distinct from Data Persistence and Storage:** Distinct from general data persistence by focusing on the specific domain of to-do list items.