awesome-repositories.com
Blog
MCP
awesome-repositories.com

Découvrez les meilleurs dépôts open-source grâce à notre recherche par IA.

ExplorerRecherches sélectionnéesAlternatives open sourceLogiciels auto-hébergésBlogPlan du site
ProjetServeur MCPÀ proposNotre méthodologiePresse
Mentions légalesConfidentialitéConditions d'utilisation
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

12 dépôts

Awesome GitHub RepositoriesData Archiving

Patterns for long-term storage and preservation of data streams.

Distinguishing note: Focuses on messaging stream archiving patterns.

Explore 12 awesome GitHub repositories matching software engineering & architecture · Data Archiving. Refine with filters or upvote what's useful.

Awesome Data Archiving GitHub Repositories

Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
  • nsqio/nsqAvatar de nsqio

    nsqio/nsq

    25,738Voir sur GitHub↗

    NSQ is a distributed, brokerless messaging platform designed for high-throughput, fault-tolerant communication. By utilizing a decentralized topology, it eliminates single points of failure and allows for horizontal scaling across clusters. The system organizes message streams into topics and channels, effectively decoupling producers from consumers to support both streaming and job-oriented workloads. The platform distinguishes itself through a lookup-service-based discovery mechanism that enables clients to dynamically locate producers at runtime without requiring centralized coordination.

    Archives message streams to disk using dedicated consumer channels.

    Godistributed-systemsgomessage-queue
    Voir sur GitHub↗25,738
  • letta-ai/lettaAvatar de letta-ai

    letta-ai/letta

    21,168Voir sur GitHub↗

    Letta is a framework for building, deploying, and managing autonomous AI agents that maintain persistent state across long-term interactions. It provides a comprehensive suite of primitives for defining agents with configurable personas, modular memory blocks, and tool-use capabilities, enabling them to retain user preferences and conversation history over extended sessions. The platform distinguishes itself through its advanced memory management and orchestration capabilities. It allows agents to autonomously update their own memory, perform retrieval-augmented generation, and coordinate com

    Adds text content to persistent archives with vector embeddings for semantic retrieval.

    Pythonaiai-agentsllm
    Voir sur GitHub↗21,168
  • microsoftdocs/azure-docsAvatar de MicrosoftDocs

    MicrosoftDocs/azure-docs

    10,894Voir sur GitHub↗

    Azure Docs is the official technical documentation repository for Microsoft Azure, the cloud computing platform. It provides comprehensive guidance on the full spectrum of Azure services, covering everything from core infrastructure components like virtual machines, Kubernetes clusters, and serverless computing to platform services for AI, machine learning, data analytics, and storage. The documentation details how to provision, manage, and govern cloud resources at scale, including policy enforcement, identity management, and cost optimization. The documentation distinguishes Azure through i

    Documents Azure's archival storage service for rarely used data at the lowest cost.

    Markdownskilling
    Voir sur GitHub↗10,894
  • blinkospace/blinkoAvatar de blinkospace

    blinkospace/blinko

    10,601Voir sur GitHub↗

    Blinko is a personal knowledge management system and an LLM-powered knowledge base that enables users to capture and organize thoughts through a bi-directional knowledge graph. It functions as a RAG-enabled note-taking application and a self-hosted Markdown editor, allowing for the creation of permanent documentation and fleeting notes. The project distinguishes itself by integrating retrieval-augmented generation to provide conversational querying and AI-powered analysis of private document libraries. It supports both cloud-based and local AI model integration, enabling users to perform sema

    Automatically moves old records into an archive at specified frequencies to maintain workspace cleanliness.

    TypeScriptmarkdownmemosnextjs
    Voir sur GitHub↗10,601
  • tyrrrz/discordchatexporterAvatar de Tyrrrz

    Tyrrrz/DiscordChatExporter

    10,392Voir sur GitHub↗

    Discord Chat Exporter is a tool for extracting messages and media from Discord channels and direct messages into offline files. It functions as a backup utility and archival tool, using authentication tokens to retrieve chat history and metadata for long-term storage or history recovery. The system converts API data into readable documents and supports multi-format export options, including HTML, TXT, CSV, and JSON. It includes capabilities for automated chat backups by creating recurring tasks through the host operating system's task scheduler. The tool provides data management features suc

    Saves chat history as HTML, JSON, CSV, or TXT files for permanent long-term storage.

    C#archivalarchiverchat
    Voir sur GitHub↗10,392
  • 1n3/sn1perAvatar de 1N3

    1N3/Sn1per

    10,049Voir sur GitHub↗

    Sn1per is a vulnerability management platform and penetration testing orchestrator designed to automate reconnaissance, vulnerability scanning, and exploit verification. It functions as a dockerized security toolkit that coordinates multiple tools into a unified automated pipeline to identify security flaws across network and web assets. The platform features an attack surface manager for discovering internet-facing assets through OSINT, DNS enumeration, and certificate transparency. It distinguishes itself with an AI-powered security analyzer that uses large language models to summarize scan

    Bundles workspace reports, loot, and vulnerability files into a single compressed archive for backup.

    Shellattack-surfaceattack-surface-managementattacksurface
    Voir sur GitHub↗10,049
  • supabase/realtimeAvatar de supabase

    supabase/realtime

    7,488Voir sur GitHub↗

    Realtime is a real-time data distribution and synchronization engine that enables applications to stream database changes and coordinate state between clients. It functions as a synchronization layer that monitors database write-ahead logs to provide change data capture and pushes updates to authorized clients via WebSockets. The project features a real-time presence server for tracking the online status of active users and a broadcast service for sending ephemeral messages without database persistence. It organizes communication through channel-based message routing and uses a structured JSO

    Stores processed messages in an archive to maintain audit trails and reference history.

    Elixircdcchange-data-capturecrdt
    Voir sur GitHub↗7,488
  • infobyte/faradayAvatar de infobyte

    infobyte/faraday

    6,523Voir sur GitHub↗

    Faraday is a vulnerability management platform and security tool aggregator designed to centralize security findings from multiple scanners into a single dashboard. It utilizes a relational security database to catalog hosts, services, and security flaws, enabling users to track remediation and analyze organizational risk. The platform distinguishes itself through a plugin-based system that normalizes diverse security tool outputs into a unified data model. It supports deep integration with a wide array of scanners and CLI tools, intercepting shell command output or parsing report files to ag

    Allows hiding completed or inactive workspaces to maintain focus on active security engagements.

    Python
    Voir sur GitHub↗6,523
  • j3ssie/osmedeusAvatar de j3ssie

    j3ssie/Osmedeus

    6,425Voir sur GitHub↗

    Osmedeus is a security workflow orchestration engine that coordinates AI agents, shell commands, and scanning tools through declarative YAML pipelines. It functions as a distributed security scanner, a declarative workflow automator, and an AI agent framework for security, enabling automated multi-step security analysis with conditional branching, parallel execution, and distributed workers. The engine distinguishes itself through a hybrid runner model that executes workflow steps on the local host, inside Docker containers, or over SSH to remote machines, selected per step or module. It supp

    Packages workspace files, database records, and metadata into a ZIP archive for backup or transfer.

    Go
    Voir sur GitHub↗6,425
  • owid/covid-19-dataAvatar de owid

    owid/covid-19-data

    5,663Voir sur GitHub↗

    Data on COVID-19 (coronavirus) cases, deaths, hospitalizations, tests • All countries • Updated daily by Our World in Data

    A structured archive of global COVID-19 indicators with consistent column metadata and versioned releases.

    Pythoncoronaviruscovidcovid-19
    Voir sur GitHub↗5,663
  • yacy/yacy_search_serverAvatar de yacy

    yacy/yacy_search_server

    3,966Voir sur GitHub↗

    yacysearchserver est un système d'indexation décentralisé et un moteur de recherche peer-to-peer. Il fonctionne comme un crawler web distribué et un appareil de recherche intranet qui permet la découverte et l'indexation de contenu web sans dépendance à une autorité centrale. Le projet permet la création de portails de recherche privés pour indexer des sites web internes et des systèmes de fichiers. Il utilise un réseau peer-to-peer pour partager des index de recherche et distribuer le routage des requêtes à travers plusieurs instances de serveur, mettant à l'échelle les résultats de recherche via une gestion d'index décentralisée. Le système couvre le crawling web récursif, l'indexation de contenu web et l'affinement avancé des requêtes. Il inclut des utilitaires pour la sauvegarde, la restauration et l'exportation et l'importation de données indexées. L'application prend en charge le déploiement via des conteneurs Docker et permet la configuration du système via des variables d'environnement.

    Preserves system state and search indices by serializing data volumes into compressed backup files.

    Javadecentralizedintranet-searchintranet-search-engine
    Voir sur GitHub↗3,966
  • swe-agent/mini-swe-agentAvatar de SWE-agent

    SWE-agent/mini-swe-agent

    2,947Voir sur GitHub↗

    mini-swe-agent is an autonomous software engineering system designed to develop features and fix bugs by combining large language models with a bash interface. It operates as an agentic framework that executes coding tasks and documentation updates through a continuous cycle of model reasoning and tool execution. The project differentiates itself with a strong focus on safety and evaluation, utilizing container-based sandbox execution via Docker or Singularity to isolate command execution. It includes a batch-parallel evaluation harness to measure code-fixing accuracy against standardized sof

    Archives the state of the containerized workspace into local tarballs to preserve task outcomes.

    Pythonagentagentic-aiagentic-ai-cli
    Voir sur GitHub↗2,947
  1. Home
  2. Software Engineering & Architecture
  3. Data Archiving

Explorer les sous-tags

  • Epidemiological Data ArchivesStructured archives of global health indicators with consistent column metadata and versioned releases. **Distinct from Data Archiving:** Distinct from Data Archiving: focuses on public health epidemiological data with metadata rather than general messaging stream archiving.
  • Infrequent Data ArchivalStores rarely used data at the lowest cost with retrieval times measured in hours. **Distinct from Data Archiving:** Distinct from Data Archiving: focuses on low-cost archival storage for infrequently accessed data, not general archiving patterns.
  • Workspace Archiving1 sous-tagBundling and compressing engagement-specific data, reports, and loot for backup. **Distinct from Data Archiving:** Specifically archives penetration testing workspace data rather than generic system data streams.