112 repositorios
This group covers various services and utilities for data storage in the cloud.
Explore 112 awesome GitHub repositories matching devops & infrastructure · Cloud Storage. Refine with filters or upvote what's useful.
Este proyecto es un directorio curado por la comunidad de software de código abierto diseñado para su implementación en entornos de servidores privados y laboratorios domésticos. Sirve como un recurso integral para descubrir alternativas independientes y autohospedadas a los servicios en la nube convencionales, permitiendo a los usuarios mantener la propiedad total de los datos y el control sobre su infraestructura digital. El directorio está estructurado a través de una taxonomía jerárquica que organiza una vasta colección de aplicaciones en categorías lógicas, que van desde la gestión de medios y análisis de datos hasta la comunicación privada y herramientas de productividad en equipo. Se distingue por un proceso de revisión por pares colaborativo, donde los miembros de la comunidad validan la calidad y relevancia de cada envío para garantizar que el directorio siga siendo preciso y confiable. El proyecto cubre una amplia superficie de capacidades, incluyendo automatización de infraestructura, implementación de servicios basados en contenedores y gestión de configuración declarativa. Estas herramientas ayudan a los usuarios a mantener entornos de servidor reproducibles y gestionar dependencias de servicios complejas en hardware privado. El directorio se mantiene como un repositorio con control de versiones, asegurando que todas las actualizaciones y cambios impulsados por la comunidad sean rastreados y transparentes.
Provides an interface compatible with standard object storage protocols to allow integration with existing cloud-native applications.
This project provides an integrated backend platform built around a relational database. It automatically generates REST and GraphQL APIs from database schemas, allowing for direct data interaction through standard requests and client libraries. The platform includes a comprehensive authentication system that manages user identity, session handling, and fine-grained access control through database-native row-level security policies. Beyond core data management, the platform offers specialized services for object storage, vector data processing for semantic search, and real-time communication
Manages unstructured data with S3-compatible APIs, global content delivery, and integrated access control for secure file storage.
Bun is a high-performance runtime environment designed to execute JavaScript and TypeScript applications with minimal latency and high throughput. Built on a native core implemented in Zig, it provides a unified execution engine that leverages JavaScriptCore for efficient memory management and low-latency startup. The project functions as an all-in-one toolchain, integrating a native bundler, transpiler, package manager, and test runner into a single command-line interface. What distinguishes Bun is its focus on native system integration and developer productivity. It features a high-performa
Uploads files and data directly to object storage services using static methods for simplified cloud integration.
This project is an enterprise-grade Java framework designed for building scalable, full-stack e-commerce applications. It provides a comprehensive foundation for microservice-based distributed architectures, enabling the development of complex retail platforms that include product management, order processing, and secure user authentication. By leveraging modular service patterns and centralized API gateways, the framework supports the construction of resilient systems that decompose monolithic business logic into independent, manageable services. The platform distinguishes itself through a r
Connects applications to S3-compatible object storage services to ensure seamless data interoperability.
rclone is a command-line utility and sync engine for managing, synchronizing, and migrating files across numerous cloud storage providers. It functions as a storage management tool that enables bidirectional or one-way synchronization between local filesystems and remote cloud backends. The project acts as a unified cloud storage gateway, capable of merging multiple remote providers into a single directory tree. It further provides a filesystem mount to expose remote cloud storage as a local disk for direct operating system access and a transparent encryption wrapper to secure data before it
Provides a FUSE-based mount to expose remote cloud storage as a local directory tree.
Multica is an autonomous coding agent manager and LLM agent orchestration platform. It coordinates teams of autonomous agents to execute coding tasks and manage their lifecycles through a centralized dashboard. The system provides multi-tenant agent workspaces that isolate agents, settings, and project issues into distinct organizational boundaries. The platform distinguishes itself through an agent skill library that captures successful task solutions as reusable, versioned skills. These skills are shared across the agent team and pinned using content hashes to ensure consistent behavior acr
Stores file artifacts and uploads in S3-compatible buckets using signed URLs for secure access.
Rustfs is a distributed object storage system designed for high availability and horizontal scalability. It functions as a cluster-based platform that manages data across multiple nodes, providing a self-hosted infrastructure for large-scale storage requirements. The system is built to be container-native, utilizing an operator to automate deployment and management within orchestrated environments. It provides compatibility with standard object storage protocols, allowing existing applications and tools to interact with the storage layer through a translation interface. To ensure long-term re
Provides S3-compatible object storage to integrate with existing applications and tools through standard protocols.
ArchiveBox is a self-hosted web archiving system designed to capture and preserve permanent static copies of webpages, media, and PDFs on personal infrastructure. It functions as a digital content curator and personal web archive manager, allowing users to import URLs from bookmarks, RSS feeds, and browser history to create a centralized, searchable knowledge base. The project is distinguished by its ability to archive private, paywalled, or login-protected content using browser cookies and authenticated session persistence. It ensures long-term availability by saving pages in multiple concur
Supports saving archived content to remote cloud storage providers and network shares.
Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network
Writes logs, metrics, and traces to S3 buckets with support for batching, storage classes, and encryption.
Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep
Configures persistent storage blocks for reading and writing data to S3-compatible object storage services.
Wekan es una herramienta de gestión de proyectos Kanban de código abierto y autohospedada utilizada para organizar flujos de trabajo a través de tableros, listas y tarjetas. Es una aplicación web en tiempo real que permite a los equipos gestionar tareas en infraestructura privada. La plataforma se distingue por herramientas de migración de datos extensas, específicamente para importar tableros y tarjetas desde Trello. Admite integración de identidad de nivel empresarial a través de LDAP, OpenID Connect y OAuth2, y ofrece opciones de almacenamiento flexibles, incluyendo PostgreSQL como backend relacional principal y almacenamiento en la nube conectable para archivos adjuntos. El sistema cubre una amplia gama de capacidades de gestión de tareas, incluyendo visualizaciones de diagramas de Gantt, seguimiento de tiempo y agregación de tareas entre tableros. Incluye herramientas administrativas para control de acceso basado en roles, programación de copias de seguridad automatizadas y extensibilidad programática a través de una API REST y webhooks basados en eventos. La aplicación está disponible para su despliegue a través de Docker y admite configuraciones multi-inquilino.
Supports pluggable backends for storing application files and backups in S3, Azure, or Google Cloud.
This project is a deep learning framework designed for constructing, training, and deploying neural networks across diverse hardware environments. It functions as a high-performance tensor computation library that provides both imperative and symbolic programming interfaces, allowing developers to balance flexible, step-by-step model building with the efficiency of compiled computation graphs. The framework distinguishes itself through a hybrid execution engine that integrates declarative graph compilation with imperative runtime logic. It supports scalable, distributed training across multip
Streams large datasets directly from cloud object storage into data iterators to optimize memory usage.
Cube is a semantic data layer that provides a unified framework for defining business metrics, dimensions, and relationships across diverse data sources. By acting as a headless business intelligence engine, it transforms raw data into a governed model that can be queried via SQL, REST, and GraphQL interfaces. This architecture ensures consistent data definitions and logic across all downstream analytical applications and reporting tools. The platform distinguishes itself through its integrated conversational AI capabilities, which allow users to explore data using natural language. It orches
Configures cloud storage buckets for high-performance retrieval of pre-aggregated data.
Goreplay is an HTTP traffic mirroring tool designed to capture live network traffic from production environments and replay it into test systems for validation. It includes a specialized Kubernetes traffic capturer that operates as a daemonset to mirror traffic from specific pods using label selectors and namespace filters, alongside a TCP traffic recorder for intercepting raw network packets. The project features a Kafka traffic pipeline for streaming captured payloads to topics or ingesting messages for playback, and an HTTP request transformer to mask sensitive data or rewrite headers and
Writes captured network traffic data directly to Amazon S3 buckets for long-term archival.
GoReplay is a network traffic recording and replay tool used to capture live HTTP and binary protocol requests. It functions as a traffic shadowing proxy that duplicates incoming network requests to test environments and a utility for recording traffic to local or cloud storage for later analysis and playback. The system is capable of processing non-textual data formats, such as Thrift and Protocol Buffers, allowing for the capture and replay of specialized application-to-application communication. The tool supports live traffic capture and asynchronous duplication to validate infrastructure
Stores captured network requests in remote cloud storage for retrieval and playback across environments.
Verdaccio is a registry server that implements the npm protocol to store, serve, and manage software dependencies. It functions as a private registry proxy for Node.js packages, allowing organizations to host internal packages while caching public modules. The system features a pluggable middleware architecture that enables the extension of registry behavior through custom plugins for authentication and storage backends. It supports scalable remote data management by offloading package files to S3-compatible object stores. The platform covers package lifecycle management, including publishin
Supports offloading package files to S3-compatible object stores for scalable remote data management.
LinkSwift is a cloud storage link extractor and file transfer utility designed to retrieve direct download addresses from various cloud storage providers. By generating raw download links, the tool enables users to bypass web interface restrictions and simplify the process of moving files between cloud platforms, local systems, or remote servers. The project functions as a direct download URL generator that automates file acquisition by programmatically parsing cloud storage pages. It employs a combination of headless browser automation and HTTP request interception to navigate dynamic web pa
Retrieves direct file download addresses from various cloud storage providers to bypass web interface restrictions.
Cap is a self-hosted screen recording and video collaboration platform designed for teams to replace synchronous meetings with asynchronous video updates. It provides a comprehensive suite for capturing high-resolution desktop activity, including system audio, microphone input, and camera overlays, which are then processed through an integrated post-production workflow. The platform distinguishes itself by offering full data sovereignty through containerized deployment and object storage abstractions, allowing users to host their media assets on private infrastructure or S3-compatible buckets
Integrates S3-compatible object storage for hosting and managing recorded media assets.
Linsa.io is an end-to-end encrypted cloud storage service and zero-knowledge data vault. It functions as a private content sharing platform that encrypts files and data on the client side, ensuring only the owner can access the stored content. The project employs a local-first approach, processing data updates and encryption on the local device before syncing encrypted blobs to a remote persistence layer. It uses a zero-knowledge architecture where the service provider cannot access decryption keys or view the plaintext content of stored files. The platform provides capabilities for private
Provides cloud storage where the provider has no access to the encryption keys.
kops is a Kubernetes cluster provisioner and lifecycle manager designed to automate the creation, maintenance, and destruction of production-grade clusters on cloud infrastructure. It functions as a declarative infrastructure manager, synchronizing the live state of a cluster with versioned manifests stored in remote object storage to ensure idempotent operations. The project distinguishes itself by offering comprehensive automation for the entire cluster lifecycle, including high-availability control plane deployment, incremental rolling updates, and automated version upgrades. It also serve
Copies required images and files into cloud storage buckets to enable deployments in restricted networks.