Awesome GitHub RepositoriesRemote Storage Protocols

PhotoPrism is a self-hosted digital asset management platform designed to organize, classify, and manage large collections of photos and videos on personal infrastructure. It functions as a private alternative to cloud-based services, ensuring that all media remains under the user's control. The platform utilizes neural-network-based media analysis to automatically detect objects, faces, and locations, providing a comprehensive, AI-powered approach to library organization. The project distinguishes itself through its containerized architecture, which simplifies deployment and lifecycle manage

Rustfs is a distributed object storage system designed for high availability and horizontal scalability. It functions as a cluster-based platform that manages data across multiple nodes, providing a self-hosted infrastructure for large-scale storage requirements. The system is built to be container-native, utilizing an operator to automate deployment and management within orchestrated environments. It provides compatibility with standard object storage protocols, allowing existing applications and tools to interact with the storage layer through a translation interface. To ensure long-term re

Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

Ceph is a unified, software-defined storage platform designed to provide object, block, and file storage services from a single distributed cluster. By decoupling data management from physical hardware, it enables elastic scaling across commodity hardware, allowing organizations to build large-scale storage infrastructure without reliance on proprietary vendor equipment. The system distinguishes itself through a shared-nothing, distributed architecture that utilizes deterministic hashing for data placement. This approach eliminates centralized metadata bottlenecks, allowing the cluster to sca

MaterialFiles is a graphical Linux file manager and system utility designed for local file management, archive manipulation, and system administration. It provides a filesystem interface for navigating directories, inspecting file attributes, and managing local files. The application supports privileged root access to manage restricted system files and directories. It includes specialized tools for Linux system administration, such as inspecting security contexts, symbolic links, and file permissions. The utility features a network file client for connecting to remote servers and managing fi

SSHFS is a network filesystem client that maps remote server directories to local mount points. It functions as a userspace implementation of a filesystem, allowing users to access and manage remote files as if they were on a local disk using the SFTP protocol over SSH. The tool distinguishes itself by offering multiple transport options, including encrypted SSH tunnels for security and direct TCP socket connections to bypass encryption for higher throughput. It also supports connectivity via virtual sockets to access directories hosted inside virtual machines. The system includes capabiliti

The AWS SDK for PHP is a software development kit that provides HTTP client classes for every supported Amazon Web Service, enabling PHP applications to send authenticated requests and receive structured, typed response objects. It includes a credential resolution chain that automatically locates credentials from environment variables, instance profiles, or configuration files, and supports promise-based asynchronous execution for running multiple API calls concurrently to improve throughput. The SDK distinguishes itself through a middleware pipeline architecture that allows interception and

CubeFS es un sistema de almacenamiento en la nube distribuido diseñado para gestionar almacenamiento de archivos y objetos en centros de datos y nubes híbridas. Funciona como un sistema de archivos distribuido multi-inquilino y un almacén de objetos capaz de manejar datos a escala de exabytes, utilizando una arquitectura distribuida para almacenar contenido no estructurado. El sistema se distingue por una capa de interfaz multiprotocolo que permite el acceso simultáneo a datos a través de interfaces S3, POSIX y HDFS. Emplea una arquitectura desacoplada de cómputo-almacenamiento para escalar el procesamiento y la persistencia de forma independiente e implementa políticas de aislamiento de grano fino para separar recursos y datos entre diferentes inquilinos. La fiabilidad se gestiona mediante estrategias de redundancia configurables, incluyendo replicación múltiple y codificación de borrado (erasure coding). La plataforma incluye un sistema de caché multinivel para acelerar el acceso a los datos y se integra con Kubernetes a través de un controlador de Container Storage Interface (CSI) para automatizar el aprovisionamiento de volúmenes persistentes.

LXD is a unified platform for managing both system containers and virtual machines through a single REST API and command-line interface. It provides a programmatic HTTP interface for controlling the full lifecycle of instances, enabling automation and integration with external tools. The system runs unprivileged containers with per-instance UID/GID mappings, seccomp filters, and AppArmor profiles for kernel-level isolation, while supporting multiple storage backends including directory, Btrfs, LVM, ZFS, Ceph, LINSTOR, and TrueNAS through a unified driver interface. The platform distinguishes

lakeFS es un sistema de versionado de lagos de datos que proporciona ramificaciones (branching) y commits similares a Git para grandes conjuntos de datos almacenados en almacenamiento de objetos. Funciona como una capa de control de versiones, permitiendo la creación de instantáneas inmutables, commits atómicos y ramificaciones de copia cero para crear entornos aislados para la experimentación de datos sin duplicar archivos físicos. El sistema sirve como una puerta de enlace de almacenamiento compatible con S3 y un catálogo REST de Iceberg, permitiendo que los protocolos de almacenamiento en la nube estándar y los clientes compatibles gestionen tablas versionadas. Actúa como un guardián de calidad de datos mediante el uso de un sistema de hooks basado en eventos para validar conjuntos de datos contra políticas de gobernanza antes de que los cambios se fusionen en producción. La plataforma cubre amplias capacidades para la gobernanza de datos, incluyendo colaboración mediante pull requests, control de acceso basado en roles y seguimiento del linaje de datos. Proporciona integración para la orquestación de flujos de trabajo, pipelines de aprendizaje automático y varios motores de cómputo de big data, soportando conectividad de almacenamiento multi-nube y sincronización de identidad mediante SSO y SCIM. El software se puede instalar utilizando binarios, contenedores o Helm charts para su despliegue en Kubernetes.

GlusterFS es un sistema de archivos distribuido definido por software y un clúster de almacenamiento escalable que agrega recursos de disco de múltiples servidores en un único espacio de nombres global. Funciona como una plataforma de almacenamiento unificada, permitiendo que los mismos datos subyacentes se expongan a través de interfaces de almacenamiento de archivos, bloques y objetos. El sistema se distingue por una arquitectura descentralizada que utiliza hashing consistente para distribuir archivos a través de nodos de red sin un servidor de metadatos central. Asegura la integridad y disponibilidad de los datos utilizando replicación de autocuración, consistencia basada en quórum para evitar escenarios de split-brain y georeplicación asíncrona para recuperación ante desastres en sitios geográficos distantes. La plataforma proporciona una amplia cobertura de capacidades a través de exportación de almacenamiento multiprotocolo vía NFS, Samba e iSCSI, así como una gestión integral de volúmenes que incluye instantáneas puntuales, cuotas de almacenamiento y expansión elástica de capacidad. La seguridad se maneja a través de cifrado TLS para datos en tránsito, integración de gestión de identidad y control de acceso POSIX granular. El software está disponible como paquetes binarios a través de repositorios comunitarios para varias distribuciones de Linux.

Sigma File Manager is a cross-platform desktop application for browsing, organizing, and searching local and network files through a modern graphical interface. It functions as an integrated file previewer, network storage client, and local network file share utility. The application is built on an extensible architecture that allows users to add custom commands, pages, and settings through a plugin system and marketplace. It distinguishes itself with the ability to stream local files and folders to other devices over a local network using browser access, QR codes, and FTP. Core capabilities

Incus is a unified orchestration platform for managing system containers, OCI application containers, and virtual machines through a single control plane. It brings together cluster infrastructure management, secure multi-tenancy, software-defined networking, and pluggable storage backend orchestration into one cohesive system exposed via a full REST API and command-line interface. What distinguishes Incus is its ability to run multiple instance types side by side—full Linux system containers, OCI application containers, and QEMU virtual machines—all managed with consistent tooling. Networkin

s3cmd is a command-line client for Amazon S3 and S3-compatible cloud storage services. It provides a unified terminal interface for managing buckets and objects, transferring files, and synchronising directories. All operations are performed over HTTPS with HMAC-SHA256 request signing and TLS encryption to secure data in transit. The tool supports incremental transfers using S3 ETags, so only new or modified files are sent during sync. Large files are handled with multipart upload chunking, reducing the impact of failures and improving throughput. Directory synchronisation works recursively w

Cyberduck es un cliente de transferencia de archivos multiprotocolo y gestor de archivos multiplataforma para Mac y Windows. Funciona como un gestor de almacenamiento en la nube y montador de almacenamiento remoto, permitiendo a los usuarios subir, organizar y sincronizar datos entre unidades locales y endpoints remotos. La aplicación proporciona una interfaz unificada para gestionar archivos a través de FTP, SFTP, WebDAV, S3 y otros protocolos de almacenamiento en la nube, incluyendo Amazon S3, Backblaze B2, Microsoft Azure y OneDrive. Se distingue por una bóveda de cifrado del lado del cliente que cifra archivos y carpetas localmente antes de ser transmitidos a servidores remotos. El sistema admite la gestión de archivos remotos y transferencias de archivos entre protocolos, incluyendo la capacidad de montar directorios de servidores remotos o de la nube directamente en el explorador de archivos del sistema operativo local. Capacidades adicionales incluyen una interfaz de línea de comandos para ejecutar tareas de transferencia de archivos.

Contour is a Kubernetes ingress controller that manages external HTTP and TCP traffic by orchestrating Envoy proxy instances. It functions as a control plane that translates declarative Kubernetes API objects into dynamic proxy configurations, enabling real-time traffic management without requiring infrastructure restarts. The system distinguishes itself through a focus on multi-tenant cluster networking and delegated configuration. By allowing routing rules to be partitioned across namespaces, it enables multiple teams to securely manage their own traffic policies within a shared cluster. It

This project is a comprehensive framework for iOS application development, centered on building mobile applications that feature custom user interface components, asynchronous task management, and local data persistence. It serves as a technical knowledge base for software engineering, providing tools to organize and publish architectural analyses and notes in Markdown format. The framework distinguishes itself through a robust document-based storage layer that utilizes BSON-formatted records to perform CRUD operations within a NoSQL document store. It provides extensive system integration ca

Garage is a distributed object storage system that provides an S3-compatible API gateway. It is designed to synchronize metadata across distributed nodes using conflict-free replicated data types and Merkle-tree state alignment to maintain cluster-wide consistency. The system ensures data resilience through zone-aware replication, distributing data copies across multiple physical locations. It employs quorum-based request routing and versioned layout management to validate and commit cluster configuration changes. The project covers a broad range of operational capabilities, including automa

minio-go is a client library and software development kit for interacting with S3-compatible object storage. It provides a programmatic interface for Go applications to manage buckets and objects using the S3 protocol. The library enables the execution of complex storage operations, including multi-part uploads for large datasets, data synchronization between filesystems, and the management of bucket lifecycle and replication policies. It also supports advanced data retrieval through object searching and SQL-based querying of stored data. The toolkit covers a broad range of administrative an