30 open-source projects similar to peak/s5cmd, ranked by how many features they have in common. Compare stars, activity and what each one does to find the best S5cmd alternative.
s3cmd is a command-line client for Amazon S3 and S3-compatible cloud storage services. It provides a unified terminal interface for managing buckets and objects, transferring files, and synchronising directories. All operations are performed over HTTPS with HMAC-SHA256 request signing and TLS encryption to secure data in transit. The tool supports incremental transfers using S3 ETags, so only new or modified files are sent during sync. Large files are handled with multipart upload chunking, reducing the impact of failures and improving throughput. Directory synchronisation works recursively w
minio-go is a client library and software development kit for interacting with S3-compatible object storage. It provides a programmatic interface for Go applications to manage buckets and objects using the S3 protocol. The library enables the execution of complex storage operations, including multi-part uploads for large datasets, data synchronization between filesystems, and the management of bucket lifecycle and replication policies. It also supports advanced data retrieval through object searching and SQL-based querying of stored data. The toolkit covers a broad range of administrative an
The AWS SDK for Java is a set of client libraries providing a programmatic interface for managing cloud resources and services through the Java language and JVM. It serves as a cloud service client library for executing synchronous and asynchronous API calls to infrastructure components. The library is distinguished by its use of non-blocking asynchronous I/O and a reactive cloud client model, utilizing publishers and subscribers to stream data and manage backpressure. It employs a modular design to decouple services and reduce binary size, while utilizing immutable builders for thread-safe c
WinSCP is a remote file manager and SFTP and FTP client that provides a graphical interface for browsing, editing, and organizing files on remote servers and cloud storage. It serves as a multi-protocol tool supporting SFTP, FTP, FTPS, SCP, S3, and WebDAV, and functions as an S3 object storage browser for managing files and buckets. The application distinguishes itself through its automation capabilities, allowing for scripted file transfers and remote command execution via a command-line interface or a .NET assembly. It also features directory synchronization to match local and remote folder
The AWS SDK for PHP is a software development kit that provides HTTP client classes for every supported Amazon Web Service, enabling PHP applications to send authenticated requests and receive structured, typed response objects. It includes a credential resolution chain that automatically locates credentials from environment variables, instance profiles, or configuration files, and supports promise-based asynchronous execution for running multiple API calls concurrently to improve throughput. The SDK distinguishes itself through a middleware pipeline architecture that allows interception and
GAM is a command-line tool for administering Google Workspace and Cloud Identity. It translates command-line arguments into structured API calls, enabling administrators to manage users, groups, organizational units, and domain settings across a Google Workspace environment. The tool handles authentication through OAuth2 flows, service accounts, and workload identity federation, and supports multi-tenant configurations for managing multiple domains or cloud projects from a single installation. GAM distinguishes itself through its batch processing and automation capabilities. It can process la
Webuploader is a browser-based file upload library written in JavaScript. It provides a system for managing multi-file uploads with support for drag-and-drop, clipboard pasting, and type filtering. The library includes a chunked file upload manager that splits large files into smaller segments to improve stability and enable partial retries. It features a client-side image compressor that generates previews and reduces file size before transmission, as well as an MD5 file integrity verifier to identify and skip redundant uploads of files already stored on the server. The tool manages file co
Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac
fsearch is a high-performance desktop file search tool and filesystem indexing engine. It provides near-instant location of files and folders on a local filesystem by utilizing a background indexing system that monitors filesystem changes in real time. The utility distinguishes itself through advanced query capabilities, including support for boolean search logic using AND, OR, and NOT operators, as well as regular expression and wildcard filtering. It allows for precise result refinement using literal character handling and specific search modifiers such as case sensitivity and exact matches
TagSpaces is an offline-first file tagging and organization platform that lets you manage local files with portable metadata stored directly in filenames or sidecar JSON files, eliminating the need for a central database. It functions as a full-text file search engine, a Kanban board file organizer, a local AI file assistant, an S3-compatible cloud file manager, and a web clipper and bookmark manager, all within a single application. The project distinguishes itself through a local-first architecture where all file operations, indexing, and AI processing run entirely on the device, with cloud
Boto3 is the AWS SDK for Python, providing a programmatic interface for managing and automating AWS cloud infrastructure and services. It serves as a cloud management API client and resource manager for provisioning, configuring, and scaling virtual servers, databases, and storage. The library enables the implementation of infrastructure-as-code through declarative templates and scripts, allowing for the deployment of identical resource stacks across multiple accounts and geographic regions. It also provides a framework for coordinating distributed workflows, serverless functions, and contain
Traefik is a cloud-native load balancer and dynamic reverse proxy designed for microservices traffic routing. It automatically discovers services and generates network routes by listening to infrastructure changes in orchestrators and service registries. The project distinguishes itself through auto-configuring service routing, which eliminates manual configuration by updating routing rules in real time as infrastructure scales. It also provides automated SSL certificate management, utilizing ACME-based automation to request and renew certificates from remote authorities. Additional capabili
The AWS Cloud Development Kit is an infrastructure-as-code framework that enables developers to define and provision cloud resources using familiar programming languages. By utilizing construct-based synthesis, it translates high-level, object-oriented code into declarative templates, allowing for the automated management of complex cloud environments through a centralized, code-driven control plane. The framework distinguishes itself through its ability to model infrastructure as a dependency-aware resource graph, ensuring that components are provisioned and updated in the correct order. It
This project is a comprehensive command-line reference and toolkit designed for Linux system administration and network security assessment. It provides a collection of technical snippets and operational guides focused on managing remote environments, orchestrating shell sessions, and executing administrative tasks through native terminal utilities. The repository distinguishes itself by offering specialized techniques for stealthy operations and infrastructure manipulation. It covers methods for establishing encrypted tunnels to bypass firewalls, obfuscating process identities and command hi
lakeFS is a data lake versioning system that provides Git-like branching and commits for large datasets stored in object storage. It functions as a version control layer, enabling the creation of immutable snapshots, atomic commits, and zero-copy branching to create isolated environments for data experimentation without duplicating physical files. The system serves as an S3-compatible storage gateway and an Iceberg REST catalog, allowing standard cloud storage protocols and compatible clients to manage versioned tables. It acts as a data quality gatekeeper by using an event-driven hook system
CubeFS is a distributed cloud storage system designed for managing file and object storage across datacenters and hybrid clouds. It functions as a multi-tenant distributed file system and object store capable of handling data at exabyte scale, utilizing a distributed architecture to store unstructured content. The system is distinguished by a multi-protocol interface layer that allows simultaneous data access through S3, POSIX, and HDFS interfaces. It employs a decoupled compute-storage architecture to scale processing and persistence independently and implements fine-grained isolation polici
SeaweedFS is a distributed object store and high-performance file system designed to manage massive volumes of unstructured data. It utilizes a decoupled architecture that separates metadata management from raw data storage, allowing for independent scalability and the efficient handling of billions of files. By providing a POSIX-compliant interface, it enables applications to interact with a unified namespace while maintaining the performance characteristics of a distributed object store. The system distinguishes itself through a multi-region data fabric that supports active-active replicati
GlusterFS is a software-defined distributed file system and scale-out storage cluster that aggregates disk resources from multiple servers into a single global namespace. It functions as a unified storage platform, allowing the same underlying data to be exposed through file, block, and object storage interfaces. The system distinguishes itself through a decentralized architecture that uses consistent hashing to distribute files across network nodes without a central metadata server. It ensures data integrity and availability using self-healing replication, quorum-based consistency to prevent
This project is a community-curated directory of open-source software designed for deployment in private server environments and home labs. It serves as a comprehensive resource for discovering independent, self-hosted alternatives to mainstream cloud services, enabling users to maintain full data ownership and control over their digital infrastructure. The directory is structured through a hierarchical taxonomy that organizes a vast collection of applications into logical categories, ranging from media management and data analytics to private communication and team productivity tools. It dis
JuiceFS is a distributed file system designed to mount object storage as a local, POSIX-compliant drive. It functions as a cloud-native persistent storage layer that decouples file metadata from raw data, storing metadata in a transactional database while keeping data blocks in object storage. This architecture enables multiple hosts across different regions to access the same storage simultaneously while maintaining strong consistency. The system distinguishes itself by performing data processing, including compression and encryption, directly on the client side before transmission. By split
This project is a JavaScript library and SDK used to integrate web and mobile applications with cloud services. It serves as a bridge to backend providers for user authentication, binary object storage, and real-time data synchronization. The library provides a unified interface for managing cloud identity and access, executing queries and mutations against GraphQL endpoints, and consuming REST APIs with secure request signing. It also includes tools for accessing machine learning services for natural language processing and computer vision. Broad capability areas include offline-first data
Lsyncd is an event-driven directory synchronization daemon that monitors local filesystem changes to mirror data to remote servers. It functions as a customizable sync orchestrator, watching for file creations, modifications, and deletions to trigger automated synchronization tasks or user-defined scripts. The tool specifically automates rsync workflows, utilizing secure shell connections to maintain consistent copies of local data on remote targets. It optimizes data transmission by performing remote file management, such as renames and deletions, via secure shells to avoid unnecessary re-tr
Lefthook is a Git hook manager and workflow automator that uses a configuration-driven approach to manage scripts triggered by version control events. It functions as a parallel task runner, executing multiple scripts simultaneously to reduce the time required for pre-commit or pre-push checks. The project provides a containerized hook runner to ensure consistent dependencies and toolsets across different developer machines. It also supports monorepo orchestration by executing scripts within specific subdirectories and managing complex project structures. The system includes capabilities for
Grequests is an asynchronous HTTP batcher and Gevent-based client library used to execute large sets of network requests simultaneously. It functions as a concurrent request wrapper for the Requests library, enabling non-blocking operations to reduce the total time spent waiting for server responses. The project provides a task-pool execution model to handle batch network operations, such as high-throughput web scraping and API polling. It can stream responses as they arrive via a generator, allowing for immediate data processing without waiting for the entire batch to complete. The library
Garage is a distributed object storage system that provides an S3-compatible API gateway. It is designed to synchronize metadata across distributed nodes using conflict-free replicated data types and Merkle-tree state alignment to maintain cluster-wide consistency. The system ensures data resilience through zone-aware replication, distributing data copies across multiple physical locations. It employs quorum-based request routing and versioned layout management to validate and commit cluster configuration changes. The project covers a broad range of operational capabilities, including automa
This is a high-performance C++ HTTP web server designed for delivering static content. It operates as an asynchronous event loop server that combines IO multiplexing with worker thread pools to handle concurrent requests without blocking. The server specializes in TCP connection management, featuring keep-alive sessions, request pipelining, and timer-based timeout tracking for idle sockets. To ensure high throughput, it utilizes double-buffered asynchronous logging and background threads to prevent disk I/O latency from impacting network traffic. The system covers broad capability areas incl
Mutagen is a cross-platform file synchronization engine and development environment orchestrator. It provides real-time bidirectional file synchronization between local machines, remote servers, and Docker containers, often utilizing SSH for transport. The tool functions as a high-performance alternative to Docker bind mounts by mirroring files between a local host and a running container. It also operates as a network port forwarding utility, routing TCP traffic and Unix domain sockets between local and remote environments via SSH. Its capability surface includes filesystem event monitoring
Lefthook is a Git hook manager and workflow automation tool designed to execute user-defined scripts during version control events. It automates the execution of linters, tests, and formatters to validate changes and maintain code quality before commits or pushes. The project distinguishes itself through a parallel execution model that runs multiple shell commands concurrently to reduce latency. It supports complex configuration management, including the ability to merge global, remote, and local settings and synchronize configurations from external repositories to standardize development env
k3sup is a command-line tool that installs and manages lightweight Kubernetes clusters entirely over SSH, without requiring any pre-installed software on the target machines. It bootstraps clusters on remote hosts using a single statically-linked binary, then provides immediate kubectl access by fetching and merging the cluster's admin credentials into the local configuration. The tool supports plan-driven cluster topology, allowing users to define node roles and configuration in a YAML file for automated multi-node deployment. The tool distinguishes itself by enabling parallel execution of i
Guzzle is a PHP HTTP client used for sending synchronous and asynchronous requests to web services. It serves as a concurrent HTTP request manager, an HTTP stream handler, and a middleware-based HTTP pipeline. The project is a PSR-7 compliant client, utilizing standardized PHP interfaces for requests, responses, and streams. The library differentiates itself through a customizable functional handler stack that allows for the interception and modification of the request and response lifecycle. It features an adapter-based transport system that enables swapping between network implementations,