s3tools/s3cmd
S3cmd
s3cmd is a command-line client for Amazon S3 and S3-compatible cloud storage services. It provides a unified terminal interface for managing buckets and objects, transferring files, and synchronising directories. All operations are performed over HTTPS with HMAC-SHA256 request signing and TLS encryption to secure data in transit.
The tool supports incremental transfers using S3 ETags, so only new or modified files are sent during sync. Large files are handled with multipart upload chunking, reducing the impact of failures and improving throughput. Directory synchronisation works recursively with checksum-based comparisons, and includes bidirectional sync, one-way mirroring, and incremental updates.
Beyond transfer and sync, s3cmd manages bucket lifecycle rules, configures access controls via ACLs and bucket policies, and handles object listing, copying, deletion, and metadata. The tool is written in Python and operates entirely from the terminal, making it straightforward to integrate into scripts and automation pipelines.
Features
- S3 Protocol Clients - Provides a command-line interface for managing Amazon S3 and compatible cloud storage services.
- Checksum-Based Directory Syncs - The storage CLI synchronizes local directories with remote storage, uploading only changed files for efficient backups.
- Cloud File Uploaders - Uploads local files and directories with recursive, multipart, and incremental options.
- Object Storage - Lists, copies, deletes, and manages metadata and access controls on storage objects.
- Bucket Management - Manages S3 buckets with creation, deletion, listing, access controls, policies, metadata, and usage monitoring.
- Bucket Access Policies - Manages bucket and object permissions through Access Control Lists and bucket policies via API calls.
- Local-to-Cloud File Transfers - Uploads and downloads files to and from S3 with recursive and integrity-verified transfers.
- Bidirectional Cloud Directory Synchronizers - Synchronizes local and remote directories bi-directionally with incremental updates based on checksums.
- Command Line Interfaces - Operates entirely through a terminal command-line interface with flags and subcommands.
- AWS S3 - Provides a full client for the Amazon S3 REST API.
- S3 File Transfer Tools - Transfers files between local systems and S3 storage with upload, download, copy, and multipart support.
- Multipart Uploaders - Provides multipart upload with chunking and resumption for reliable large file transfers to S3.
- Cloud File Downloads - Downloads objects from remote storage with recursive support and integrity verification.
- Incremental S3 ETag Transfers - Uses ETags to incrementally sync only changed files for efficient transfers.
- Directory Structure Mirroring - Mirrors local directories to remote storage, copying only new or changed files.
- Multipart Transfer Utilities - Transfers files using multipart and incremental uploads for efficient large-file handling.
- Datenmanagement - Command-line interface for Amazon S3 storage.
- Infrastrukturmanagement - Full-featured S3 storage client.
- Command Line Tools - Legacy command-line interface for managing cloud storage buckets and objects.
- Networking and Storage - CLI for managing S3 and CloudFront.
- Downloaders - Command-line tool for managing Amazon S3.
