2 个仓库
Execution of a single operation across millions of objects in a single request.
Distinguishing note: None of the candidates cover the massive-scale batch execution capability of S3.
Explore 2 awesome GitHub repositories matching devops & infrastructure · S3 Batch Operations. Refine with filters or upvote what's useful.
Boto3 is the AWS SDK for Python, providing a programmatic interface for managing and automating AWS cloud infrastructure and services. It serves as a cloud management API client and resource manager for provisioning, configuring, and scaling virtual servers, databases, and storage. The library enables the implementation of infrastructure-as-code through declarative templates and scripts, allowing for the deployment of identical resource stacks across multiple accounts and geographic regions. It also provides a framework for coordinating distributed workflows, serverless functions, and contain
Executes a single operation across millions of objects in a single request for large-scale data management.
s5cmd is a command line interface for high-performance data transfers and management tasks across S3 compatible storage services. It functions as a parallel data transfer tool and bucket synchronization utility, designed to accelerate the uploading and downloading of large volumes of files using concurrent workers. The tool acts as a batch command processor capable of executing thousands of object management operations in parallel from command files or piped streams. It also serves as an S3 Select query client, allowing the execution of SQL expressions against stored JSON records to retrieve
Executes massive-scale batch operations across millions of S3 objects in a single request.