将一个s3文件夹聚合到另一个
s3clumper的Python项目详细描述
生成的内容的tarball(可以选择使用gzip压缩) S3前缀(保留内部路径和文件名)将 tarball到指定的s3 url并选择性地删除源文件。
s3clumper: Aggregate S3 prefixes into tarballs in S3 Usage: s3clumper {{arguments}} {{options}} Arguments: from [text] S3 URL prefix to clump to [text] S3 URL for target clump Options: -h, --help Show this help message and exit -H, --HELP Help for all sub-commands -c, --check Don't check for target (may over-write) -C, --nocolour Suppress colours in reports -D, --debug Enable debug logging -d, --delete Don't delete source files -q, --quiet Be quiet, be vewy vewy quiet -v, --verbose Verbose output -V, --version Report installed version -z, --compress Don't compress the target
示例:
$ s3clumper s3://bucket1/data/ s3://bucket2/archive/logs-20170302.tar.gz Fetching |################################| 55/55 Sending |################################| 100/100 Deleting |################################| 55/55