从S3存储桶获取所有文件的工具
sucket的Python项目详细描述
糟透了
从S3存储桶下载所有对象的工具。支持按前缀过滤键。在
帮助
-> % sucket --help
Usage: sucket [OPTIONS] BUCKET_NAME [PREFIX]
Download all files from a S3 bucket
Everything from the bucket BUCKET_NAME is downloaded, with an optional key
filter, specified with PREFIX
By default the "folder" mode is used, which will keep the bucket "folder
structure" when downloading.
The "flat" mode will download all objects into the current folder by
adding the folder structure into the key name.
The "keys-only" will completely disregard the folders and put all files in
the current folder.
Options:
-m, --mode [folder|flat|keys-only]
The structure to download the objects in.
-y, --yes Don't prompt for continuing
-q, --quiet Don't print out any info, assumes --yes
-s, --semaphores INTEGER Max number of asynchronous requests to make.
Default: 1000
--help Show this message and exit.
示例
注释
这只在相当有限的一组数据上进行了测试，但效果良好 大约3500个小文件。需要调整和试验更大的文件。在
特别感谢
这主要是基于@steinnes前一段时间编写的一个脚本 很长一段时间以来对我很有用，这个名字完全是他的主意。在
- 项目
标签：
欢迎加入QQ群-->： 979659372
