无法基于标记值获取S3 bucket名称

Question

在我们的aws帐户中，我们有大约1000个S3 bucket，每个S3 bucket都有对应于应用程序名称的标记（例如，key=application，value=app1）。我试图找出一个特定应用程序拥有多少S3存储桶。因此，首先得到所有S3 bucket的列表；然后遍历该列表以匹配“app1”的标记值。这应该很简单，但由于某些原因，它给出了“AccessDenied when calling GetBucketTagging operation”错误。我验证了我假设的IAM角色对GetBucketTagging有权限

1）获得了使用凭据的s3 bucket的列表（我假设是IAM角色） 2） 遍历列表并尝试匹配标记的key，value对（key=application，value=application1）

第一种选择

import boto3
client = boto3.client('s3')
buckets = client.list_buckets()['Buckets']
matching_buckets = []
# tag key and value to search for
tag_key = 'application'
tag_value = 'app1'
for bucket in buckets:
    tags = client.get_bucket_tagging(Bucket=bucket['Name'])['TagSet']

    for tag in tags:
        if tag['Key'] == tag_key and tag['Value'] == tag_value:
        matching_buckets.append(bucket['Name'])

第二种选择

import boto3
s3 = boto3.client('s3')
app = "app1"
bucketlist = s3.list_buckets()['Buckets']
print(len(bucketlist))
bucketname = []
n=0
#iterate thru the list of {Name, CreationDate} to get all the bucket names and append to empty list

def bucket_tagging_method(b,app):
    mybucketlist = []
    bucket_tagging = s3.get_bucket_tagging(Bucket=b)
    tag_set = bucket_tagging['TagSet']
    for tag in tag_set:
        if (tag['Key'] == "application") and (tag['Value'] == app) :
            mybucketlist.append(b)
            pass
    return(mybucketlist)


while n < len(bucketlist):
    d = bucketlist[n]
    bucketname.append(d['Name'])
    n+=1

for i in bucketname:
    print(bucket_tagging_method(i,app))

它给出了以下错误

tags = client.get_bucket_tagging(Bucket=bucket['Name'])['TagSet']
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetBucketTagging operation: Access Denied

Answer 1

import boto3

boto3.setup_default_session(profile_name='my_profile')
client = boto3.client('s3')
# get list of buckets this iam role can see
buckets = client.list_buckets()['Buckets']

# iterate through list of buckets looking at tags
matching_buckets = []
# tag key and value to search for
tag_key = 'app'
tag_value = 'application1'

for idx, bucket in enumerate(buckets):
#comment out following line if you don't want to see the progress
#print(f'{idx+1} of {len(buckets)} - {bucket["Name"]}')
try:
    tags = client.get_bucket_tagging(Bucket=bucket['Name'])['TagSet']
except client.exceptions.ClientError:
    continue
# iterate through tags looking for specific key
for tag in tags:
    if tag['Key'] == tag_key and tag['Value'] == tag_value:
        matching_buckets.append(bucket['Name'])

print("buckets belonging to", tag_value, "are: ", matching_buckets)