我正在尝试使用python在Athena上执行查询
示例代码
client = boto3.client(
'athena',
region_name=region,
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY
)
execution = client.start_query_execution(
QueryString=query,
QueryExecutionContext={
'Database': database
},
WorkGroup=workgroup,
ResultConfiguration={
'OutputLocation': S3_OUTPUT_LOCATION
}
)
这是工作代码,但我得到了一个不寻常的情况
InvalidRequestException: An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: Unable to verify/create output bucket <BUCKET NAME>
我们观察了这一场景几天,每24小时python脚本抛出一次错误,然后在Athena控制台(查询编辑器)上执行查询并重新运行python脚本。 我不明白为什么会这样,是否有任何许可问题
权限:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"athena:GetWorkGroup",
"athena:StartQueryExecution",
"athena:ListDatabases",
"athena:StopQueryExecution",
"athena:GetQueryExecution",
"athena:GetQueryResults",
"athena:GetDatabase",
"athena:GetDataCatalog",
"athena:ListQueryExecutions",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<BUCKET NAME>",
"arn:aws:s3:::<BUCKET NAME>/*",
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"athena:UpdateWorkGroup",
],
"Resource": [
"arn:aws:s3:::<BUCKET NAME>/*",
"arn:aws:s3:::<BUCKET NAME>",
"arn:aws:athena:*:<BUCKET NAME>/<PATH>",
]
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"athena:ListDataCatalogs",
"s3:ListAllMyBuckets"
],
"Resource": "*"
}
]
}
我今天也遇到了同样的错误,发现执行角色还需要s3:GetBucketLocation权限,AWS文档:https://aws.amazon.com/premiumsupport/knowledge-center/athena-output-bucket-error/
我也遇到了同样的问题——随机失败。问题是
s3:GetBucketLocation
策略配置错误。它与其他s3操作捆绑在同一个集群中,其中资源指向s3存储桶,包括路径。它不是这样工作的我修复了它如下,现在工作
见文件:https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-actions.html
相关问题 更多 >
编程相关推荐