如何使用aws胶水与s3扣合?

2024-04-26 06:21:45 发布

您现在位置:Python中文网/ 问答频道 /正文

我试着在S3上使用AWS胶水进行分区和扣合。但这种扣法不起作用。只有分区有效。我怎样才能用AWS胶水扣?你知道吗

datasink4 = glueContext.write_dynamic_frame.from_options(
    frame = dropnullfields3,
    connection_type = "s3",
    connection_options = {"path": s3_output_full,
                          "partitionKeys": ["PARTITIONKEY"],
                          "bucketColumns": ["ROW_ID"],
                          "numberOfBuckets": 12},
    format = "parquet",
    transformation_ctx = "datasink4")

job.commit()

Tags: fromawss3typedynamicconnectionframewrite

热门问题