如何将弹性搜索索引的响应保存到s3 bucket中

Question

如何生成低于我的索引

1. 在本地环境中创建环境

2. 使用pip安装必要的库（elasticsearch、requests、requests\u aws4auth、boto3）

3. 使用lambda_function.py在env\Lib\site-packages\内创建文件并添加以下代码

4. 压缩上面的文件夹并将其命名为lambda_function.Zip，然后上传到lambda function中，您可以在其中创建具有必要IAM角色的函数

    import boto3
    from requests_aws4auth import AWS4Auth
    from elasticsearch import Elasticsearch, RequestsHttpConnection
    session = boto3.session.Session()
    credentials = session.get_credentials()
   
    awsauth = AWS4Auth(credentials.access_key,
                       credentials.secret_key,
                       session.region_name, 'es',
                       session_token=credentials.token)
    es = Elasticsearch(
        ['https://search-testelastic-2276kyz2u4l3basec63onfq73a.us-east-1.es.amazonaws.com'],
        http_auth=awsauth,
        use_ssl=True,
        verify_certs=True,
        connection_class=RequestsHttpConnection
    )
   
   
    def lambda_handler(event, context):
        es.cluster.health()
        es.indices.create(index='my-index', ignore=400)
        r = [{'Name': 'Dr. Christopher DeSimone', 'Specialised and Location': 'Health'},
     {'Name': 'Dr. Tajwar Aamir (Aamir)', 'Specialised and Location': 'Health'}]
        for e in enumerate(r):
             es.index(index="my-index", body=e[1])
   

答复如下

{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":3,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"my-index_1","_type":"_doc","_id":"elqrJHMB10jKFvejVaNM","_score":1.0,"_source":{"Name":"Dr. Christopher DeSimone","Specialised and Location":"Health"}},{"_index":"my-index_1","_type":"_doc","_id":"e1qrJHMB10jKFvejVqMK","_score":1.0,"_source":{"Name":"Dr. Tajwar Aamir (Aamir)","Specialised and Location":"Health"}},{"_index":"my-index_1","_type":"_doc","_id":"fFqrJHMB10jKFvejVqMR","_score":1.0,"_source":{"Name":"Dr. Bernard M. Aaron","Specialised and Location":"Health"}}]}}

• 如何在s3 bucket的文件夹中将上述响应保存为json

bucket name=test20220elastic

Answer 1

您可以重用session对象来创建S3资源：

es = ...
s3 = session.resource('s3')
bucket = s3.Bucket('test20220elastic')

def lambda_handler(event, context):
    ...
    for e in enumerate(r):
        result = es.index(index="my-index", body=e[1])
        bucket.put_object(Body=json.dumps(result), Key="my_folder/my_result.json",
                          ContentType="application/json")

不过，您可能需要为每个结果生成不同的键名