使用Python脚本向ES发送数据,在内存中修改json

2024-04-19 09:22:21 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一个python代码,它从SQS队列中提取数据,然后创建一个下载.json使用一个小的.sh脚本索引到ES服务器的文件。在

用于提取SQS数据的Python代码是:

#!/usr/bin/python 

import os
import json
import uuid
import time
import boto.sqs
import boto
from boto.sqs.connection import SQSConnection
from boto.sqs.message import Message
from boto.sqs.message import RawMessage
from ConfigParser import SafeConfigParser

parser = SafeConfigParser()
parser.read('/home/ubuntu/config.ini')

#get details via config file

region = parser.get('default', 'aws_region')
access_key = parser.get('default', 'aws_access_key')
secret_key = parser.get('default', 'aws_secret_key')
queue_name = parser.get('default', 'sqs_queue_name')

sqs = boto.sqs.connect_to_region(region,aws_access_key_id=access_key,aws_secret_access_key=secret_key)
q = sqs.get_queue(queue_name) #SQS queue name

#text_file = open('download.json', 'w')
m = q.read(visibility_timeout=15)
if m == None:
  print "No message!"
else:
  with open('/home/ubuntu/download.json', 'w') as json_data:
    print m.get_body()
    json_data.write(m.get_body())   

    json_data.close()
    q.delete_message(m)
    print "Push To ES"
    os.system('/home/ubuntu/./push_to_ES.sh')
    print "Cleaning the temporary json file"
    os.remove('/home/ubuntu/download.json')
    print "++++++ SUCCESSFUL RUN +++++++"

推到_上海脚本看起来像:

^{pr2}$

我需要去掉这个脚本,只需要一个python脚本就可以了,即使用python脚本向ES发送数据,而不将内容保存到'下载.json'

在下载.json公司名称:

{
    "facter": {
        "blockdevice_xvda_size": 42949672960,
        "blockdevices": "xvda,xvdb",
        "fqdn": null,
        "hardwaremodel": "x86_64",
        "hostname": "qatest",
        "instanceid": "i-4a87",
        "ipaddress": "192.168.00.11",
        "is_virtual": "true",
        "kernelrelease": "3.13.0-48-generic",
        "lsbdistcodename": "trusty",
        "lsbdistdescription": "Ubuntu 14.04.2 LTS",
        "macaddress": "02:9d:c2:c3:200:df",
        "memoryfree": "2.74 GB",
        "memorytotal": "3.68 GB",
        "netmask": "255.255.255.234",
        "operatingsystem": "Ubuntu",
        "operatingsystemrelease": "14.04",
        "processor0": "Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz",
        "processorcount": "1",
        "timezone": "UTC",
        "uniqueid": "007f0100",
        "uptime": "5 days"
    },
    "webapps": {
        "kill_switch": {
            "version": "4.0.671"
        }
    }
}

能做什么??在


Tags: keyfromimport脚本awsjsonparsermessage