Python批处理问题和csv中的输出编号

Question

我正在处理一个api调用，它可以工作，但并不完全是我想要的方式。我在代码中添加了batch_idxfor循环，因为我想运行前4个指标，然后将结果写入csv，然后运行后4个指标，然后将它们写入另一个csv，依此类推，直到all_other_indicators列表已用尽。下面的代码输出多个csv，但第一个csv有前4个，但第二个csv包含前8个指标，而不是所需的第二个4，第三个csv包含前12个指标，而不是第三组4，依此类推。如何调整此代码以满足所需的csv输出

import pandas as pd
import requests
import json
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)

url = 'https://something.com/api/v2/intelligence/?username='
username = '<username>'
api_key = '<key>'
WAIT_SECONDS = 5

'''ALL OTHERS'''
all_other_indicators = 
pd.read_csv('Capture_Indicators_Months_Jan_Feb_2019/initial_all-other_indicators.csv')
all_other_indicators = all_other_indicators[0:20]
#all_other_list = all_other_indicators['value'].tolist()
column1 = list()
column2 = list()
column3 = list()
column4 = list()
column5 = list()

i=0
while i < len(all_other_list):
    try:
        for batch_idx in range(5): #how many time it breaks up the data(number of batches)
            current_anomali_indicators = all_other_indicators[batch_idx*4:(batch_idx+1)*4] #number in each batch
            all_other_list = current_anomali_indicators['value'].tolist()

            for row in all_other_list:
                value_api = url+username+'&api_key='+api_key+'&limit=10000&value='
                responses = requests.get(value_api+row, verify=False)
                data = json.loads(responses.text)
                valuess = data['objects']

                for index in valuess:
                    column1.append(index['value'])
                    column2.append(index['source'])
                    column3.append(index['trusted_circle_ids'])
                    column4.append(index['created_ts'])
                    column5.append(index['tags'])
                    all_other__output = pd.DataFrame({'Indicator': column1,'Source': column2,'Trusted Circle': column3,'Created': column4,'Tags': column5})
                    all_other__output.to_csv('Capture_Indicators_Months_Jan_Feb_2019/First_Seen_All_Others_Alone/all_other_with_others_first_seen_'+str(batch_idx)+'.csv', header=True, index=False)
        i += 1
        break
    except requests.exceptions.ConnectionError:
        print("Time to sleep for 5 seconds")
    time.sleep(WAIT_SECONDS)

电流输出： 第一个csv：

Indicator,Source,Trusted Circle,Created,Tags
193.215.253.92,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.252Z,"[{'remote_api': True, 'id': '0ze', 'name': 'IP-used-by-necurs-C', 'resource_uri': None}]"
199.231.105.253,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.299Z,"[{'remote_api': True, 'id': '32b', 'name': 'IP-used-by-necurs-C', 'resource_uri': None}]"
52.210.177.133,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.432Z,"[{'remote_api': True, 'id': 'r5c', 'name': 'IP-used-by-simda-C', 'resource_uri': None}]"
199.167.151.143,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.475Z,"[{'remote_api': True, 'id': 'br3', 'name': 'IP-used-by-suppobox-C', 'resource_uri': None}]"
199.167.151.143,Blocklist Brute Force,[146],2015-01-31T01:16:17.770Z,"[{'remote_api': True, 'name': 'Blocklist-Brute-Force-IPs', 'resource_uri': None}]"

当前输出秒csv：

Indicator,Source,Trusted Circle,Created,Tags
193.215.253.92,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.252Z,"[{'remote_api': True, 'id': '0ze', 'name': 'IP-used-by-necurs-C', 'resource_uri': None}]"
199.231.105.253,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.299Z,"[{'remote_api': True, 'id': '32b', 'name': 'IP-used-by-necurs-C', 'resource_uri': None}]"
52.210.177.133,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.432Z,"[{'remote_api': True, 'id': 'r5c', 'name': 'IP-used-by-simda-C', 'resource_uri': None}]"
199.167.151.143,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.475Z,"[{'remote_api': True, 'id': 'br3', 'name': 'IP-used-by-suppobox-C', 'resource_uri': None}]"
199.167.151.143,Blocklist Brute Force,[146],2015-01-31T01:16:17.770Z,"[{'remote_api': True, 'name': 'Blocklist-Brute-Force-IPs', 'resource_uri': None}]"
88.77.192.110,Snort IP BlockList,[146],2019-01-01T08:53:30.643Z,
88.77.192.110,Talos Intelligence IP Blacklist,[146],2019-01-01T05:38:21.091Z,
88.77.192.110,Blutmagie TOR Nodes,,2019-01-01T00:28:26.492Z,"[{'remote_api': True, 'id': 'v9t', 'name': 'tor', 'resource_uri': None}]"
88.77.192.110,TOR Exit Nodes,[146],2018-12-31T20:43:17.244Z,
195.88.242.31,Botscout BOT IPs,[146],2019-01-01T00:38:25.180Z,
http://hukum.ub.ac.id/Order/Document.zip,URLHaus ,[146],2019-01-22T11:18:30.065Z,
http://hukum.ub.ac.id/Order/Document.zip,URLHaus,,2019-01-01T00:58:59.597Z,"[{'id': '1q5', 'remote_api': True, 'org_id': '2316', 'name': 'URLHaus', 'resource_uri': None}]"
 http://easydown.workday360.cn/pubg/union_plugin_a2af16fdafe50c3f0faecce317c46e57_xzq.exe,URLHaus ,[146],2019-01-22T11:19:17.014Z,

所需输出：第一个csv

193.215.253.92,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.252Z,"[{'remote_api': True, 'id': '0ze', 'name': 'IP-used-by-necurs-C', 'resource_uri': None}]"
199.231.105.253,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.299Z,"[{'remote_api': True, 'id': '32b', 'name': 'IP-used-by-necurs-C', 'resource_uri': None}]"
52.210.177.133,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.432Z,"[{'remote_api': True, 'id': 'r5c', 'name': 'IP-used-by-simda-C', 'resource_uri': None}]"
199.167.151.143,Bambenek Consulting C&C IP Addresses,[146],2019-01-01T00:23:02.475Z,"[{'remote_api': True, 'id': 'br3', 'name': 'IP-used-by-suppobox-C', 'resource_uri': None}]"
199.167.151.143,Blocklist Brute Force,[146],2015-01-31T01:16:17.770Z,"[{'remote_api': True, 'name': 'Blocklist-Brute-Force-IPs', 'resource_uri': None}]"

第二个csv：

88.77.192.110,Snort IP BlockList,[146],2019-01-01T08:53:30.643Z,
88.77.192.110,Talos Intelligence IP Blacklist,[146],2019-01-01T05:38:21.091Z,
88.77.192.110,Blutmagie TOR Nodes,,2019-01-01T00:28:26.492Z,"[{'remote_api': True, 'id': 'v9t', 'name': 'tor', 'resource_uri': None}]"
88.77.192.110,TOR Exit Nodes,[146],2018-12-31T20:43:17.244Z,
195.88.242.31,Botscout BOT IPs,[146],2019-01-01T00:38:25.180Z,
http://hukum.ub.ac.id/Order/Document.zip,URLHaus ,[146],2019-01-22T11:18:30.065Z,
http://hukum.ub.ac.id/Order/Document.zip,URLHaus,,2019-01-01T00:58:59.597Z,"[{'id': '1q5', 'remote_api': True, 'org_id': '2316', 'name': 'URLHaus', 'resource_uri': None}]"
 http://easydown.workday360.cn/pubg/union_plugin_a2af16fdafe50c3f0faecce317c46e57_xzq.exe,URLHaus ,[146],2019-01-22T11:19:17.014Z,

Answer 1

您可以在最外部的范围内在此定义这些列列表（包含值）：

column1 = list()
column2 = list()
column3 = list()
column4 = list()
column5 = list()

稍后，将值附加到这些列表中

                column1.append(index['value'])
                column2.append(index['source'])
                column3.append(index['trusted_circle_ids'])
                column4.append(index['created_ts'])
                column5.append(index['tags'])

处理下一批时，您从未清空这些列表（column1等），它们仍然包含旧条目。您可以通过将定义行移动到all_other_list = current_anomali_indicators['value'].tolist()行的正下方来修复它