如何查看日志文件每行每10秒的解析速度

2024-04-24 22:19:02 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一个日志文件,我希望它被解析每一行,在10秒钟内,我需要知道解析的速度。但是我的代码是解析来自日志文件的数据,所以它只需要日志的数据,而不是日志文件的速度。你知道吗

我的python代码:

import re
import schedule
import threading
import time
log_file = '/Users/kiya/Desktop/mysql/ipscan/ip.txt'
output_file ='/Users/kiya/Desktop/mysql/ipscan/puu.csv'
name_to_check = 'MBX_AUTHENTICATION_FAILED'
start = time.time()
class Log_minning:
    def __init__(self):
        self.data_counter = 0
        self.time_counter = 1
    def get_userdata(self):
        user_att = []
        list_usr = []
        counterr = 0
        with open(log_file) as infile:
            for line in infile:
                if name_to_check in line:
                    date = re.search(r"([12]\d{3}(0[1-9]|1[0-2])+"
                                     "(0[1-9]|[12]\d|3[01]))", line)
                    date = date.group()
                    )
                    ip = ip.group()                        
                    user_att.append(date)                       
                    list_usr.append(user_att)
                    counterr = counterr + 1
                    # print("get " + username + " " + date + " " + time + " "
                    #       + ip)
        self.data_counter = counterr
        return list_usr
    def procces(self):
        self.get_userdata()
    def check_length(self):
        speed = self.data_counter / (self.time_counter * 10)
        print("Speed =" + str(speed) + " data/sec")
        self.time_counter = self.time_counter + 1
    def log_control(self):
        schedule.every(10).seconds.do(self.check_length)

    def schedule_triger(self):
        while True:
            schedule.run_pending()
time.sleep(10)
if __name__ == "__main__":
    lm = Log_minning()
    lm.log_control()
    t1 = threading.Thread(lm.procces())
    t2 = threading.Thread(lm.schedule_triger())
    t1.start()
    t2.start()
    t1.join()
    t2.join()

我的输出是:

Speed =11.7 data/sec
Speed =5.85 data/sec
Speed =3.9 data/sec

这是我期望从代码中得到的结果:

92748.4 line/s
91840.8 line/s
91867.1 line/s
92661.2 line/s   

Tags: importselfiplogdatadatetimedef