分析大日志文件 - Python

Question

我有一个防火墙日志文件，如下所示：

"No.","Time","Source","Destination","Protocol","Info" "1","0.000000","172.16.113.168","172.16.112.50","TELNET","Telnet Data ..." "2","0.000426","172.16.112.50","172.16.113.168","TELNET","Telnet Data ..." "3","0.019849","172.16.113.168","172.16.112.50","TCP","21582 > telnet [ACK] Seq=2Ack=2 Win=32120 Len=0" "4","0.530125","172.16.113.168","172.16.112.50","TELNET","Telnet Data ..." "5","0.530634","172.16.112.50","172.16.113.168","TELNET","Telnet Data ..." "6","0.549962","172.16.113.168","172.16.112.50","TCP","21582

telnet [ACK] Seq=3 Ack=3 Win=32120 Len=0"

我想能够运行文件的名称（我使用的是Linux）例如

log1.py日志文件.csv（程序名后接日志文件名） 得到以下输出：

$ log1.py logfile.csv Source IP Destination IP Protocol Count

        0.0.0.0     255.255.255.255      BOOTP         20
    0.1.125.174         131.84.1.31        TCP          2
    192.168.1.1         172.168.1.2        TCP        100 

       (............lots more here .....................)

Oracle_89:a5:9f       3com_9c:b2:54        ARP         14

                                        Total:     649787

另一个非常有用的特性是当我用源IP地址和目标IP地址运行程序时。我希望输出类似于以下内容：

$ log1.py 172.16.112.50 logfile.csv

      Source IP      Destination IP   Protocol      Count

  172.16.112.50      135.13.216.191        IMF          4
                                          SMTP         53
                                           TCP         43
                                        TELNET         35
        (............lots more here .....................)

                     172.16.112.194       SMTP          7
                                           TCP         42
                                        TELNET       3745

                                        Total:      38369

最后，我希望能够指定源IP地址和目标IP地址，并获得以下输出：

$ log1.py 172.16.112.50 202.77.162.213 packets.csv Source IP Destination IP Protocol Count

  172.16.112.50      202.77.162.213       ICMP          1
                                       Portmap          5
                                           RSH          9
                                       SADMIND          1
                                           TCP         30
                                        TELNET         41

                                        Total:         87

我是一个初级系统管理员，在编程方面没有太多的经验（只有HTML），我已经开始学习了。但是，在过去的3天里，我一直被这个问题困扰着，这里是到目前为止我所拥有的：

# Function for validating IP address is valid or not 
def ip_validation(ip_address):
    ip_regex= re.match('^[\d]{1,3}[.][\d]{1,3}[.][\d]{1,3}[.][\d]{1,3}$', ip_address)
    return ip_regex
def filereader(file_name):
    file_dump= open(file_name,'r')
    for eachline in file_dump:
        line_a= eachline.replace('\"','') # removes all quotes from the file
        line_b= line_a.split(',') # Delimate each fild based on ','             
        src_ip= line_b[2] # Source IP
        dst_ip= line_b[3] # Destination IP
        prot= line_b[4] # Protocol
        eachline= src_ip, dst_ip, prot      
        itlist.append(eachline) 
        itlist.sort()
        print itlist

Answer 1

解析日志文件并创建列表列表，其中每个子列表包含（源IP、目标IP、协议、计数）。在

现在，您需要在这个外部列表上应用filter函数。 如需进一步澄清，请回复。在