建立连接日志系统

Question

我正在构建一个“智能”日志系统，在这里我可以监控客户的连接，比如，启动和停止服务器的连接建立时间。你知道吗

原始日志：

Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: terminating... - peer is not responding
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info,account customer1 logged out, 4486 1009521 23444247 12573 18159
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: disconnected
Dec 19 00:00:07 172.16.20.24 pppoe,info PPPoE connection established from 60:E3:27:A2:60:09
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info,account customer2 logged in, 10.171.3.185
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: authenticated
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: connected
Dec 19 00:00:13 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41
Dec 19 00:00:14 172.16.20.24 pppoe,ppp,error <ccfa>: user customer3 authentication failed
Dec 19 00:00:32 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41
Dec 19 00:00:36 172.16.20.24 pppoe,ppp,error <ccfb>: user customer3 authentication failed
Dec 19 00:01:06 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41

对我来说重要的是：用连接的和断开的字符串捕获线。你知道吗

我知道了：

import os
import re
import sys

f = open('log.log','r')
log = []
for line in f:
 if re.search(r': connected|: disconnected',line):
  ob = dict()
  ob['USER'] = re.search(r'<pppoe(.*?)>',line).group(0).replace("<pppoe-","").replace(">","")
  ob['DATA'] = re.search(r'^\w{3} \d{2} \d{2}:\d{2}:\d{2}',line).group(0)
  ob['CONNECTION'] = re.search(r': .*',line).group(0).replace(": ", "")
  log.append(ob)

我还在学习，所以这不是最聪明的正则表达式，但没关系！ 需要现在优化此日志列表，想要获得此示例：

{"connection" : [{
"start" : "Dec 19 10:12:58", 
"username" : "customer2"}

{"connection" : [{
"start" : "Dec 20 10:12:58", 
"username" : "customer1"}

{"connection" : [{
"start" : "Dec 19 10:12:58", 
"stop" : Dec 22 10:04:35",
"username" : "customer4"}

{"connection" : [{
"start" : "Dec 19 10:12:58",
"stop" : "Dec 24 10:04:35" 
"username" : "customer3"}

我的障碍

• 原始日志不断生成，我需要确定 用户已存在。 如果是：更新连接（customer2丢弃他的连接，需要注册它！）但如果他有联系怎么办？你知道吗

例如：

Dec 19 10:20:58 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: disconnected    
Dec 19 01:00:36 172.16.20.24 pppoe,ppp,error <ccfb>: user customer3 authentication failed
Dec 19 01:01:06 172.16.20.24 pppoe,info PPPoE connection established from C0:25:E9:7F:C0:41
Dec 19 10:21:38 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: authenticated
Dec 19 10:21:48 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: connected
Dec 19 10:22:38 172.16.20.24 pppoe,ppp,info <pppoe-customer3>: authenticated
Dec 19 10:22:58 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: disconnected  

首先，简单地添加它。你知道吗

{"connection" : [{
"start" : "Dec 19 10:12:58"
"stop" : "Dec 19 10:20:58", 
"username" : "customer2"}

在下一次身份验证中，我需要搜索这个特定的用户，插入新的“开始”连接时间并删除“停止”。等等。你知道吗

{"connection" : [{
"start" : "Dec 19 10:21:48" 
"username" : "customer2"}

• 我的下一个挑战者，就是创建这个新的精炼列表。你知道吗

我试过了，但没用！你知道吗

conn = []
for l in log:
 obcon = dict()
 if not obcon:
    obcon['USER'] = l['USER']
    if l['DATA'] == 'connected':
        obcon['START'] = l['DATA']      
        obcon['STOP'] = ""
    else:
        obcon['STOP'] = l['DATA']
 conn.append(obcon)

在建立新的名单，我需要检查是否存在一些用户，如果没有，让我们建立它！我用来标识启动/停止连接的['CONNECTION']：

Disconnected -> STOP
Connected -> START

我不知道我是否需要更具体一些。 我需要主意。拜托！你知道吗

Answer 1

在我看来，varlog应该是dict类型，因为它可以帮助您更容易地找到现有的用户数据。
接下来，您在任何地方都使用了re(...).group(0)，这就是entire matching string。例如，提取用户名时，您编写了'<pppoe(.*?)>'，但它位于group(1)（在regex中，括号用于匹配提取）。

我的建议是（注意-我删除了sys和os的导入，因为它们没有被使用）： 你知道吗

import re

f = open('log.log', 'r')
log = dict()
for line in f:
    reg = re.search(r': ((?:dis)?connected)', line) # finds connected or disconnected
    if reg is not None:
        user = re.search(r'<pppoe-(.*?)>', line).group(1)
        # if the user in the log, get it, else create it with empty dict
        ob = log.setdefault(user, dict({'USER': user})) 
        ob['CONNECTION'] = reg.group(1)
        time = re.search(r'^\w{3} \d{2} \d{2}:\d{2}:\d{2}', line).group(0)
        if ob['CONNECTION'].startswith('dis'):
            ob['END'] = time
        else:
            ob['START'] = time
            if 'END' in ob:
                ob.pop('END')

如果日志文件是：

Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: terminating... - peer is not responding
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info,account customer1 logged out, 4486 1009521 23444247 12573 18159
Dec 19 00:00:03 172.16.20.24 pppoe,ppp,info <pppoe-customer1>: disconnected
Dec 19 00:00:07 172.16.20.24 pppoe,info PPPoE connection established from 00:00:00:00:00:00
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info,account customer2 logged in, 127.0.0.1
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: authenticated
Dec 19 00:00:08 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: connected
Dec 19 00:00:13 172.16.20.24 pppoe,info PPPoE connection established from 00:00:00:00:00:00
Dec 19 00:00:14 172.16.20.24 pppoe,ppp,error <ccfa>: user customer3 authentication failed
Dec 19 00:02:03 172.16.20.24 pppoe,ppp,info,account customer2 logged out, 4486 1009521 23444247 12573 18159
Dec 19 00:02:03 172.16.20.24 pppoe,ppp,info <pppoe-customer2>: disconnected
Dec 19 00:02:08 172.16.20.24 pppoe,ppp,info,account customer3 logged in, 127.0.0.1
Dec 19 00:02:08 172.16.20.24 pppoe,ppp,info <pppoe-customer3>: authenticated
Dec 19 00:02:08 172.16.20.24 pppoe,ppp,info <pppoe-customer3>: connected

log的值将是：

{
    'customer1': {
        'CONNECTION': 'disconnected',
        'END': 'Dec 19 00:00:03',
        'USER': 'customer1'
    }, 
    'customer3': {
        'START': 'Dec 19 00:02:08',
        'CONNECTION': 'connected',
        'USER': 'customer3'
    }, 
    'customer2': {
        'START': 'Dec 19 00:00:08',
        'CONNECTION': 'disconnected',
        'END': 'Dec 19 00:02:03', 
        'USER': 'customer2'
    }
}