我尝试在使用的多处理时创建简单的日志记录多处理管道。其思想是通过每个正在运行的进程将记录的消息放入队列中,并仅通过父进程将这些消息从队列中取出,然后保存到文件中。这样,只有父进程才能访问该文件。虽然这在Ubuntu上正常工作,但在windows10上保存到文件时会丢失一些日志。将日志打印到控制台工作正常,没有日志丢失。我使用多处理日志库。Anaconda上的Python版本3.6.8。你知道吗
我在追踪问题的源头文件处理程序.emit()似乎针对每个日志执行。问题似乎发生在多处理库中。你知道吗
import logging
import multiprocessing
from multiprocessing import freeze_support
import attr
import multiprocessing as mp
import multiprocessing_logging
FORMAT = '[%(asctime)s] [%(levelname)s] ' \
'[%(process)s] [%(process_)s] --- %(message)s'
formatter = logging.Formatter(FORMAT)
fh = logging.FileHandler("test.log")
fh.setFormatter(formatter)
logger = logging.getLogger()
logger.addHandler(fh)
sh = logging.StreamHandler()
sh.setFormatter(formatter)
logger.addHandler(sh)
logger.setLevel(logging.DEBUG)
multiprocessing_logging.install_mp_handler(logger)
@attr.s
class Message:
id = attr.ib()
value = attr.ib(repr=False)
def test_func1(name, in_stream, out_stream):
for a in range(1, 101):
m = Message(a, a)
logging.info(m, extra={'process_': name})
out_stream.send(m)
def test_func2(name, in_stream, out_stream):
while 1:
if in_stream.poll(0.1):
m: Message = in_stream.recv()
m.value = m.value**2
logging.warning(m, extra={'process_': name})
out_stream.send(m)
def test_func3(name, in_stream, out_stream):
while 1:
if in_stream.poll(0.1):
m: Message = in_stream.recv()
m.value = m.value*2
logging.error(m, extra={'process_': name})
services = []
@attr.s
class Service:
name = attr.ib()
in_stream = attr.ib()
out_stream = attr.ib()
func = attr.ib()
def run(self):
worker = mp.Process(
target=self.func,
args=(
self.name,
self.in_stream,
self.out_stream,
),
)
worker.start()
if __name__ == '__main__':
freeze_support()
prev_reader = None
next_reader, next_writer = mp.Pipe()
s1 = Service('f1', prev_reader, next_writer, test_func1)
services.append(s1)
prev_reader = next_reader
next_reader, next_writer = mp.Pipe()
s2 = Service('f22', prev_reader, next_writer, test_func2)
services.append(s2)
prev_reader = next_reader
next_reader, next_writer = mp.Pipe()
s3 = Service('f333', prev_reader, next_writer, test_func3)
services.append(s3)
for s in services:
s.run()
部分日志保存到文件中:
[2019-08-12 10:09:30,317] [ERROR] [10748] [f333] --- Message(id=31)
[2019-08-12 10:09:30,320] [WARNING] [14060] [f22] --- Message(id=40)
[2019-08-12 10:09:30,311] [INFO] [2828] [f1] --- Message(id=53)
2)
[2019-08-12 10:09:30,320] [WARNING] [14060] [f22] --- Message(id=41)
[2019-08-12 10:09:30,311] [INFO] [2828] [f1] --- Message(id=54)
[2019-08-12 10:09:30,320] [WARNING] [14060] [f22] --- Message(id=42)
[2019-08-12 10:09:30,318] [ERROR] [10748] [f333] --- Message(id=33)
控制台输出中日志的相同部分:
[2019-08-12 10:09:30,317] [ERROR] [10748] [f333] --- Message(id=31)
[2019-08-12 10:09:30,318] [WARNING] [14060] [f22] --- Message(id=33)
[2019-08-12 10:09:30,311] [INFO] [2828] [f1] --- Message(id=52)
[2019-08-12 10:09:30,311] [INFO] [2828] [f1] --- Message(id=53)
[2019-08-12 10:09:30,317] [ERROR] [10748] [f333] --- Message(id=32)
[2019-08-12 10:09:30,318] [WARNING] [14060] [f22] --- Message(id=34)
[2019-08-12 10:09:30,311] [INFO] [2828] [f1] --- Message(id=54)
[2019-08-12 10:09:30,318] [ERROR] [10748] [f333] --- Message(id=33)
请告诉我这里有什么问题,是否可以在Windows上使用这种方法,或者我是否应该改变它。你知道吗
目前没有回答
相关问题 更多 >
编程相关推荐